From patchwork Wed Jan 13 05:22:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12015795 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A3C5C433E0 for ; Wed, 13 Jan 2021 05:25:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8529D23117 for ; Wed, 13 Jan 2021 05:25:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8529D23117 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D56D56B00BE; Wed, 13 Jan 2021 00:25:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CDFD98D000F; Wed, 13 Jan 2021 00:25:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF4F18D0007; Wed, 13 Jan 2021 00:25:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0029.hostedemail.com [216.40.44.29]) by kanga.kvack.org (Postfix) with ESMTP id A354C6B00BE for ; Wed, 13 Jan 2021 00:25:37 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 73F0B8245578 for ; Wed, 13 Jan 2021 05:25:37 +0000 (UTC) X-FDA: 77699614314.16.jump91_42012d92751b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id 560DF100E690B for ; Wed, 13 Jan 2021 05:25:37 +0000 (UTC) X-HE-Tag: jump91_42012d92751b X-Filterd-Recvd-Size: 4443 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Wed, 13 Jan 2021 05:25:36 +0000 (UTC) Received: by mail-pl1-f174.google.com with SMTP id x12so417458plr.10 for ; Tue, 12 Jan 2021 21:25:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=YM0v4j/Vw+PE0j0qtyYxkInLgfChRzFtd1IYXVhtaBs=; b=UMBMQFjAJ/KleNdrCLc87puq8f82M8nlP3Nlp9Htg8u8SfL+gDjlPEqAOMUsZRA/w5 QrOVLDLG5hbF1ZaqfQA/mitoaGSUq1xnqtF0D0VRy1Vy6BR6fk1y3FuHmYKD67VJTN12 g4OQbOdhAYdQQKfuUvvVRWQJqUquk/cSSyhf5/RafWb6WWGaLqAORg6SQ1PsTOpioG5A y5xY0rFuv3pZD/P4JCT39VRYlOb4zqh+i/D6m0IV4+hy8VZOnYjanvBnASOIeJggxtxv mKd/C7VH6yzziINWeIMJBpBH6nOU/SS6SL3ee5Rxy0Id2T5C2R1bemu+l06tCvPOo4Ib FqKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=YM0v4j/Vw+PE0j0qtyYxkInLgfChRzFtd1IYXVhtaBs=; b=nyNtVFE2Ey6x1PwL5ovM1ayos2kHyTQ8XfMf5zQt+E8duE90wAuI0vUwrifw55hi4C PP+s9k0cG0PiXwiBhlMlms5QECCGxipDNdGpqY1Zkph72X3E3lwmEsMVmVQiTpW4dlol nTNDnhb4oJg9UDCMmSJbmJbyLQCW0HXJ3elZ9IcxOGRw0QCZ5EZE99Rg7vsU3B+eCCfV YX3qIxADFGJWWwqU8jTDrfU70R5ECAJTkhBEePIXnOgfvCKxMZZoVGrCRwLyTej4l7bs OQuOL2P6GpfvK5bs/70FayFK41wpvnQ86R6azp188wuHGXEz9gPd5XKFOy0VUBIHDhoH vkMg== X-Gm-Message-State: AOAM533T9mu0rDi2RvG++aHBZU4zr6y789lkzfemmoMlRdqKKFmL5Hy4 bU3fJy1e3b1v0YpFpv4EZ0uEag== X-Google-Smtp-Source: ABdhPJxHkGLvnYePvtJVAinVs5t2rPLGqeF5cswjO6y60ppYk2XM0JruFQuehDmZQ5o6gWa3rkMO2w== X-Received: by 2002:a17:902:848e:b029:dc:b38:98f0 with SMTP id c14-20020a170902848eb02900dc0b3898f0mr622492plo.82.1610515535725; Tue, 12 Jan 2021 21:25:35 -0800 (PST) Received: from localhost.localdomain ([139.177.225.247]) by smtp.gmail.com with ESMTPSA id x15sm893081pfa.80.2021.01.12.21.25.31 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Jan 2021 21:25:35 -0800 (PST) From: Muchun Song To: mike.kravetz@oracle.com, akpm@linux-foundation.org Cc: n-horiguchi@ah.jp.nec.com, ak@linux.intel.com, mhocko@suse.cz, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Yang Shi , Michal Hocko Subject: [PATCH v4 1/6] mm: migrate: do not migrate HugeTLB page whose refcount is one Date: Wed, 13 Jan 2021 13:22:04 +0800 Message-Id: <20210113052209.75531-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210113052209.75531-1-songmuchun@bytedance.com> References: <20210113052209.75531-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: All pages isolated for the migration have an elevated reference count and therefore seeing a reference count equal to 1 means that the last user of the page has dropped the reference and the page has became unused and there doesn't make much sense to migrate it anymore. This has been done for regular pages and this patch does the same for hugetlb pages. Although the likelyhood of the race is rather small for hugetlb pages it makes sense the two code paths in sync. Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz Acked-by: Yang Shi Acked-by: Michal Hocko Reviewed-by: David Hildenbrand Reviewed-by: Oscar Salvador --- mm/migrate.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/mm/migrate.c b/mm/migrate.c index 4385f2fb5d18..a6631c4eb6a6 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1279,6 +1279,12 @@ static int unmap_and_move_huge_page(new_page_t get_new_page, return -ENOSYS; } + if (page_count(hpage) == 1) { + /* page was freed from under us. So we are done. */ + putback_active_hugepage(hpage); + return MIGRATEPAGE_SUCCESS; + } + new_hpage = get_new_page(hpage, private); if (!new_hpage) return -ENOMEM; From patchwork Wed Jan 13 05:22:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12015797 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43435C433E0 for ; Wed, 13 Jan 2021 05:25:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A42D422B37 for ; Wed, 13 Jan 2021 05:25:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A42D422B37 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2F4DC8D000F; Wed, 13 Jan 2021 00:25:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2CB788D0007; Wed, 13 Jan 2021 00:25:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1BB008D000F; Wed, 13 Jan 2021 00:25:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0144.hostedemail.com [216.40.44.144]) by kanga.kvack.org (Postfix) with ESMTP id 044AF8D0007 for ; Wed, 13 Jan 2021 00:25:42 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id C0E961EE6 for ; Wed, 13 Jan 2021 05:25:41 +0000 (UTC) X-FDA: 77699614482.08.actor44_5804ebb2751b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin08.hostedemail.com (Postfix) with ESMTP id A7D111819E772 for ; Wed, 13 Jan 2021 05:25:41 +0000 (UTC) X-HE-Tag: actor44_5804ebb2751b X-Filterd-Recvd-Size: 5317 Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Wed, 13 Jan 2021 05:25:40 +0000 (UTC) Received: by mail-pj1-f54.google.com with SMTP id b5so445712pjl.0 for ; Tue, 12 Jan 2021 21:25:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RQxBzrH/aoWEdG3IjoZ+7PAOMjNKjJoK1+V0rHsE7hs=; b=squ9zmJAhHIGqokb1c1CENeZTx1PBd+M1MWOPjBpY2CspZ7CWbwStQxbPHEkzGDfVw r3GllIzxRSrFSki7Hs7vOhtOSTmatJJybzgdTqoXbnFCV327hVJLS2GNi3Kv1waVcmoM nfuiNcJICOfTESCwE6oXVXep6SiDdNj1CtPav0peKhrYdFCbKaLoqflQDZrBEgbRmG7d gauTrw3fsWXuPN0zQhnuAexRRfzsZPDkbYM9nuIVz52PFAAiHIZBxJ2nMWJ2Raiggkq8 wD08ZfI/qBKg8Jf7EEcspHVrWm+4x2U+v9XYP1386+QaQ0WyKvEtmA4zU/v803Lz/YZG NPNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RQxBzrH/aoWEdG3IjoZ+7PAOMjNKjJoK1+V0rHsE7hs=; b=MnEb1hxy3cbxm4Xr8AmD4OwgnWS54sdfO7tWTMEafBaSEpr/dfJAneMdgTYzrQe5bb 0QgzD4zMXBQ/v/a2eWYdBL87HeJ3roH8mY2dmc9TVlr+2ljTw/CjivJuaqtuAmBby7cn V7CR28BoMoNlLVlnaajns5KHStlhIeihzZ6GwCXW9BbvxWFzETLuClhHeQd0I4G2nVM+ BLZ6qc5RNw01DF8kL/oW6KOd1vlnbdPcSKYBs4kxJO7CIQVQFKtUCi8/tRvsIjvKiCVQ g10/4LThUbR/xSTNptOzahpYTxAUL+DD6HPgaTfnMF5FF6nM7lGHVdxbP5Wl6EvhWGqO Snyg== X-Gm-Message-State: AOAM532Ix4mXQOZqk5m9dl8nzx/nSnKCRizdvXhtYMAVFJ/5cCh7Bjlt wuCFP/MzoHkq28bPkObJgmEVyg== X-Google-Smtp-Source: ABdhPJwg4fxKWfXnYhu4nWUKg61Xld9nJTPPDMfQW0uTj27Vgs5KMWDaoF8RkSjjah+MGrBywkQArQ== X-Received: by 2002:a17:90a:68ce:: with SMTP id q14mr350311pjj.223.1610515539949; Tue, 12 Jan 2021 21:25:39 -0800 (PST) Received: from localhost.localdomain ([139.177.225.247]) by smtp.gmail.com with ESMTPSA id x15sm893081pfa.80.2021.01.12.21.25.36 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Jan 2021 21:25:39 -0800 (PST) From: Muchun Song To: mike.kravetz@oracle.com, akpm@linux-foundation.org Cc: n-horiguchi@ah.jp.nec.com, ak@linux.intel.com, mhocko@suse.cz, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Michal Hocko , stable@vger.kernel.org Subject: [PATCH v4 2/6] mm: hugetlbfs: fix cannot migrate the fallocated HugeTLB page Date: Wed, 13 Jan 2021 13:22:05 +0800 Message-Id: <20210113052209.75531-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210113052209.75531-1-songmuchun@bytedance.com> References: <20210113052209.75531-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If a new hugetlb page is allocated during fallocate it will not be marked as active (set_page_huge_active) which will result in a later isolate_huge_page failure when the page migration code would like to move that page. Such a failure would be unexpected and wrong. Only export set_page_huge_active, just leave clear_page_huge_active as static. Because there are no external users. Fixes: 70c3547e36f5 (hugetlbfs: add hugetlbfs_fallocate()) Signed-off-by: Muchun Song Acked-by: Michal Hocko Reviewed-by: Mike Kravetz Cc: stable@vger.kernel.org Reviewed-by: Oscar Salvador --- fs/hugetlbfs/inode.c | 3 ++- include/linux/hugetlb.h | 2 ++ mm/hugetlb.c | 2 +- 3 files changed, 5 insertions(+), 2 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index b5c109703daa..21c20fd5f9ee 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -735,9 +735,10 @@ static long hugetlbfs_fallocate(struct file *file, int mode, loff_t offset, mutex_unlock(&hugetlb_fault_mutex_table[hash]); + set_page_huge_active(page); /* * unlock_page because locked by add_to_page_cache() - * page_put due to reference from alloc_huge_page() + * put_page() due to reference from alloc_huge_page() */ unlock_page(page); put_page(page); diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index ebca2ef02212..b5807f23caf8 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -770,6 +770,8 @@ static inline void huge_ptep_modify_prot_commit(struct vm_area_struct *vma, } #endif +void set_page_huge_active(struct page *page); + #else /* CONFIG_HUGETLB_PAGE */ struct hstate {}; diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 1f3bf1710b66..4741d60f8955 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1348,7 +1348,7 @@ bool page_huge_active(struct page *page) } /* never called for tail page */ -static void set_page_huge_active(struct page *page) +void set_page_huge_active(struct page *page) { VM_BUG_ON_PAGE(!PageHeadHuge(page), page); SetPagePrivate(&page[1]); From patchwork Wed Jan 13 05:22:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12015799 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94470C433DB for ; Wed, 13 Jan 2021 05:25:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 213E422B37 for ; Wed, 13 Jan 2021 05:25:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 213E422B37 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9AA3A8D0010; Wed, 13 Jan 2021 00:25:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 95B828D0007; Wed, 13 Jan 2021 00:25:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 84A5F8D0010; Wed, 13 Jan 2021 00:25:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0037.hostedemail.com [216.40.44.37]) by kanga.kvack.org (Postfix) with ESMTP id 6ECE18D0007 for ; Wed, 13 Jan 2021 00:25:46 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 32829181AEF30 for ; Wed, 13 Jan 2021 05:25:46 +0000 (UTC) X-FDA: 77699614692.24.tail57_3f116bf2751b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id 1AF1F1A4A5 for ; Wed, 13 Jan 2021 05:25:46 +0000 (UTC) X-HE-Tag: tail57_3f116bf2751b X-Filterd-Recvd-Size: 6438 Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Wed, 13 Jan 2021 05:25:45 +0000 (UTC) Received: by mail-pg1-f176.google.com with SMTP id i5so781598pgo.1 for ; Tue, 12 Jan 2021 21:25:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JlQZQVrOm402ulHFjYHU+1nImQj/z6C/J2SHh6gZPrk=; b=ujayAIQH7e5D07vHiW8ktnM5b+1Sj0HSbBPecs9gbudPs6Gv3oWDdnyOFdH38A5eoV EvMMnU9FgwoYk1vghm+TAafxj45NdEG7v65EG2XTFOjwtTS0fqGmuBykLlZaXAmKluNq gLMMRGSswPNoNA1IdD7Pv/aImOdpFtgTu3s3qFXKokNIhsjDi1M9+a3FpM5na3PC32MA OWVoLNsEucG1t72DMPtUPB0B3iB6gVheQ9RDr5qs6UtF9qCNLDoq+76i+JTJfpYh+b4l /3B0SQyesOGSFdCS6lThNm4lVo+G8umqvRHU8MfQtA/ZlQg4tSBHd1CJyL9K5o9IqtQN zECg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JlQZQVrOm402ulHFjYHU+1nImQj/z6C/J2SHh6gZPrk=; b=O+bFFCPcIhXZfSbAfNN6yayGif4+aW/0Wj2uQL/OXmGDWmrT7NApsSfXmdLei6WUaD ZNg01xFP5m8ny2G27/8JybfCLsihjX4FFKZ78BWy4L9vajvvExSFRzBwh42OMZIB2Ylw SwF2s9+S+0yKh8yLgwoTEzvjKlFT2klBEYFK7s3pL9yjAoiitcKZzbMZVH9KTZr6JjKZ SqfJDZPkx7lu0ITJIEONsoWiH4ECiwmZ3Pe3p/c842Z5f1+7Wz2GGzdiqcSGpq8Aa9mk LjhftMTAKewbuAtj5EFexU0S+owDZEeazhk0GrfqcDQFUPoqF04IrkfK7G2K/+7tKNtq 3kqQ== X-Gm-Message-State: AOAM533AIFXPCcKaNcbcg48Z8old68oNIDtJz5LW0hfO2kMm7DvJDase 2GPZxx8dkM9L911UKscO2Nkilw== X-Google-Smtp-Source: ABdhPJxPLRQfjsTnvt/HmlGnGJNCjHsOsPGMcbaQEqmWlQVyc40MLv2lELMjR5jMm2y+VTHq3wbtcQ== X-Received: by 2002:aa7:843a:0:b029:19d:b279:73c9 with SMTP id q26-20020aa7843a0000b029019db27973c9mr447437pfn.3.1610515544150; Tue, 12 Jan 2021 21:25:44 -0800 (PST) Received: from localhost.localdomain ([139.177.225.247]) by smtp.gmail.com with ESMTPSA id x15sm893081pfa.80.2021.01.12.21.25.40 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Jan 2021 21:25:43 -0800 (PST) From: Muchun Song To: mike.kravetz@oracle.com, akpm@linux-foundation.org Cc: n-horiguchi@ah.jp.nec.com, ak@linux.intel.com, mhocko@suse.cz, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , stable@vger.kernel.org Subject: [PATCH v4 3/6] mm: hugetlb: fix a race between freeing and dissolving the page Date: Wed, 13 Jan 2021 13:22:06 +0800 Message-Id: <20210113052209.75531-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210113052209.75531-1-songmuchun@bytedance.com> References: <20210113052209.75531-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There is a race condition between __free_huge_page() and dissolve_free_huge_page(). CPU0: CPU1: // page_count(page) == 1 put_page(page) __free_huge_page(page) dissolve_free_huge_page(page) spin_lock(&hugetlb_lock) // PageHuge(page) && !page_count(page) update_and_free_page(page) // page is freed to the buddy spin_unlock(&hugetlb_lock) spin_lock(&hugetlb_lock) clear_page_huge_active(page) enqueue_huge_page(page) // It is wrong, the page is already freed spin_unlock(&hugetlb_lock) The race windows is between put_page() and dissolve_free_huge_page(). We should make sure that the page is already on the free list when it is dissolved. Fixes: c8721bbbdd36 ("mm: memory-hotplug: enable memory hotplug to handle hugepage") Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz Cc: stable@vger.kernel.org --- mm/hugetlb.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 4741d60f8955..4a9011e12175 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -79,6 +79,21 @@ DEFINE_SPINLOCK(hugetlb_lock); static int num_fault_mutexes; struct mutex *hugetlb_fault_mutex_table ____cacheline_aligned_in_smp; +static inline bool PageHugeFreed(struct page *head) +{ + return page_private(head + 4) == -1UL; +} + +static inline void SetPageHugeFreed(struct page *head) +{ + set_page_private(head + 4, -1UL); +} + +static inline void ClearPageHugeFreed(struct page *head) +{ + set_page_private(head + 4, 0); +} + /* Forward declaration */ static int hugetlb_acct_memory(struct hstate *h, long delta); @@ -1028,6 +1043,7 @@ static void enqueue_huge_page(struct hstate *h, struct page *page) list_move(&page->lru, &h->hugepage_freelists[nid]); h->free_huge_pages++; h->free_huge_pages_node[nid]++; + SetPageHugeFreed(page); } static struct page *dequeue_huge_page_node_exact(struct hstate *h, int nid) @@ -1044,6 +1060,7 @@ static struct page *dequeue_huge_page_node_exact(struct hstate *h, int nid) list_move(&page->lru, &h->hugepage_activelist); set_page_refcounted(page); + ClearPageHugeFreed(page); h->free_huge_pages--; h->free_huge_pages_node[nid]--; return page; @@ -1504,6 +1521,7 @@ static void prep_new_huge_page(struct hstate *h, struct page *page, int nid) spin_lock(&hugetlb_lock); h->nr_huge_pages++; h->nr_huge_pages_node[nid]++; + ClearPageHugeFreed(page); spin_unlock(&hugetlb_lock); } @@ -1770,6 +1788,14 @@ int dissolve_free_huge_page(struct page *page) int nid = page_to_nid(head); if (h->free_huge_pages - h->resv_huge_pages == 0) goto out; + + /* + * We should make sure that the page is already on the free list + * when it is dissolved. + */ + if (unlikely(!PageHugeFreed(head))) + goto out; + /* * Move PageHWPoison flag from head page to the raw error page, * which makes any subpages rather than the error page reusable. From patchwork Wed Jan 13 05:22:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12015801 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABA5AC433E6 for ; Wed, 13 Jan 2021 05:25:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0D96523120 for ; Wed, 13 Jan 2021 05:25:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0D96523120 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8C0C78D0011; Wed, 13 Jan 2021 00:25:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8988E8D0007; Wed, 13 Jan 2021 00:25:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 786208D0011; Wed, 13 Jan 2021 00:25:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0049.hostedemail.com [216.40.44.49]) by kanga.kvack.org (Postfix) with ESMTP id 63D028D0007 for ; Wed, 13 Jan 2021 00:25:51 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 2F169181AEF30 for ; Wed, 13 Jan 2021 05:25:51 +0000 (UTC) X-FDA: 77699614902.21.bears36_0b021792751b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id 0FAF3180442C7 for ; Wed, 13 Jan 2021 05:25:51 +0000 (UTC) X-HE-Tag: bears36_0b021792751b X-Filterd-Recvd-Size: 4813 Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Wed, 13 Jan 2021 05:25:50 +0000 (UTC) Received: by mail-pj1-f44.google.com with SMTP id md11so419281pjb.0 for ; Tue, 12 Jan 2021 21:25:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=df+OSfZ6Luc35Oz4+SS0Ts5oHiC9zu8mWZ+1lvg6xFY=; b=wBRMJy+2MZVtfgvXnwW63ZbQjCCWkgb2dOMUynxH9HAzryqJzMWO8hX3hUebpvrjPo 46M+sxORf5F6Z9VDNvKGgxBsXVzdJCpJYijbRpEPg9GwZjjo0hbNKzIqT10ABwAcz9jr ybrk4dJHUZr8fk+ZzwAbiMXkwOB97p4K4FIgDk6YJCnScgJMrRMl53HLrCUM2wIIiq88 YsEMOjimlLmRxhj+6UTzRrq8sdEbJThESelLVqg+a3pAp+1TB3PLfvuBP7Kyu2UsJcOv d5b8KYUgqK6GJgTBDmXCHG1edYjugzk/IP061E9C2RfgM/4GpT2x9V83wtXV20FY39QG uvSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=df+OSfZ6Luc35Oz4+SS0Ts5oHiC9zu8mWZ+1lvg6xFY=; b=BsJ1Ouf8+xerw8vz+++LKPbgQ3YNb+Yqr9Q2SAjlookgT89Hvb8g+QmkpA+bSrVtHW 1WMu47BoNuv9U3g4huFoJ+VAC/Mdw2umVwzqUj5mjyAeR2MpgE50gdlzOmazCsZE7beO 2i6nHFsZSakwG2N1AHpovC+yevM+pCKjZpF/mCY7b+NrmlpEvsFoUcksR/cZVqorgg9B PMTl0GfImLcditX1h5RvxAMBs6hSA+UF04n1WIU/bvkdUm5SjppFBJ3wU6QVPEE5ZSuA 8ynUM3ySJrCYZDSYtMy3dk60w/hV8GFi3CQOkgKFdqRtFx2wHJZbI6RiApiNQAH0i+HM 5KuQ== X-Gm-Message-State: AOAM530vk1gcaAt79v5kYTOwp2ZprCg/7lpxsti/OOXapQmU915fL8rD 5Puw4ra/sVjftS6djA8e4yUABA== X-Google-Smtp-Source: ABdhPJyKl1cvK6NkpSmYZQknrW7MZ4B+txOHVft4CSpdorPAXL1OO4sVsWZKeOvhYqOuCHE1W/qd3g== X-Received: by 2002:a17:90a:4208:: with SMTP id o8mr386094pjg.224.1610515548247; Tue, 12 Jan 2021 21:25:48 -0800 (PST) Received: from localhost.localdomain ([139.177.225.247]) by smtp.gmail.com with ESMTPSA id x15sm893081pfa.80.2021.01.12.21.25.44 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Jan 2021 21:25:47 -0800 (PST) From: Muchun Song To: mike.kravetz@oracle.com, akpm@linux-foundation.org Cc: n-horiguchi@ah.jp.nec.com, ak@linux.intel.com, mhocko@suse.cz, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song Subject: [PATCH v4 4/6] mm: hugetlb: retry dissolve page when hitting race Date: Wed, 13 Jan 2021 13:22:07 +0800 Message-Id: <20210113052209.75531-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210113052209.75531-1-songmuchun@bytedance.com> References: <20210113052209.75531-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There is a race between dissolve_free_huge_page() and put_page(). Theoretically, we should return -EBUSY when we encounter this race. In fact, we have a chance to successfully dissolve the page if we do a retry. Because the race window is quite small. If we seize this opportunity, it is an optimization for increasing the success rate of dissolving page. Signed-off-by: Muchun Song --- mm/hugetlb.c | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 4a9011e12175..898e4ea43e13 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1772,6 +1772,7 @@ int dissolve_free_huge_page(struct page *page) { int rc = -EBUSY; +retry: /* Not to disrupt normal path by vainly holding hugetlb_lock */ if (!PageHuge(page)) return 0; @@ -1793,8 +1794,23 @@ int dissolve_free_huge_page(struct page *page) * We should make sure that the page is already on the free list * when it is dissolved. */ - if (unlikely(!PageHugeFreed(head))) - goto out; + if (unlikely(!PageHugeFreed(head))) { + spin_unlock(&hugetlb_lock); + + /* + * Theoretically, we should return -EBUSY when we + * encounter this race. In fact, we have a chance + * to successfully dissolve the page if we do a + * retry. Because the race window is quite small. + * If we seize this opportunity, it is an optimization + * for increasing the success rate of dissolving page. + */ + while (PageHeadHuge(head) && !PageHugeFreed(head)) { + cond_resched(); + cpu_relax(); + } + goto retry; + } /* * Move PageHWPoison flag from head page to the raw error page, From patchwork Wed Jan 13 05:22:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12015803 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BAA0EC433E0 for ; Wed, 13 Jan 2021 05:25:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3B1D822B37 for ; Wed, 13 Jan 2021 05:25:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3B1D822B37 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BF43D8D0012; Wed, 13 Jan 2021 00:25:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BA41E8D0007; Wed, 13 Jan 2021 00:25:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ABBAD8D0012; Wed, 13 Jan 2021 00:25:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0143.hostedemail.com [216.40.44.143]) by kanga.kvack.org (Postfix) with ESMTP id 958B38D0007 for ; Wed, 13 Jan 2021 00:25:54 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 5FF3C180AD81F for ; Wed, 13 Jan 2021 05:25:54 +0000 (UTC) X-FDA: 77699615028.02.soup93_4d15a672751b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin02.hostedemail.com (Postfix) with ESMTP id 486FC10097AA0 for ; Wed, 13 Jan 2021 05:25:54 +0000 (UTC) X-HE-Tag: soup93_4d15a672751b X-Filterd-Recvd-Size: 5119 Received: from mail-pg1-f182.google.com (mail-pg1-f182.google.com [209.85.215.182]) by imf23.hostedemail.com (Postfix) with ESMTP for ; Wed, 13 Jan 2021 05:25:53 +0000 (UTC) Received: by mail-pg1-f182.google.com with SMTP id n7so779727pgg.2 for ; Tue, 12 Jan 2021 21:25:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7IILliy6HAELOAf2o6Y4OfnqFvSJBzlwsRQjJedxot0=; b=dJSU9wJz/3rQWPdMgFSeLg9iISXgA+mrG+M6gt+bsIjssGtFTfPEsERPEUS/5t83/a LUUhWWNg/MtzqsN1Kcpq3X30J5yof6IfmScrbd2duBHq6YKTHipysgbq0GgU5xk6pS2e zjsxPKhbFQ3aQpWwXgTAKahmN+gYTipJrmzzMWPn1qre9ExAnmti5MKJc035HZOxbRYj iHy8ulllb0ZqY2Um8SLjP+3fu+8cZOBSSgRpFiASO+QylIjnslUNIoap6l6hYXze4wgD w4WSsWwgioG3c1Cy6a1OJW3JNITIUff9+LHkCpsmdjVxblAQrvZnTkHGQvdlc1TgFK6d 1KbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7IILliy6HAELOAf2o6Y4OfnqFvSJBzlwsRQjJedxot0=; b=QcJkFuPNrhMNiLT4JhmJyOLpjXYV7GS4ocMR9xo/gc3lnPVWw74q20ZTHE//vXGIlT k8eiOQbZXtA9j0odckIIfqBN4O/sDjvvusL5Js8G+lXtyN5Vri9CjhwwpyCGLri6yBEZ GuB65AtUpMVfNdlkQAIjWCWASTTWPlsGDxQyfY9SFpujWxCi02Erg5WTfGg1df4U81kY l1ttsyH7PDq/SaZrkHVGIjjQy+qkdjFzqT6qH69g2cKkvrwIhST5yodC9rxHCADBtVMZ 5+LKEUVZ4tNLXjHUYoIXKb+iuEmyLAsfRKRPwqzF4Qwo3fiSxSzeo0F3Khc1uR/jXFyb PLwA== X-Gm-Message-State: AOAM533nTR1ix2kx5NURSCjf/hDnP2QxCBG3uf4kQ5qmUokz7ZD8ZS/D 6lQmCaYqrmnYfMwFIxIXFd6G4A== X-Google-Smtp-Source: ABdhPJyzBXCuQ9SD6Bb7oZ1eb4Ev3kY+H2KEd84rW3RSKql2lLp9DD8eqP0VXQA4l2FJ/CVALwQ2ZQ== X-Received: by 2002:a63:1110:: with SMTP id g16mr459913pgl.357.1610515552956; Tue, 12 Jan 2021 21:25:52 -0800 (PST) Received: from localhost.localdomain ([139.177.225.247]) by smtp.gmail.com with ESMTPSA id x15sm893081pfa.80.2021.01.12.21.25.48 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Jan 2021 21:25:52 -0800 (PST) From: Muchun Song To: mike.kravetz@oracle.com, akpm@linux-foundation.org Cc: n-horiguchi@ah.jp.nec.com, ak@linux.intel.com, mhocko@suse.cz, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Michal Hocko , stable@vger.kernel.org Subject: [PATCH v4 5/6] mm: hugetlb: fix a race between isolating and freeing page Date: Wed, 13 Jan 2021 13:22:08 +0800 Message-Id: <20210113052209.75531-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210113052209.75531-1-songmuchun@bytedance.com> References: <20210113052209.75531-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There is a race between isolate_huge_page() and __free_huge_page(). CPU0: CPU1: if (PageHuge(page)) put_page(page) __free_huge_page(page) spin_lock(&hugetlb_lock) update_and_free_page(page) set_compound_page_dtor(page, NULL_COMPOUND_DTOR) spin_unlock(&hugetlb_lock) isolate_huge_page(page) // trigger BUG_ON VM_BUG_ON_PAGE(!PageHead(page), page) spin_lock(&hugetlb_lock) page_huge_active(page) // trigger BUG_ON VM_BUG_ON_PAGE(!PageHuge(page), page) spin_unlock(&hugetlb_lock) When we isolate a HugeTLB page on CPU0. Meanwhile, we free it to the buddy allocator on CPU1. Then, we can trigger a BUG_ON on CPU0. Because it is already freed to the buddy allocator. Fixes: c8721bbbdd36 ("mm: memory-hotplug: enable memory hotplug to handle hugepage") Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz Acked-by: Michal Hocko Cc: stable@vger.kernel.org --- mm/hugetlb.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 898e4ea43e13..a7ed22811672 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5575,9 +5575,9 @@ bool isolate_huge_page(struct page *page, struct list_head *list) { bool ret = true; - VM_BUG_ON_PAGE(!PageHead(page), page); spin_lock(&hugetlb_lock); - if (!page_huge_active(page) || !get_page_unless_zero(page)) { + if (!PageHeadHuge(page) || !page_huge_active(page) || + !get_page_unless_zero(page)) { ret = false; goto unlock; } From patchwork Wed Jan 13 05:22:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12015805 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE1A3C433E6 for ; Wed, 13 Jan 2021 05:25:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6D23022B37 for ; Wed, 13 Jan 2021 05:25:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6D23022B37 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F28DE8D0013; Wed, 13 Jan 2021 00:25:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EFFC78D0007; Wed, 13 Jan 2021 00:25:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E15338D0013; Wed, 13 Jan 2021 00:25:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0055.hostedemail.com [216.40.44.55]) by kanga.kvack.org (Postfix) with ESMTP id CBBC48D0007 for ; Wed, 13 Jan 2021 00:25:58 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 9A8688245571 for ; Wed, 13 Jan 2021 05:25:58 +0000 (UTC) X-FDA: 77699615196.26.view13_4c0d2ec2751b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin26.hostedemail.com (Postfix) with ESMTP id 7FE091804B65C for ; Wed, 13 Jan 2021 05:25:58 +0000 (UTC) X-HE-Tag: view13_4c0d2ec2751b X-Filterd-Recvd-Size: 4282 Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Wed, 13 Jan 2021 05:25:57 +0000 (UTC) Received: by mail-pj1-f43.google.com with SMTP id j13so439201pjz.3 for ; Tue, 12 Jan 2021 21:25:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=t1Dj1BnCFYHHEbXxpzLfGjPuUeLA5YyfxkXrd8ZYcB0=; b=CkA9nt2K6f+62+T51zdVrYDbufK9iilFrW0pvsAhNTKoNhtmriES3uoKM1sUakUZZ5 F/v6UK2sK8O39JyDaqatm8tzyyn0AJoI7H+IDejD4uJmwSTg6K9E1yy89mdOJnCEYjcW k69OZYyJnR0MBsFEE1C9ZZVCWBqjSkk7uH/vBI3RCr0KaxPha+Iow4xZr7VKSwS52VRL s8JtrAKwI/5cHrpqLI4hhvXK/oKOErZGwvmjzdqAbBNT+xCp3nPOswceWh3laeh0nZ89 bkTQXsBeOJe6rDwxwZjLMtfYp2T+RluwSz5yFLGdXoqWtHGonU/iHNrM5mdXNrjzpb0c DcaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=t1Dj1BnCFYHHEbXxpzLfGjPuUeLA5YyfxkXrd8ZYcB0=; b=c+NQLxiqvTtSG87YZfzaDza1HlLhgU1Iv+7YqsM7mw8USilcWxineP64ohPOcfaNTA SFjySkayyxZIISZk81AXKsEYdE155JtEL1vXXDiCrHJL2rFKq+slx6pXYaW9GfbEYU7J /faC8TWNidYxKsKjgrDmG13Q9wemiXgHVUyHGQ1bh3XWt9Q6sbpG+Hz2Bqjwr8JqkNCt x23Xvcm9Gp1neNCi4DsAsTmVwQsn1sJqlb4A/tufYHkBNiJ333WGA7JhgV3qIcus0wS6 Nd2mvM1Bg7Adt/TcQzH3Honh5OQz8u+BIzCOTZUJPu+fqwxD/Cf+uZ0HHlpttqvwXpAq 2jIQ== X-Gm-Message-State: AOAM531xDEijr1ZrlF8p/DHnAgaqh1IC5SByRoqV/3lpIFXlq9nVbct/ sdHxot0QwmYXXKsGubhLeoNMNQ== X-Google-Smtp-Source: ABdhPJy6z9arm0rLeW481RViIUW3vRvWLSVKc/kfyfaVXhVXVXDdIrIVxIDcum+7oWvZIYQX1oAJ8Q== X-Received: by 2002:a17:90b:3011:: with SMTP id hg17mr424575pjb.22.1610515557266; Tue, 12 Jan 2021 21:25:57 -0800 (PST) Received: from localhost.localdomain ([139.177.225.247]) by smtp.gmail.com with ESMTPSA id x15sm893081pfa.80.2021.01.12.21.25.53 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Jan 2021 21:25:56 -0800 (PST) From: Muchun Song To: mike.kravetz@oracle.com, akpm@linux-foundation.org Cc: n-horiguchi@ah.jp.nec.com, ak@linux.intel.com, mhocko@suse.cz, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song , Michal Hocko , stable@vger.kernel.org Subject: [PATCH v4 6/6] mm: hugetlb: remove VM_BUG_ON_PAGE from page_huge_active Date: Wed, 13 Jan 2021 13:22:09 +0800 Message-Id: <20210113052209.75531-7-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210113052209.75531-1-songmuchun@bytedance.com> References: <20210113052209.75531-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The page_huge_active() can be called from scan_movable_pages() which do not hold a reference count to the HugeTLB page. So when we call page_huge_active() from scan_movable_pages(), the HugeTLB page can be freed parallel. Then we will trigger a BUG_ON which is in the page_huge_active() when CONFIG_DEBUG_VM is enabled. Just remove the VM_BUG_ON_PAGE. Fixes: 7e1f049efb86 ("mm: hugetlb: cleanup using paeg_huge_active()") Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz Acked-by: Michal Hocko Cc: stable@vger.kernel.org --- mm/hugetlb.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index a7ed22811672..8c6005a538a2 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1360,8 +1360,7 @@ struct hstate *size_to_hstate(unsigned long size) */ bool page_huge_active(struct page *page) { - VM_BUG_ON_PAGE(!PageHuge(page), page); - return PageHead(page) && PagePrivate(&page[1]); + return PageHeadHuge(page) && PagePrivate(&page[1]); } /* never called for tail page */