From patchwork Mon Jan 31 16:02:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12730830 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE1E5C433FE for ; Mon, 31 Jan 2022 16:04:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2163C6B00C8; Mon, 31 Jan 2022 11:04:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 19BCD8D0001; Mon, 31 Jan 2022 11:04:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 088AA6B00CA; Mon, 31 Jan 2022 11:04:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay028.a.hostedemail.com [64.99.140.28]) by kanga.kvack.org (Postfix) with ESMTP id DFF2B6B00C8 for ; Mon, 31 Jan 2022 11:04:28 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 4106B60D22 for ; Mon, 31 Jan 2022 16:04:27 +0000 (UTC) X-FDA: 79091054574.03.EADEC34 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf04.hostedemail.com (Postfix) with ESMTP id B419D40041 for ; Mon, 31 Jan 2022 16:04:07 +0000 (UTC) Received: by mail-pl1-f171.google.com with SMTP id z5so12718248plg.8 for ; Mon, 31 Jan 2022 08:04:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3P2GhQ/A4jZ2vY4MofmwB/at+DQmKpJSp18e9dRfwe4=; b=5URQXSWgeqQ7mckfU4C9lY/dFwxZNhRwVpHK8PF66Zds0N3U5nsrMNbagre+kYdnct +7o8S/vRwiYT6Zw3piBgSLEXOeagAuCSSDt5bYZ8PiiQ1WiJOzuLDBf9kGZ2xrU0DFCI Bj4Qz09SgZvWc8iA9F5vFDGmETHPkYnDxkCsLa8nhEgbbBi5icdAE1eLXYLBh8ZXnLJ1 MjO8vyzTyf6I6AAfmVEXkPHMyN5hET/0/8R3vtzopEPjnTYAD6MjTm+ZVCnuHucvkzT9 R2JPwM4NUEFLkb5Pavb8ubGKAM7YnFbERMPEt5PE/HE28UvrKKlT8px8uUdqmOm8J+sK zXFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3P2GhQ/A4jZ2vY4MofmwB/at+DQmKpJSp18e9dRfwe4=; b=V2CqICQ+izDczDo7mL5+9MVBOQ6vzRKvJlFrglkE+yO4YlHjjtXgcyPtkIjl3/k4N8 66+YiNmAXNPrfhNPVoy++aNFE36ZTQ5Shloc4oJiSd0hZCsCA5oTPjMOYOq5mwnpBwL6 gpRzYes+WSw2pyt+t6dtLTlNnsBicqfWcOLzsrQVa6izYV7g1dkOW8Ftmsjd9Yd04G1H ZFA2fv70vfcgd9c5TvoO1U3MorHUNdw0c0MJHnUjkBYlSMhrzu0znlvHrq9Kx2LFEiZK TZvtyeO9gvDawlQUUTqWxTwDwTczwNqMx4FYfpDkoedvf6tqDWFGHm8ymvAzXBuQkJdP DJqA== X-Gm-Message-State: AOAM530rapYimK5mAoA0/8Rl3TF1G+lYjj6yyIQazXXRk/BpxmwFnY4h xuRxcoNIB7FqRQ/f9MB2MC9oyQ== X-Google-Smtp-Source: ABdhPJx+s3Ymv7b02lJg4vX/CGJvKlKxbU1nqbdkqUMa4X7q/ZoBsG0DOzq5OgBQhaDtJj5SKTVUgQ== X-Received: by 2002:a17:902:ea06:: with SMTP id s6mr20895751plg.161.1643645045966; Mon, 31 Jan 2022 08:04:05 -0800 (PST) Received: from FVFYT0MHHV2J.tiktokcdn.com ([139.177.225.244]) by smtp.gmail.com with ESMTPSA id n42sm17940716pfv.29.2022.01.31.08.04.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Jan 2022 08:04:05 -0800 (PST) From: Muchun Song To: akpm@linux-foundation.org, zi.yan@cs.rutgers.edu, kirill.shutemov@linux.intel.com, rientjes@google.com, lars.persson@axis.com, mike.kravetz@oracle.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, Muchun Song , Zi Yan Subject: [PATCH v3 1/5] mm: thp: fix wrong cache flush in remove_migration_pmd() Date: Tue, 1 Feb 2022 00:02:50 +0800 Message-Id: <20220131160254.43211-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220131160254.43211-1-songmuchun@bytedance.com> References: <20220131160254.43211-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: B419D40041 X-Stat-Signature: nsyz4ccghc8rcq5oi7ajgb668efesjxh Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=5URQXSWg; dmarc=pass (policy=none) header.from=bytedance.com; spf=pass (imf04.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com X-Rspam-User: nil X-HE-Tag: 1643645047-882696 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The flush_cache_page() is supposed to be justified only if the page is already placed in process page table, and that is done right after flush_cache_page(). So using this interface is wrong. And there is no need to invalite cache since it was non-present before in remove_migration_pmd(). So just to remove it. Signed-off-by: Muchun Song Reviewed-by: Zi Yan --- mm/huge_memory.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f58524394dc1..45ede45b11f5 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3207,7 +3207,6 @@ void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new) if (pmd_swp_uffd_wp(*pvmw->pmd)) pmde = pmd_wrprotect(pmd_mkuffd_wp(pmde)); - flush_cache_range(vma, mmun_start, mmun_start + HPAGE_PMD_SIZE); if (PageAnon(new)) page_add_anon_rmap(new, vma, mmun_start, true); else @@ -3215,6 +3214,8 @@ void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new) set_pmd_at(mm, mmun_start, pvmw->pmd, pmde); if ((vma->vm_flags & VM_LOCKED) && !PageDoubleMap(new)) mlock_vma_page(new); + + /* No need to invalidate - it was non-present before */ update_mmu_cache_pmd(vma, address, pvmw->pmd); } #endif From patchwork Mon Jan 31 16:02:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12730826 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 650DFC433F5 for ; Mon, 31 Jan 2022 16:04:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE6E66B00BF; Mon, 31 Jan 2022 11:04:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C97246B00C1; Mon, 31 Jan 2022 11:04:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B5F288D0001; Mon, 31 Jan 2022 11:04:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0057.hostedemail.com [216.40.44.57]) by kanga.kvack.org (Postfix) with ESMTP id A7C256B00BF for ; Mon, 31 Jan 2022 11:04:12 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 66DB0822FC03 for ; Mon, 31 Jan 2022 16:04:12 +0000 (UTC) X-FDA: 79091053944.20.2A36341 Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) by imf12.hostedemail.com (Postfix) with ESMTP id 057DA40005 for ; Mon, 31 Jan 2022 16:04:11 +0000 (UTC) Received: by mail-pj1-f44.google.com with SMTP id g11-20020a17090a7d0b00b001b2c12c7273so158543pjl.0 for ; Mon, 31 Jan 2022 08:04:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iQH5onptg6s877J45y8Fks8F6bmcOMzezi8xj98jFE4=; b=EastST5ZIcLSESMsp5gzxuTqcCSNgh5+VOlWEXH6hIDH/Y+6Yi35lcJ+cnIcbEKd7m ys4Lc9gV7R1lfGNhqxJc+0ZWjEjrfRAh03XbwNQJ53zVIlzmP/zQztsQ3zILrR4XJDYD rdDAzYkP/5wsiH5bx7EUldm57nS/S/pVfNhfrTxu0WxmmGgw9Ah3xuHFcL6fj+0NqSGV ZkvPlQ/0AC4dX++DpJqmn/6luHcfdGPKTHwysX/OSTavRqfsce1g9gsT8z2fFQGeXmvM CrHvh4ovORh0oE2mshQS/a7a4kDp+bt1wsKPtyuevVZy/noM5LRw2Sk8WpFt2yMPhny8 JFnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iQH5onptg6s877J45y8Fks8F6bmcOMzezi8xj98jFE4=; b=TIAGXB8xfe0B1HkdZwhNFT7NQcrnYcjfTGGBcL5Xf2rEElG6Fw9TkbCJYWVsjIat+5 KRlHWa29RV1PTfnMWfImbJBGwcuISAHHwVlYwWjtpBNGbJC/SyVwLQb3hkKrhnh6jyt2 A7e1qEdwVpnEcA/V3tZhhrtmJvA1Wi7CK1QeK8YDMtXVp9whFQdXRxE+PEPXyv3r0xlY cpp5K/N3Jdvhb89snP9K9NS2l8YqJIB0CLDdUgI1hDB3C1Q/qwndZoOzkH5x4d250CCB xuZIhsJUfbE6RYtGJltxgALp/FdT9DOyCdS6dUEN7x90x3B9Qg2XXglycMz+w43LVeIs 2pnQ== X-Gm-Message-State: AOAM5334hXheYVrJFfM8UC8/RJIp2MHz9jZZ3y73HLuaUV8U7UhuyU8N /xLIiaEoW/yzN60ZdZGRyaaq+Q== X-Google-Smtp-Source: ABdhPJyKqjzk6JU+7cZI+lIZKz8qJV0tOrsdnXMJaAquGWsMKIiUnglaRbBHuWSf5yL2XPDGOcbabQ== X-Received: by 2002:a17:90b:1952:: with SMTP id nk18mr34913304pjb.101.1643645051035; Mon, 31 Jan 2022 08:04:11 -0800 (PST) Received: from FVFYT0MHHV2J.tiktokcdn.com ([139.177.225.244]) by smtp.gmail.com with ESMTPSA id n42sm17940716pfv.29.2022.01.31.08.04.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Jan 2022 08:04:10 -0800 (PST) From: Muchun Song To: akpm@linux-foundation.org, zi.yan@cs.rutgers.edu, kirill.shutemov@linux.intel.com, rientjes@google.com, lars.persson@axis.com, mike.kravetz@oracle.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, Muchun Song , Zi Yan Subject: [PATCH v3 2/5] mm: fix missing cache flush for all tail pages of compound page Date: Tue, 1 Feb 2022 00:02:51 +0800 Message-Id: <20220131160254.43211-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220131160254.43211-1-songmuchun@bytedance.com> References: <20220131160254.43211-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Stat-Signature: 8yowqq7mmkpdpfb9g58dr9wsdquycca6 X-Rspam-User: nil Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=EastST5Z; spf=pass (imf12.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.216.44 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 057DA40005 X-HE-Tag: 1643645051-600394 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The D-cache maintenance inside move_to_new_page() only consider one page, there is still D-cache maintenance issue for tail pages of compound page (e.g. THP or HugeTLB). THP migration is only enabled on x86_64, ARM64 and powerpc, while powerpc and arm64 need to maintain the consistency between I-Cache and D-Cache, which depends on flush_dcache_page() to maintain the consistency between I-Cache and D-Cache. In theory, the issue can be found on arm64 and powerpc. HugeTLB migration is enabled on arm, arm64, mips, parisc, powerpc, riscv, s390 and sh, while arm has handled the compound page cache flush in flush_dcache_page(), but most others do not. In theory, the issue exists on many architectures. Fix this by not using flush_dcache_folio() since it is not backportable. Fixes: 616b8371539a ("mm: thp: enable thp migration in generic path") Fixes: 290408d4a250 ("hugetlb: hugepage migration core") Signed-off-by: Muchun Song Reviewed-by: Zi Yan --- mm/migrate.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index c9296d63878d..c418e8d92b9c 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -933,9 +933,12 @@ static int move_to_new_page(struct page *newpage, struct page *page, if (!PageMappingFlags(page)) page->mapping = NULL; - if (likely(!is_zone_device_page(newpage))) - flush_dcache_page(newpage); + if (likely(!is_zone_device_page(newpage))) { + int i, nr = compound_nr(newpage); + for (i = 0; i < nr; i++) + flush_dcache_page(newpage + i); + } } out: return rc; From patchwork Mon Jan 31 16:02:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12730827 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D6BBC433FE for ; Mon, 31 Jan 2022 16:04:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D8EF66B00C3; Mon, 31 Jan 2022 11:04:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D37D36B00C5; Mon, 31 Jan 2022 11:04:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B37D88D0001; Mon, 31 Jan 2022 11:04:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0194.hostedemail.com [216.40.44.194]) by kanga.kvack.org (Postfix) with ESMTP id 903016B00C3 for ; Mon, 31 Jan 2022 11:04:22 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 4EAE792DEC for ; Mon, 31 Jan 2022 16:04:22 +0000 (UTC) X-FDA: 79091054364.10.AF29CC7 Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) by imf01.hostedemail.com (Postfix) with ESMTP id AA40F40026 for ; Mon, 31 Jan 2022 16:04:17 +0000 (UTC) Received: by mail-pj1-f43.google.com with SMTP id b1-20020a17090a990100b001b14bd47532so14329324pjp.0 for ; Mon, 31 Jan 2022 08:04:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cll0HiNGaOM9yDWI60AkgZZ8Vcgf+tRA3oZc4UBL7nk=; b=kqcwDry5uO65hxuBb6yCzUOH/a1vV6vXhHnghWYmZE0wFwKVg4sDSVN/IFdkti1vxa gH7qY1qLoeC+cKiFHLrznIF/YZZj/5AhGagvceyAcyEcWWJMLsMqThARBDdu+bmGE+PX x6nXbH2i4BHnV1zU3rPxoNPPkey5wfOwbpO2l++PZiNesvMkQiFad2sph+apeltel7aQ Ls1SJTkFiJ4YqR9pdlN4r7l0tC31BgRrY8AyIlvoJPiUU/g7awJtPbivAom+ouD7h5rh 4fLe2DLqgbdp4M3crNsgRcakjECmczJ67k4TbCKpIhxWYZao52P3BXYCUZuUbN4s2YZS rVFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cll0HiNGaOM9yDWI60AkgZZ8Vcgf+tRA3oZc4UBL7nk=; b=2YX1nUu2ihRDqJclUY/jg9g8gr3dGDfgYORAgCFnNr9zT2tYKyzxMzcQ+76P4GD+Bx UrRAUVE9+1Y3G20/nlHM6z3994O5NurhvsZZB0sQXpeM8i0afPeGLE6VGsr2z++DUv3y JusRaXwxad7zBRv1S2lspgqR1SgA4vOLUD3zzbjplxRm3g+S5Gezn04THmDw693wRzZj BZgK5a/HFJhd04ake7liMtUV1e/JmCWbSmFwOjMDQKoi6ndrgii76+JTu0PMp+ZCtZPe Ptxsp+nYYv1ColLaoGyrhv94F8sFfMdufiYWUNz+FmiS0gbl4UtsoB22qpgI3PVb9dZf kQcQ== X-Gm-Message-State: AOAM532AXJ9qVzXi3S3QG+CygJoyQxt8fcoQioS1zPL89SiwyrQ/hBks +AU824zy8Jo8MrHZjWDEcuX5Cw== X-Google-Smtp-Source: ABdhPJxZJ+tEgViLwu3F1PuG3ikPoO/2NiO4jJZJUhCFbhH2t+GN/XLwxc/o0L5yrU9nizfHfRLOTw== X-Received: by 2002:a17:902:e788:: with SMTP id cp8mr21004336plb.172.1643645055966; Mon, 31 Jan 2022 08:04:15 -0800 (PST) Received: from FVFYT0MHHV2J.tiktokcdn.com ([139.177.225.244]) by smtp.gmail.com with ESMTPSA id n42sm17940716pfv.29.2022.01.31.08.04.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Jan 2022 08:04:15 -0800 (PST) From: Muchun Song To: akpm@linux-foundation.org, zi.yan@cs.rutgers.edu, kirill.shutemov@linux.intel.com, rientjes@google.com, lars.persson@axis.com, mike.kravetz@oracle.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, Muchun Song Subject: [PATCH v3 3/5] mm: hugetlb: fix missing cache flush in copy_huge_page_from_user() Date: Tue, 1 Feb 2022 00:02:52 +0800 Message-Id: <20220131160254.43211-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220131160254.43211-1-songmuchun@bytedance.com> References: <20220131160254.43211-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: AA40F40026 X-Rspam-User: nil Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=kqcwDry5; dmarc=pass (policy=none) header.from=bytedance.com; spf=pass (imf01.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.216.43 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com X-Stat-Signature: xyjakwjipjixetqmnjm1xkh1jkyun5zo X-Rspamd-Server: rspam08 X-HE-Tag: 1643645057-473753 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The userfaultfd calls copy_huge_page_from_user() which does not do any cache flushing for the target page. Then the target page will be mapped to the user space with a different address (user address), which might have an alias issue with the kernel address used to copy the data from the user to. Fix this issue by flushing dcache in copy_huge_page_from_user(). Fixes: fa4d75c1de13 ("userfaultfd: hugetlbfs: add copy_huge_page_from_user for hugetlb userfaultfd support") Signed-off-by: Muchun Song --- mm/memory.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/memory.c b/mm/memory.c index e8ce066be5f2..eb027da68aa7 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5405,6 +5405,8 @@ long copy_huge_page_from_user(struct page *dst_page, if (rc) break; + flush_dcache_page(subpage); + cond_resched(); } return ret_val; From patchwork Mon Jan 31 16:02:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12730828 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4C66C4332F for ; Mon, 31 Jan 2022 16:04:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 016086B00C1; Mon, 31 Jan 2022 11:04:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F07996B00C3; Mon, 31 Jan 2022 11:04:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DCFB38D0001; Mon, 31 Jan 2022 11:04:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0104.hostedemail.com [216.40.44.104]) by kanga.kvack.org (Postfix) with ESMTP id CDDD76B00C1 for ; Mon, 31 Jan 2022 11:04:21 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 8F9151816C8DC for ; Mon, 31 Jan 2022 16:04:21 +0000 (UTC) X-FDA: 79091054322.19.1F137ED Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf26.hostedemail.com (Postfix) with ESMTP id 1C48E140008 for ; Mon, 31 Jan 2022 16:04:20 +0000 (UTC) Received: by mail-pl1-f176.google.com with SMTP id j16so12747156plx.4 for ; Mon, 31 Jan 2022 08:04:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pht+u875/nHbiyfmnh2sHRDlYx7sxAKFTWEwLAeL/xQ=; b=asLzFLOtnhAt1wJ9MkFxljPA6I+O9Q3lLsoNk3YTh57vC01yj731U/WEvoQVHChNPv YmPjjWrQVJpd+K5pDqdHk7/Ga+moURiIO0F5YaZp2iEYDbWl0QhBcftDlCDJhERdof// PW6WNKKdyMCNG87pMWXQV+Aj5f9tKti/Uvg/D9Wcjem4fI1UyGOCMPBdSppQhkr2Ans4 8DD/GaoXJkl+gHf4awk75ojdsvXeezRplz7byycXAAk4SvhrjmCccOlB4PEkINoxtgCS pcbaOJ6CDCAdwhYM/2vh/kAdKvw3Di51GKUxoUc0E6zOEe5QRkyjCHdmC4Ro5wuy+yXh 4Bhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pht+u875/nHbiyfmnh2sHRDlYx7sxAKFTWEwLAeL/xQ=; b=txZGuhHOaipe/XyjYA17eb+SdzTIi3P1O2GgrmuODqh5durZcnTbwDhkTQ4KTt9QKm XalXgl/D3gwzScCxAsQMvPW649qBHmJ7XDZFfhoi74hE+6Yt4oX2sefbr6CaCq78YgI2 sKYyI8Xd4OyMD+D0gOqHQyytM0YkFg3O0iuz56cVtsmogchXjJE8kABVVyBvkmlbgNAo AEtMBC34kozcX6T2OfPpj4YJSq1fZb9EvJSY/tU1OugZkIZF0KEyH8N52RhV6fujv9GH GNkZDg9WUNkFoo1pATvJItBrU58rYdebK1glPn4q3Gt/tyqt3IWjEliqIgsAj8+ifv6X Vtyw== X-Gm-Message-State: AOAM530BRWTJ3b+3i90eiG892wOCQ9LOR6z1wtsjlkkkdkUWxVIED72L BK0tRhwKUocW8zGYnGHuYlCVww== X-Google-Smtp-Source: ABdhPJxtq4AiRBDI6E+iaufhjEeklExYl8CNg78gPPyn/nVhJ3VlwPpiHVau25255hprz6v62IPy1w== X-Received: by 2002:a17:902:8213:: with SMTP id x19mr18163640pln.150.1643645060156; Mon, 31 Jan 2022 08:04:20 -0800 (PST) Received: from FVFYT0MHHV2J.tiktokcdn.com ([139.177.225.244]) by smtp.gmail.com with ESMTPSA id n42sm17940716pfv.29.2022.01.31.08.04.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Jan 2022 08:04:19 -0800 (PST) From: Muchun Song To: akpm@linux-foundation.org, zi.yan@cs.rutgers.edu, kirill.shutemov@linux.intel.com, rientjes@google.com, lars.persson@axis.com, mike.kravetz@oracle.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, Muchun Song Subject: [PATCH v3 4/5] mm: hugetlb: fix missing cache flush in hugetlb_mcopy_atomic_pte() Date: Tue, 1 Feb 2022 00:02:53 +0800 Message-Id: <20220131160254.43211-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220131160254.43211-1-songmuchun@bytedance.com> References: <20220131160254.43211-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 1C48E140008 X-Stat-Signature: yrzz5m3pmfyzqr4bc7ir11mdqecm5pj4 X-Rspam-User: nil Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=asLzFLOt; spf=pass (imf26.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-HE-Tag: 1643645060-808722 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: folio_copy() will copy the data from one page to the target page, then the target page will be mapped to the user space address, which might have an alias issue with the kernel address used to copy the data from the page to. Fix this issue by flushing dcache but not use flush_dcache_folio() since it is not backportable. Fixes: 8cc5fcbb5be8 ("mm, hugetlb: fix racy resv_huge_pages underflow on UFFDIO_COPY") Signed-off-by: Muchun Song --- mm/hugetlb.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index a1baa198519a..f1f1ab31dc8a 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5804,6 +5804,8 @@ int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, goto out; } } else { + int i, nr; + if (vm_shared && hugetlbfs_pagecache_present(h, dst_vma, dst_addr)) { put_page(*pagep); @@ -5819,6 +5821,9 @@ int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, goto out; } folio_copy(page_folio(page), page_folio(*pagep)); + nr = compound_nr(page); + for (i = 0; i < nr; i++) + flush_dcache_page(page + i); put_page(*pagep); *pagep = NULL; } From patchwork Mon Jan 31 16:02:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12730829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5930C433F5 for ; Mon, 31 Jan 2022 16:04:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 19C636B00C5; Mon, 31 Jan 2022 11:04:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 17DF88D0001; Mon, 31 Jan 2022 11:04:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 03B236B00C8; Mon, 31 Jan 2022 11:04:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0072.hostedemail.com [216.40.44.72]) by kanga.kvack.org (Postfix) with ESMTP id E848E6B00C5 for ; Mon, 31 Jan 2022 11:04:27 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id A457318216AE7 for ; Mon, 31 Jan 2022 16:04:27 +0000 (UTC) X-FDA: 79091054574.07.84A0309 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf18.hostedemail.com (Postfix) with ESMTP id 081631C000F for ; Mon, 31 Jan 2022 16:04:25 +0000 (UTC) Received: by mail-pj1-f52.google.com with SMTP id qe6-20020a17090b4f8600b001b7aaad65b9so8165427pjb.2 for ; Mon, 31 Jan 2022 08:04:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gisX+SmHlGPaod6BmlULNl1QqGpeThEpw2gv3Ed6fPs=; b=5Kju0dWpSHwXhYEWvUiml8KB/0ZMWI8SxY+Qz6NYPYq89/Rcn6kjtmlsEGtAmVwJoV 7UHjH692hPcUl3swiilP52F9+cY+ioBqdsilrfv8fyAr3nyRomOfaJ0DgIobtvOYXmHa xLVYYwa1pYNhqIDLyrj0GVbQc089q2E3TI/YbhKZr3Nuv66N5mnCSnrH1ZEbKxM/KuF/ 8m3792ODuO8Q2a+VlrTtC4n0FkTsRVP7BJ8Fy1Xj7Z0KimoGrsiJHh9CIAD4fClr/mi0 a7ZMNNBfQr2jcIC63VZUkaEmE2I8HeG3ec7oLHCYwwu/Srwvs59VWyFub+A/ec54/yny 4qxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gisX+SmHlGPaod6BmlULNl1QqGpeThEpw2gv3Ed6fPs=; b=H+bVR7D/3Ww5ogWWWCID3utMduQ8BKqfRncMX2nkgJknhhPk9vD8GNF+kJPO7lFHli 98wKFTKys+DySMeZr+oBfD4gTbWDTkainHvQfutC6Px+fmHQLIBmtyhAmeWot5mNCPTw Wo4gqbO/b62XTc+qKw/n7aPtmjc1hxV9pg+Ejb+xGz/soFaVFOUXE0u02z+V8aQngZxj TB/C1abUGeVS/yJVEyehVrADr4afDYb6SrLnPcSK5Net7CXDBCvj1vkShYokA9fKgyiC h86150VfP7xgczo7vRdhRpQNnvYA7xy34iKFgV30sOuMIfieUneuPfkwKyCbijPhKFrD QCcw== X-Gm-Message-State: AOAM5336F2m+FolDy5w9EcFlHbA9MzR4P0DfXSq6KrJ01YJfHWHiosC8 swqJYsEPwaKR1hoPggRIDaqcZg== X-Google-Smtp-Source: ABdhPJwnOHSIH/tVeMiDIsL5x0suj1BcB23ljOBPvIUrK1xrla1SShg+QsGbaZcLU6k68TnfcGxnWQ== X-Received: by 2002:a17:902:d2cf:: with SMTP id n15mr20697725plc.33.1643645065059; Mon, 31 Jan 2022 08:04:25 -0800 (PST) Received: from FVFYT0MHHV2J.tiktokcdn.com ([139.177.225.244]) by smtp.gmail.com with ESMTPSA id n42sm17940716pfv.29.2022.01.31.08.04.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 31 Jan 2022 08:04:24 -0800 (PST) From: Muchun Song To: akpm@linux-foundation.org, zi.yan@cs.rutgers.edu, kirill.shutemov@linux.intel.com, rientjes@google.com, lars.persson@axis.com, mike.kravetz@oracle.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, Muchun Song Subject: [PATCH v3 5/5] mm: replace multiple dcache flush with flush_dcache_folio() Date: Tue, 1 Feb 2022 00:02:54 +0800 Message-Id: <20220131160254.43211-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220131160254.43211-1-songmuchun@bytedance.com> References: <20220131160254.43211-1-songmuchun@bytedance.com> MIME-Version: 1.0 Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=5Kju0dWp; spf=pass (imf18.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.216.52 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-Rspam-User: nil X-Rspamd-Queue-Id: 081631C000F X-Stat-Signature: 1fwoc3tk9uqbdn1ocss6bhj37xepreff X-Rspamd-Server: rspam12 X-HE-Tag: 1643645065-327766 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Simplify the code by using flush_dcache_folio(). Signed-off-by: Muchun Song --- mm/hugetlb.c | 6 +----- mm/migrate.c | 8 ++------ 2 files changed, 3 insertions(+), 11 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index f1f1ab31dc8a..828240aee3f9 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5804,8 +5804,6 @@ int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, goto out; } } else { - int i, nr; - if (vm_shared && hugetlbfs_pagecache_present(h, dst_vma, dst_addr)) { put_page(*pagep); @@ -5821,9 +5819,7 @@ int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, goto out; } folio_copy(page_folio(page), page_folio(*pagep)); - nr = compound_nr(page); - for (i = 0; i < nr; i++) - flush_dcache_page(page + i); + flush_dcache_folio(page_folio(page)); put_page(*pagep); *pagep = NULL; } diff --git a/mm/migrate.c b/mm/migrate.c index c418e8d92b9c..daf2b3508670 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -933,12 +933,8 @@ static int move_to_new_page(struct page *newpage, struct page *page, if (!PageMappingFlags(page)) page->mapping = NULL; - if (likely(!is_zone_device_page(newpage))) { - int i, nr = compound_nr(newpage); - - for (i = 0; i < nr; i++) - flush_dcache_page(newpage + i); - } + if (likely(!is_zone_device_page(newpage))) + flush_dcache_folio(page_folio(newpage)); } out: return rc;