From patchwork Tue Feb 8 07:36:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12738256 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1AFF8C433EF for ; Tue, 8 Feb 2022 07:36:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 82A356B0075; Tue, 8 Feb 2022 02:36:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7DA116B0078; Tue, 8 Feb 2022 02:36:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6A2006B007B; Tue, 8 Feb 2022 02:36:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0158.hostedemail.com [216.40.44.158]) by kanga.kvack.org (Postfix) with ESMTP id 5CBDE6B0075 for ; Tue, 8 Feb 2022 02:36:39 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 18E069676A for ; Tue, 8 Feb 2022 07:36:39 +0000 (UTC) X-FDA: 79118805318.22.F8AC0F4 Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by imf21.hostedemail.com (Postfix) with ESMTP id A3D1F1C000C for ; Tue, 8 Feb 2022 07:36:38 +0000 (UTC) Received: by mail-pj1-f53.google.com with SMTP id ki18-20020a17090ae91200b001b8be87e9abso1049301pjb.1 for ; Mon, 07 Feb 2022 23:36:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=gKFpQhrIc0qg1OGn2vjlbFux2cfH4pwiDVgfg/+s2Gk=; b=aesHRx4mkuJbzEXODBVC2NuJ3XIEly0uKuZ0fmmOCkbIBaQTKzJFrpHLliZ6LydIDC ZC6nmHSeL7VuMK9rDPMrgXApR4hnw1zY9zkx3KRUddNG14JwZrOt0HceyD46+rolqZSx 5HGsRA4wl3xJql/bNDiaoEcP5SL7tVmEM7fgqxVDJXJ0tBSkiUrnWmm8pAaDWr0inTKH A1TNo7Qo7cAjAJwTEXN/9nx4/YP86wP+m5YLCsjmA9PDo8Zwq+CGiG7a8HGwA1yRkFLt ziW2W/YhD7crzFt9s/t23pfDkgrlu90mkwCSBpAoO68ukfznQJvW2mLcsjBEhGGXcioK 9Gsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gKFpQhrIc0qg1OGn2vjlbFux2cfH4pwiDVgfg/+s2Gk=; b=gJqafxUoqclYYrjetMnJ4Ohq61JPOQHmUUD8fNS8J/fMxhEs5T5IGmMPmGuArWEGq+ 0hS5UR/5mJxARlcQbnSWnR0DHPPGWLvSqHHmhanHQR4VzX2y9Q/Frmrjk2U84b/nh5iC tS7myudnrLrgm3icBPus9XaxBJ8X9vY2CpGpqsRa8ZLiHTPD8wkB2sNqaz3SxGvF3A0b or/HdLn/4iW7Bn8695v5Re5dk0wFvLiAuayST/QNYcIFqL5cxPHD7zKMwFbqhxreLF+8 3owm/eQbFQ7FGn5z47dVSZNcflELMvY+F5AMkulGeMc/KBTCZt3HMgS+RW8M+XSLiIv0 K8OQ== X-Gm-Message-State: AOAM531Ud5FSBua6Dgv+IrZK1S+xdQwYxENbHD4ITLSBOQuuovr/rQod tie2GYU6MqWdmFDGR0TWE1E8Xg== X-Google-Smtp-Source: ABdhPJzEt4E48DQEs4fdQH9qP/j0O1H1fFoyW4x9658/zRKEKEvOfX+7jWzf6rShGjw7nmSjy1Bt3w== X-Received: by 2002:a17:90a:8904:: with SMTP id u4mr2969489pjn.137.1644305797603; Mon, 07 Feb 2022 23:36:37 -0800 (PST) Received: from FVFYT0MHHV2J.tiktokcdn.com ([139.177.225.225]) by smtp.gmail.com with ESMTPSA id gx10sm1621017pjb.7.2022.02.07.23.36.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Feb 2022 23:36:37 -0800 (PST) From: Muchun Song To: akpm@linux-foundation.org, zi.yan@cs.rutgers.edu, kirill.shutemov@linux.intel.com, rientjes@google.com, lars.persson@axis.com, mike.kravetz@oracle.com, ziy@nvidia.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, Muchun Song Subject: [PATCH v4 1/5] mm: thp: fix wrong cache flush in remove_migration_pmd() Date: Tue, 8 Feb 2022 15:36:13 +0800 Message-Id: <20220208073617.70342-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220208073617.70342-1-songmuchun@bytedance.com> References: <20220208073617.70342-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Stat-Signature: bjby89zhrrq6zb71ebuyo1pz5drucp6n Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=aesHRx4m; spf=pass (imf21.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: A3D1F1C000C X-HE-Tag: 1644305798-421627 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The flush_cache_range() is supposed to be justified only if the page is already placed in process page table, and that is done right after flush_cache_range(). So using this interface is wrong. And there is no need to invalite cache since it was non-present before in remove_migration_pmd(). So just to remove it. Signed-off-by: Muchun Song Reviewed-by: Zi Yan --- mm/huge_memory.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f58524394dc1..45ede45b11f5 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3207,7 +3207,6 @@ void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new) if (pmd_swp_uffd_wp(*pvmw->pmd)) pmde = pmd_wrprotect(pmd_mkuffd_wp(pmde)); - flush_cache_range(vma, mmun_start, mmun_start + HPAGE_PMD_SIZE); if (PageAnon(new)) page_add_anon_rmap(new, vma, mmun_start, true); else @@ -3215,6 +3214,8 @@ void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new) set_pmd_at(mm, mmun_start, pvmw->pmd, pmde); if ((vma->vm_flags & VM_LOCKED) && !PageDoubleMap(new)) mlock_vma_page(new); + + /* No need to invalidate - it was non-present before */ update_mmu_cache_pmd(vma, address, pvmw->pmd); } #endif From patchwork Tue Feb 8 07:36:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12738257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A478C433EF for ; Tue, 8 Feb 2022 07:36:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 125086B0078; Tue, 8 Feb 2022 02:36:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0D5A46B007B; Tue, 8 Feb 2022 02:36:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EDED06B007D; Tue, 8 Feb 2022 02:36:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0074.hostedemail.com [216.40.44.74]) by kanga.kvack.org (Postfix) with ESMTP id E098D6B0078 for ; Tue, 8 Feb 2022 02:36:43 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id A1C4C181DF74E for ; Tue, 8 Feb 2022 07:36:43 +0000 (UTC) X-FDA: 79118805486.20.F8CD757 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf19.hostedemail.com (Postfix) with ESMTP id 43E9C1A0002 for ; Tue, 8 Feb 2022 07:36:43 +0000 (UTC) Received: by mail-pl1-f177.google.com with SMTP id t9so10947481plg.13 for ; Mon, 07 Feb 2022 23:36:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=I/ARb1Au1FcGhHT5OSe5P8uQ767qWIf49sbqK0ZF8no=; b=hIUCyUYTQeWIyj0y3xa/lyJzJfBL9YeIpDkgIOVfa9Clb0wtstmPR610H1IH10cyNC IG+zGUOO/ZeEVHE8BfrdZHeMAPXZJrhPIZ5Fm2Tamk5Xsyp2QOnEPsMIAJkaPCHR1gfa U32l7A+wL1flsdi7qlpHY+HbrTGtygA2afkAodp9Xx4VKZDhbKB5Md2FpFPw1lKz1Us7 lJvwf8JjE0koHnrQ1fNx3hwWEWzZMKQl0hl921uE1+/CB+qidA3V6kqFUA7qx1owiBSo ZKfSFuUUUODfiZlV0CGELceZ3dOxNuZQUzREfn5T/pI5Po6WOrr2dQ3HwuByV8LbMakC SBYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=I/ARb1Au1FcGhHT5OSe5P8uQ767qWIf49sbqK0ZF8no=; b=X7/nSgGn+kKd7e1YHu5Izp/Yk8DuwEnOlryN6JhALoGQ2uAOOiXtyJr0Pr9QlI7ItH qbGKCj8IUpITFJ5wNCpeHv5BffuwdlObwxdLwcH0WEOlTCeUurZkWflFccErDkgaKivW xFSLuIa+dikmDLot504l/qaGVIyXmwqXpy1Hrgi4sVAPh0UC0AQ1CEqo7RJdaOun0Snf XvmQPIUdtHLQBABn+E1Rl2C7IuVJzfPh1btYuMK8jQ4oKEKEKLIV7OsRidtTk50vLG/Y E17sHpn2Lv35EMNZTH5uOjzQJKGewVUagml9MsLoEIsE7dw6MALun9oe+ZIE3Jk/utyW 8FOQ== X-Gm-Message-State: AOAM530yT8P3XMu0TIZlSOgoLf9iuMMdy2BaCwKTrc7wD2jMzP7409rt Q2orydbt0rbYheyIxLI618UcZw== X-Google-Smtp-Source: ABdhPJwsZdtiU96OKE7UAsec4IC2Z4kjKKWlUx3J30L+vHnc5YKdB/6S38QvVd5szKB7Y7RVjoAr2w== X-Received: by 2002:a17:902:bf4a:: with SMTP id u10mr3025261pls.113.1644305802222; Mon, 07 Feb 2022 23:36:42 -0800 (PST) Received: from FVFYT0MHHV2J.tiktokcdn.com ([139.177.225.225]) by smtp.gmail.com with ESMTPSA id gx10sm1621017pjb.7.2022.02.07.23.36.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Feb 2022 23:36:41 -0800 (PST) From: Muchun Song To: akpm@linux-foundation.org, zi.yan@cs.rutgers.edu, kirill.shutemov@linux.intel.com, rientjes@google.com, lars.persson@axis.com, mike.kravetz@oracle.com, ziy@nvidia.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, Muchun Song Subject: [PATCH v4 2/5] mm: fix missing cache flush for all tail pages of compound page Date: Tue, 8 Feb 2022 15:36:14 +0800 Message-Id: <20220208073617.70342-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220208073617.70342-1-songmuchun@bytedance.com> References: <20220208073617.70342-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Stat-Signature: ujahozoj8wkg79qkfchfff5dm4tg7szi X-Rspam-User: Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=hIUCyUYT; dmarc=pass (policy=none) header.from=bytedance.com; spf=pass (imf19.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 43E9C1A0002 X-HE-Tag: 1644305803-34685 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The D-cache maintenance inside move_to_new_page() only consider one page, there is still D-cache maintenance issue for tail pages of compound page (e.g. THP or HugeTLB). THP migration is only enabled on x86_64, ARM64 and powerpc, while powerpc and arm64 need to maintain the consistency between I-Cache and D-Cache, which depends on flush_dcache_page() to maintain the consistency between I-Cache and D-Cache. But there is no issues on arm64 and powerpc since they already considers the compound page cache flushing in their icache flush function. HugeTLB migration is enabled on arm, arm64, mips, parisc, powerpc, riscv, s390 and sh, while arm has handled the compound page cache flush in flush_dcache_page(), but most others do not. In theory, the issue exists on many architectures. Fix this by not using flush_dcache_folio() since it is not backportable. Fixes: 290408d4a250 ("hugetlb: hugepage migration core") Signed-off-by: Muchun Song Reviewed-by: Zi Yan --- mm/migrate.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index c9296d63878d..c418e8d92b9c 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -933,9 +933,12 @@ static int move_to_new_page(struct page *newpage, struct page *page, if (!PageMappingFlags(page)) page->mapping = NULL; - if (likely(!is_zone_device_page(newpage))) - flush_dcache_page(newpage); + if (likely(!is_zone_device_page(newpage))) { + int i, nr = compound_nr(newpage); + for (i = 0; i < nr; i++) + flush_dcache_page(newpage + i); + } } out: return rc; From patchwork Tue Feb 8 07:36:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12738258 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFC04C433F5 for ; Tue, 8 Feb 2022 07:36:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6C9CC6B007B; Tue, 8 Feb 2022 02:36:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 679D76B007D; Tue, 8 Feb 2022 02:36:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 543DD6B007E; Tue, 8 Feb 2022 02:36:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id 4583D6B007B for ; Tue, 8 Feb 2022 02:36:48 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 0A69E20F87 for ; Tue, 8 Feb 2022 07:36:48 +0000 (UTC) X-FDA: 79118805696.01.E685BCB Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf31.hostedemail.com (Postfix) with ESMTP id B880920002 for ; Tue, 8 Feb 2022 07:36:47 +0000 (UTC) Received: by mail-pj1-f47.google.com with SMTP id p22-20020a17090adf9600b001b8783b2647so1325194pjv.5 for ; Mon, 07 Feb 2022 23:36:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cll0HiNGaOM9yDWI60AkgZZ8Vcgf+tRA3oZc4UBL7nk=; b=y0jpOk6gAkKem1qpuxCydTJQeI0RYuJhd3oYTOoGI39XA5VyoB+ep0jJ6o+P/dV+6o mlwuzoo+SR5YLg3y2a/BC+MJRoaSWHaJ+qAJZb5Y5tHw3Dvyj2R7bsDTORVKf+0aoy9q Jn9P8oSuy9WMEeo5jqDcltiB2MAuJdmewps+UxFLCGBfsHbB4Nz8BgoO7bfS4ookfhRt fJVNLLDP8w7JHk+3AsBSjsnnR7LoF4EmZz1NN9NARpNJUzV4ELzDvSQGQOwb5dl3rKMT pSdNUaU72b0vAJoGUIJ5R22jT7AVOdSonWMx6C0mW3i1zT2otqX9O90SuABNbMer7Tl8 XOnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cll0HiNGaOM9yDWI60AkgZZ8Vcgf+tRA3oZc4UBL7nk=; b=FjXoyhnSjo99+J1PNVj6cn8cQqg7mewvWj7+e6Y5w1kCEep9X40ggF9BHbdkKymV1T MBLAlhClFxytXt1YBFDpj+97Z7PUgdQy/6alEMsnObfcAw3sqpkeXlrDyVU4quIuoEdx CyA+2lg5+MteuhzQqY56yQd+WMcZhnDX36u4xyJypvUZP4tRBHlxM7EUVdugMvHMTRZg 2HyHmjpvFRsYbG/gwO6pBhiu57g8tvnGyWSdN1buUTsZjnKqcYoACGTz+z/Ug4ubYCQ0 5BRx5eNRQp8I/n5ZjZbXuDd6JfnGQScE6M98j8cbBTa3jC0eteqFa8h18fjuPxc+AjEy 9aqg== X-Gm-Message-State: AOAM532MAKT2GNSQziaz5SyFahXxlkleicre3SiHO0W3YkYvCPXjFyHp JeioRJWG6ILFtJpDhG1pQE6iwQ== X-Google-Smtp-Source: ABdhPJzvsh4kBT5/wdRGyGAAQcGNW7TSFAisLlWR6Pe1OtowTbWhv0vQtaLjIv9HxR4IbxCYWBl48w== X-Received: by 2002:a17:902:b90a:: with SMTP id bf10mr3230897plb.36.1644305806765; Mon, 07 Feb 2022 23:36:46 -0800 (PST) Received: from FVFYT0MHHV2J.tiktokcdn.com ([139.177.225.225]) by smtp.gmail.com with ESMTPSA id gx10sm1621017pjb.7.2022.02.07.23.36.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Feb 2022 23:36:46 -0800 (PST) From: Muchun Song To: akpm@linux-foundation.org, zi.yan@cs.rutgers.edu, kirill.shutemov@linux.intel.com, rientjes@google.com, lars.persson@axis.com, mike.kravetz@oracle.com, ziy@nvidia.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, Muchun Song Subject: [PATCH v4 3/5] mm: hugetlb: fix missing cache flush in copy_huge_page_from_user() Date: Tue, 8 Feb 2022 15:36:15 +0800 Message-Id: <20220208073617.70342-4-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220208073617.70342-1-songmuchun@bytedance.com> References: <20220208073617.70342-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspam-User: Authentication-Results: imf31.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=y0jpOk6g; dmarc=pass (policy=none) header.from=bytedance.com; spf=pass (imf31.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.216.47 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: B880920002 X-Stat-Signature: ppc7acp47cgxsaz9x3pbce66uoas8oi1 X-HE-Tag: 1644305807-784313 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The userfaultfd calls copy_huge_page_from_user() which does not do any cache flushing for the target page. Then the target page will be mapped to the user space with a different address (user address), which might have an alias issue with the kernel address used to copy the data from the user to. Fix this issue by flushing dcache in copy_huge_page_from_user(). Fixes: fa4d75c1de13 ("userfaultfd: hugetlbfs: add copy_huge_page_from_user for hugetlb userfaultfd support") Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz --- mm/memory.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/memory.c b/mm/memory.c index e8ce066be5f2..eb027da68aa7 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5405,6 +5405,8 @@ long copy_huge_page_from_user(struct page *dst_page, if (rc) break; + flush_dcache_page(subpage); + cond_resched(); } return ret_val; From patchwork Tue Feb 8 07:36:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12738259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6416DC433FE for ; Tue, 8 Feb 2022 07:36:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F2E206B007D; Tue, 8 Feb 2022 02:36:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EDE106B007E; Tue, 8 Feb 2022 02:36:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA6406B0080; Tue, 8 Feb 2022 02:36:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0235.hostedemail.com [216.40.44.235]) by kanga.kvack.org (Postfix) with ESMTP id CB70D6B007D for ; Tue, 8 Feb 2022 02:36:52 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 91341181DF74E for ; Tue, 8 Feb 2022 07:36:52 +0000 (UTC) X-FDA: 79118805864.23.18F00A3 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by imf07.hostedemail.com (Postfix) with ESMTP id 3B43640008 for ; Tue, 8 Feb 2022 07:36:52 +0000 (UTC) Received: by mail-pf1-f177.google.com with SMTP id e6so17428851pfc.7 for ; Mon, 07 Feb 2022 23:36:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/rv2PEt6IIiPLW9lZEeUjxxgQpIiSoXGPYPSbKMenEs=; b=mWirpXFETjjymQb9FwJYTXT9RWcZSQSLT5Nsh/1DBOfuh2Biuy/KM+f9R5CeJIlYYC GDwg8fBYZTDlssF3zNfvwJnkHHLbN9g9hR/k68bwOWP4LlYKrolMPPGjF06x1diK+Vpg Oeq7EOUsr9mUAe81PFskxE4Owu3enMTWXolrKTdSt0ZY2vd98WKWzyNREyikVcUun4xE SyrT606wjX2rhlV21mzXjz0B6XCMwG0ldcYNkSV1MewRFtJ3CeKOt8uPKd04zxFdx6ft e8aAg8sn9SGEQdDo3KLTRL2+fX4oI1DR15XPBg91kF7l+/phcejECxxqPkHZHbrWx/sK 7lKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/rv2PEt6IIiPLW9lZEeUjxxgQpIiSoXGPYPSbKMenEs=; b=KvEqkeek1XVcZ/874LqF8GmusXQ9WZhWUupGYuDzH/70TP+sEqpWkznSmb+/Zx0E9p CaCcsqvnT8u9WXyunA2lZkTJm4VMj+X+X3jWM+qclU2G78Q7XAOq+j65UTz5P9au/L4o H5nbpFv/W5lENgyMWHN3Q22DC0dRIjcgrhMjz0/bpkiSNDW2L1BexxdnNbZNjTGe4Gng XIapK6wpaBjnJHB5mSs2FA9wwNP2mqmG5LGYJUTs8/ztHO9rjr3Iq9gLiND/PaUpsg+T sScZ2AK5ZeBZuIl9Wo+VNnOHwZvjqqfZWOW/GJ9rVtYuEdzMLcNTM3YZIRPqyD9EI7VT qM5g== X-Gm-Message-State: AOAM530GNdxrpOPTR3da+vF5cQU+LkfNl7Nv0DZkrfpmLDpEWeiGK9OB 5v0hqZ+Mepz3xmvG9mafujjfwg== X-Google-Smtp-Source: ABdhPJx5wI1WRTMKEZCVCDQaBQBLj00V8f1A43/kPGwa6KwBioaxK5T2yS6JmK55WW3Uw7U4O59aTA== X-Received: by 2002:a05:6a00:1253:: with SMTP id u19mr3314465pfi.8.1644305811236; Mon, 07 Feb 2022 23:36:51 -0800 (PST) Received: from FVFYT0MHHV2J.tiktokcdn.com ([139.177.225.225]) by smtp.gmail.com with ESMTPSA id gx10sm1621017pjb.7.2022.02.07.23.36.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Feb 2022 23:36:50 -0800 (PST) From: Muchun Song To: akpm@linux-foundation.org, zi.yan@cs.rutgers.edu, kirill.shutemov@linux.intel.com, rientjes@google.com, lars.persson@axis.com, mike.kravetz@oracle.com, ziy@nvidia.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, Muchun Song Subject: [PATCH v4 4/5] mm: hugetlb: fix missing cache flush in hugetlb_mcopy_atomic_pte() Date: Tue, 8 Feb 2022 15:36:16 +0800 Message-Id: <20220208073617.70342-5-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220208073617.70342-1-songmuchun@bytedance.com> References: <20220208073617.70342-1-songmuchun@bytedance.com> MIME-Version: 1.0 Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=mWirpXFE; dmarc=pass (policy=none) header.from=bytedance.com; spf=pass (imf07.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com X-Rspam-User: X-Rspamd-Queue-Id: 3B43640008 X-Stat-Signature: eukzue9rc1uy4a3ybzqc16fpk7kw45r1 X-Rspamd-Server: rspam07 X-HE-Tag: 1644305812-756089 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: folio_copy() will copy the data from one page to the target page, then the target page will be mapped to the user space address, which might have an alias issue with the kernel address used to copy the data from the page to. There are 2 ways to fix this issue. 1) insert flush_dcache_page() after folio_copy(). 2) replace folio_copy() with copy_user_huge_page() which already considers the cache maintenance. We chose 2) way to fix the issue since architectures can optimize this situation. Fixes: 8cc5fcbb5be8 ("mm, hugetlb: fix racy resv_huge_pages underflow on UFFDIO_COPY") Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz --- mm/hugetlb.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index a1baa198519a..eba7681d15d0 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5818,7 +5818,8 @@ int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, *pagep = NULL; goto out; } - folio_copy(page_folio(page), page_folio(*pagep)); + copy_user_huge_page(page, *pagep, dst_addr, dst_vma, + pages_per_huge_page(h)); put_page(*pagep); *pagep = NULL; } From patchwork Tue Feb 8 07:36:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12738260 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0DB27C433F5 for ; Tue, 8 Feb 2022 07:37:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A24506B007E; Tue, 8 Feb 2022 02:37:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9D5456B0080; Tue, 8 Feb 2022 02:37:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8C3B26B0081; Tue, 8 Feb 2022 02:37:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0003.hostedemail.com [216.40.44.3]) by kanga.kvack.org (Postfix) with ESMTP id 7FC176B007E for ; Tue, 8 Feb 2022 02:37:00 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 4CFC192EC6 for ; Tue, 8 Feb 2022 07:37:00 +0000 (UTC) X-FDA: 79118806200.06.992F6FE Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf21.hostedemail.com (Postfix) with ESMTP id EEC3C1C000B for ; Tue, 8 Feb 2022 07:36:59 +0000 (UTC) Received: by mail-pl1-f172.google.com with SMTP id c3so13160489pls.5 for ; Mon, 07 Feb 2022 23:36:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ppq4XlgFUv42e60ZLq/R/m9c3uPkEPmuzaFcdHPRBnk=; b=xVoHJj/4zyPvyg7fneG+v1w0Ib0159bkGEZFF8GmxqGz1GIqJQvjHPPWQUUBQVBWSi yR2SPqfsbh97O8Bk0wkaV2DY7IDHGcuTgyVWO/W/8a0okGQfamKHXNVAIF4if1wm6/w5 iiwk1B4JgZBA4eWoYz+PU1VYZ55CZdIYVNH4J5zSrekqulSmRsmD2G0K+Pf8fVKb53hx t4ihL4kylEoaMDcmJaTJcAiXjkdVmkC43SCUedvI61iLAerc6p8houdOghfe1doOVpQa oubwmMrsU31yjFg4iMICpMtw3IcU5yLSUYRvgyulqzAWcjvp0pMZyUz3r23mxXP1tv2z smFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ppq4XlgFUv42e60ZLq/R/m9c3uPkEPmuzaFcdHPRBnk=; b=vJT4t7OFMTeW19MnWSURtm5IVMLU4LQ3j/D3nTsCyznc4CLbsFuA0UfNLxowrFf5Pp s/coQBoQZ+fN6aVB634eaLHmPAppzvrPip9n2ZuvSoO0nD8rtQxficg+CyXPCF+Zpy6W EnVT89mWuXGqZUmm1aX81YJe7RiBlRtONzlcHzqIH0DoqczSD3qFfY3akkLsdJnb3Ktp ZG/kHFB+T4BwnO/fHvOq2j7GvP8Y/lZ1xhek6ziuSbMnn3EiHfse+ynpLwWrsWN4n5x+ QYGEo/6ZWAt1IFcLM159BVd+tgiMSJzWfh5UlpgEJ20Q6u3FCr/0WInyRP7u6+7qaOnA YnuQ== X-Gm-Message-State: AOAM5327Oq/9/dDJAs48q/Q9CncAdERw312wc0YujC7A3N40/Iu8aStW QiayUfuFE4aEG+vKJ++Gyka+NA== X-Google-Smtp-Source: ABdhPJzNE1tP0FCmJnNbT+MwdBxWiom5QGlNn45IpoyK3Oqddhm8dX950DKcNeuV59Q8o0qQlea8fA== X-Received: by 2002:a17:902:ab06:: with SMTP id ik6mr3065211plb.162.1644305818957; Mon, 07 Feb 2022 23:36:58 -0800 (PST) Received: from FVFYT0MHHV2J.tiktokcdn.com ([139.177.225.225]) by smtp.gmail.com with ESMTPSA id gx10sm1621017pjb.7.2022.02.07.23.36.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 07 Feb 2022 23:36:58 -0800 (PST) From: Muchun Song To: akpm@linux-foundation.org, zi.yan@cs.rutgers.edu, kirill.shutemov@linux.intel.com, rientjes@google.com, lars.persson@axis.com, mike.kravetz@oracle.com, ziy@nvidia.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, Muchun Song Subject: [PATCH v4 5/5] mm: replace multiple dcache flush with flush_dcache_folio() Date: Tue, 8 Feb 2022 15:36:17 +0800 Message-Id: <20220208073617.70342-6-songmuchun@bytedance.com> X-Mailer: git-send-email 2.32.0 (Apple Git-132) In-Reply-To: <20220208073617.70342-1-songmuchun@bytedance.com> References: <20220208073617.70342-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Stat-Signature: zc6rdawwpi5pgow7zejs5a98ty3k49fs X-Rspam-User: Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b="xVoHJj/4"; dmarc=pass (policy=none) header.from=bytedance.com; spf=pass (imf21.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: EEC3C1C000B X-HE-Tag: 1644305819-392737 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Simplify the code by using flush_dcache_folio(). Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz --- mm/migrate.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index c418e8d92b9c..daf2b3508670 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -933,12 +933,8 @@ static int move_to_new_page(struct page *newpage, struct page *page, if (!PageMappingFlags(page)) page->mapping = NULL; - if (likely(!is_zone_device_page(newpage))) { - int i, nr = compound_nr(newpage); - - for (i = 0; i < nr; i++) - flush_dcache_page(newpage + i); - } + if (likely(!is_zone_device_page(newpage))) + flush_dcache_folio(page_folio(newpage)); } out: return rc;