From patchwork Fri Apr 14 14:23:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chih-En Lin X-Patchwork-Id: 13211617 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BABFC77B72 for ; Fri, 14 Apr 2023 14:25:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CDF386B0075; Fri, 14 Apr 2023 10:25:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C8F9A280002; Fri, 14 Apr 2023 10:25:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B3068900004; Fri, 14 Apr 2023 10:25:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A65FB6B0075 for ; Fri, 14 Apr 2023 10:25:47 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 6FFBB40316 for ; Fri, 14 Apr 2023 14:25:47 +0000 (UTC) X-FDA: 80680220334.22.C8D1243 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf14.hostedemail.com (Postfix) with ESMTP id 6EB3D10001A for ; Fri, 14 Apr 2023 14:25:45 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b="hh51/J2G"; spf=pass (imf14.hostedemail.com: domain of shiyn.lin@gmail.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=shiyn.lin@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681482345; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kpWPxfuvaIk0mWPLrjy7A4XPzxhA7fLlfGdW1mVCUr4=; b=cToSSyzYgdHABytbltoInEAgHM0XrJYQ3VBAarWZ6mbJzH+JAE+/IuRYOx/HjC+pluLVHc N3seipRgOvssZr0GJ130FXDKwWI7AwEhtyKX5U5phjdH7jsuecaWZWRLsUcDHOxFfuJeXb ZSsla8z4wBlNulQpjjzPkVgKumsLLVg= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b="hh51/J2G"; spf=pass (imf14.hostedemail.com: domain of shiyn.lin@gmail.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=shiyn.lin@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681482345; a=rsa-sha256; cv=none; b=GzPps+3GDTYa1Fs1PThBheNtEIYkC8KoNGZ2FK/gyrNSsITnJ76qSfG+xnyKsSZsAqqT3f MgL4gFqDVK3BkyrcPzZlNaN7H8+4xx4LLSzMms3XFVJ9hj1davIrLPdkS2JtxxK0/MjqKc 3G4z4/hOEXTehykhwu2laItYdNVghuU= Received: by mail-pl1-f169.google.com with SMTP id d15so1219038pll.12 for ; Fri, 14 Apr 2023 07:25:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681482344; x=1684074344; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kpWPxfuvaIk0mWPLrjy7A4XPzxhA7fLlfGdW1mVCUr4=; b=hh51/J2GfHz8TwBXTTAD8ByIt7vqu+CJYMOv8Gn6VP6zDZ8jsx1AQK/YylhhDx4Yy+ 0CRLCxANjdDWRbaxrpOrHOgnYRD+lcAf3QSkDS3YbWO/xcoE3Gbv43mriuPlxclwVTP9 phx5iCBpOFU3ljdbu0G2CPDJwoQgv/dk9OnKHsSHbvxVBZS7ab+FJCrdG2Pr8gDTkNbA cLPqeTL9RK1Gw4vF6qbn/cWyXpQXTI1KYeztWpb3aEC0oy5a7FRBFGm+8ZwfNEXjkHbL Rw88PFqPGh3pJ+S1C9QijWHos/UCC72XeH/rxFcOom0qMDyPSeMXOfp7PnqjaxjFryYV 7CUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681482344; x=1684074344; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kpWPxfuvaIk0mWPLrjy7A4XPzxhA7fLlfGdW1mVCUr4=; b=izewc6sxqY5DEqhOvnnbaj6cW0VGxSGZ4ccrfbQfX2hxwx+zWpZlJ7CFTelpKJpvJ4 Afd3to4WsCp0BLtvlbVTt6pdGjWyYHCQjHMp1cA111o4ijA/r/zTQGHI9QdrPnvPlKV7 LKcS1uY8ysZKoQysPLZqRFYECYcUo3p9HcYKAjVgRlcYFLFp0JN1mh9j+KL9a1I7hvRs 23cgq8eE7RxWZSXw7VPevHjmORn47UO76Um3JwI3ZTT5Z/FAeyAkPBGK5MZuLSKdeKBp eM8cwUCIBwlQ+jdmaZ0XS+WbXCAv0JyxkyaEU2gc1J+6kWqcrsAZahl5q/1Htt7TcW/o /06w== X-Gm-Message-State: AAQBX9eKLkq4RCqHqiaHywsQ/Renls3WDBAyU25I5qInP2j2a8BXaJV8 14KviTs9kqJhSEGYx9IL68c= X-Google-Smtp-Source: AKy350aBdG9vJ3XmlVaPR+W/QhjaJcPhPOUy0mPoGFQqm07k6eNoRRTZfMByKXsSMfstD+2cplrm3g== X-Received: by 2002:a17:90a:4925:b0:23f:7ff6:eba with SMTP id c34-20020a17090a492500b0023f7ff60ebamr5879955pjh.0.1681482344318; Fri, 14 Apr 2023 07:25:44 -0700 (PDT) Received: from strix-laptop.. (2001-b011-20e0-1499-8303-7502-d3d7-e13b.dynamic-ip6.hinet.net. [2001:b011:20e0:1499:8303:7502:d3d7:e13b]) by smtp.googlemail.com with ESMTPSA id h7-20020a17090ac38700b0022335f1dae2sm2952386pjt.22.2023.04.14.07.25.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 14 Apr 2023 07:25:43 -0700 (PDT) From: Chih-En Lin To: Andrew Morton , Qi Zheng , David Hildenbrand , "Matthew Wilcox (Oracle)" , Christophe Leroy , John Hubbard , Nadav Amit , Barry Song , Pasha Tatashin Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Steven Rostedt , Masami Hiramatsu , Peter Zijlstra , Arnaldo Carvalho de Melo , Mark Rutland , Alexander Shishkin , Jiri Olsa , Namhyung Kim , Ian Rogers , Adrian Hunter , Yu Zhao , Steven Barrett , Juergen Gross , Peter Xu , Kefeng Wang , Tong Tiangen , Christoph Hellwig , "Liam R. Howlett" , Yang Shi , Vlastimil Babka , Alex Sierra , Vincent Whitchurch , Anshuman Khandual , Li kunyu , Liu Shixin , Hugh Dickins , Minchan Kim , Joey Gouly , Chih-En Lin , Michal Hocko , Suren Baghdasaryan , "Zach O'Keefe" , Gautam Menghani , Catalin Marinas , Mark Brown , "Eric W. Biederman" , Andrei Vagin , Shakeel Butt , Daniel Bristot de Oliveira , "Jason A. Donenfeld" , Greg Kroah-Hartman , Alexey Gladkov , x86@kernel.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Dinglan Peng , Pedro Fonseca , Jim Huang , Huichun Feng Subject: [PATCH v5 09/17] mm/madvise: Handle COW-ed PTE with madvise() Date: Fri, 14 Apr 2023 22:23:33 +0800 Message-Id: <20230414142341.354556-10-shiyn.lin@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230414142341.354556-1-shiyn.lin@gmail.com> References: <20230414142341.354556-1-shiyn.lin@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 6EB3D10001A X-Stat-Signature: w1kqq59d5nat6ikd8qbntrsicutdtccg X-Rspam-User: X-HE-Tag: 1681482345-623415 X-HE-Meta: U2FsdGVkX18xQ3fyIGu/4t7K2BzgfHLpR1TpQkHw0Ptd9l/Gk2i0pTVE5K2l/uwr+Wh1PI4IYK2xACneCDCrXL1Ula4jFXchuuUOru1cS1Bf0cb37N4UbpZUvPu0viRlIDom2wAnwJR8LjukFpyCB+jCUZMcE7ovwwVIl4YF34uKE2a9iHIyjRM0+wFZAJr3XramQyojK3wwiI8+14aa+PelfmDEqPxhk6kdT/TLrzlZujARiRkKU+zg+mqlkADd+TUGCcxpYgqAtSGL2FjLDiBdgDH9RcvADCoOvVrbTa1SABH6CQ3Wa+HBK1OWT+eLYHwqC+Rrk0YDbsxHgncV8Uc4YeYA69Apnsbmenk0incwuAB4fwXAAQsU9p1cwiKH3hFohwmOJmrU8S+g7N5oSu2vhn83sDcuDK4XVCD36rnXR9jdMmZfvn662e/EStHluoy9L0p7QCkV3llaFYNyrFiXjpkUVEkk2yhdPi8snfZFBkvSjNx4WATs/za4tKv/G1JIrMSP/UbSxnsgyCQRDfFHgmr00Nv4PZ5O/+RZ64AC9loYqRdOQHH79415MLgaNopA1N5/61VPqc5Pim35FM+7iL+hfNQeanJrDs/3s29ada63K3I2LIsh8P8PeqnrZ2EnfisqhDaSxmCR1OPwfx7T/gXX7JuySGoAhPiJmJ+9T7Dt+S510QZWLra/QKs8kks0Qg+60ySS5ufx2GUafa2E+0U0FklyF1KPhinZ1LkjTr/YhWosKENe5PF5NjjDeh52Nvzfv1EJiBXjpWtJ2CX17m0bvfbGrqrHNb8zOP6SUA0mZXS78ZcVOgMvV7T9k8eN+mOC6S7p9JK5EyPEKiEZ0ZCy8BuAoOlmM0NT84+gmPo6MPTG2xrebzZo/6CBgPzuVBK08nXiRrC+YABY1a3Gl/H2fkwp4+oqbA3IHt2aut3uiC+KrUQEKAvRVkGIK/h6QhXg+v/nEmG7cBQ SiPmoExn e2EC8TyfsIVLxxqgMyur9IN2ypjcwIUooY6veUxFDszJjRj9mSH5F3hx422IZKpA0EbAVnm6QOpaKmqogN+r1TaMC/R2ohABuJwVGPUbru613BqCjmJkA+GSUJ+bxeYPUJ23+FKcfXjOsnETgwH0I0bX44Pkge+Ivq9Bx0yg7/BPbMHID8mLKaXxu73YCutcgZWn6KJCoU6qqc9L0YnrSppe9bR1JU9t1C0Q2yShsgMWpjMc9DHClzBwf91XtsrGcY8M83OEmagOy/SojZkkVX+QldrfGoN/LWv7VaNwEoJ1r2yTH+M8rraAm6HFN7W/WDlE4NUCok4PGuGWBwDBJsPMZmlAr04K5NhSWSYcBA+pYKk7QJcOxZSt3i2b2u++lWfehpHXO9wUPjoyKOjg2Fly6Jyo/dGZKhKJ+5XVfQuHcSqwLfGkLxDmgmWYZaqbVSPWKuMHX1eeBEL/MMD6mtwOfu34V44gJNJl/V8EAnxd/PsWrrxqeJbcBkEeU90DBWDJ8UFBjD0AWiEgjMrX4Dimf9pJcRb23bGKY2oLGHaJ7Yi0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Break COW PTE if madvise() modify the pte entry of COW-ed PTE. Following are the list of flags which need to break COW PTE. However, like MADV_HUGEPAGE and MADV_MERGEABLE, we should handle it respectively. - MADV_DONTNEED: It calls to zap_page_range() which already be handled. - MADV_FREE: It uses walk_page_range() with madvise_free_pte_range() to free the page by itself, so add break_cow_pte(). - MADV_REMOVE: Same as MADV_FREE, it remove the page by itself, so add break_cow_pte_range(). - MADV_COLD: Similar to MAD_FREE, break COW PTE before pageout. - MADV_POPULATE: Let GUP deal with it. Signed-off-by: Chih-En Lin --- mm/madvise.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/mm/madvise.c b/mm/madvise.c index 340125d08c03..71176edb751e 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -425,6 +425,9 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd, if (pmd_trans_unstable(pmd)) return 0; #endif + if (break_cow_pte(vma, pmd, addr)) + return 0; + tlb_change_page_size(tlb, PAGE_SIZE); orig_pte = pte = pte_offset_map_lock(vma->vm_mm, pmd, addr, &ptl); flush_tlb_batched_pending(mm); @@ -625,6 +628,10 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr, if (pmd_trans_unstable(pmd)) return 0; + /* We should only allocate PTE. */ + if (break_cow_pte(vma, pmd, addr)) + goto next; + tlb_change_page_size(tlb, PAGE_SIZE); orig_pte = pte = pte_offset_map_lock(mm, pmd, addr, &ptl); flush_tlb_batched_pending(mm); @@ -984,6 +991,12 @@ static long madvise_remove(struct vm_area_struct *vma, if ((vma->vm_flags & (VM_SHARED|VM_WRITE)) != (VM_SHARED|VM_WRITE)) return -EACCES; + error = break_cow_pte_range(vma, start, end); + if (error < 0) + return error; + else if (error > 0) + return -ENOMEM; + offset = (loff_t)(start - vma->vm_start) + ((loff_t)vma->vm_pgoff << PAGE_SHIFT);