From patchwork Sat May 7 05:47:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: CGEL X-Patchwork-Id: 12841861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3798C433F5 for ; Sat, 7 May 2022 05:47:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5366A6B0071; Sat, 7 May 2022 01:47:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E4EA6B0073; Sat, 7 May 2022 01:47:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 386046B0074; Sat, 7 May 2022 01:47:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 2616F6B0071 for ; Sat, 7 May 2022 01:47:09 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E668B20698 for ; Sat, 7 May 2022 05:47:08 +0000 (UTC) X-FDA: 79437863736.30.175DEA4 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf04.hostedemail.com (Postfix) with ESMTP id 3F1E240036 for ; Sat, 7 May 2022 05:47:00 +0000 (UTC) Received: by mail-pj1-f52.google.com with SMTP id a15-20020a17090ad80f00b001dc2e23ad84so12572362pjv.4 for ; Fri, 06 May 2022 22:47:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=V4XZHagSWEnKC6Qw3MYHzFuq9+VP2v5ZnQJxJGUJwXc=; b=f9mfNMSz0YiO0lOZNfj7RfhvtB+tTUz3ixNSX2fdaxeBrYi2ldyoNAewrefY0DC8vT iNtIPbW4YK1B11PUAZLFfqhVAjT3VnZ+6P5rA3j4d24GUE8fhVxbpiTMQ9HL0iwv9k3j jgQl5oSK4tDrS4+btUN2eFNqXzDzk8nXsiY8R9mTEG6o4f5YxCviLE+Mnc3eJco1q5+r u0CeEpmvGgt5NX6HqCaUnbMsIJrSFVLcoG105bFjxRKYPyrmMPq4idq77NTiVtr1iLqg iMW5XNugh5JHWO9lmJPgujRCZlElTSC33NksH8k7e7g/ejh4v/R8sUFxFpjzKWyhJ2p7 pNTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=V4XZHagSWEnKC6Qw3MYHzFuq9+VP2v5ZnQJxJGUJwXc=; b=KeRsan5Y3GyLF7Dm52SeF7b0NbGG3qrRxmsQucI2Rjc4/ob3g1Q6EZLVHaMeOqeLWa ab6SAM5BM8RG0kwLkdMnA+onyAINMPT8YAoh0hPrh0LEfzshVyJmo8qTY3xTzGqWVt+B idmeVjbkAxdlbpg9TMh/GNh7c4WwUebLXIgrK83Gvf2A6W9q+hdBGmm0dSGGrrE4M0z3 2hkZ70h7oRkI0TrtU+8FtLFAiJxqsymUPhTLBe8LzScqmprwhuSUMdw9H1hiH2JupDJq TyCX36zHzo8eUj/rIR85NRDooCciY3QMN8a041QgWbygYGZfUel92wBHFNXhcpx2xhBe TACw== X-Gm-Message-State: AOAM533UsK+2RpucRnMEreQLQYxTpTPbnX4poUbfN7yjFEKYpaBvH32r eMODyC8AQNofCcNAwfYjdy0= X-Google-Smtp-Source: ABdhPJyOdTkOi3E0xzeZNk0WXxLeCOZtbpjnSDBxnGkRwJyz2wfP8U6lcQwrS/p9VyyZCBTyB1/zSQ== X-Received: by 2002:a17:903:283:b0:152:157:eb7 with SMTP id j3-20020a170903028300b0015201570eb7mr7026554plr.109.1651902427135; Fri, 06 May 2022 22:47:07 -0700 (PDT) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id q1-20020a170902788100b0015e8d4eb1dfsm2746669pll.41.2022.05.06.22.47.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 May 2022 22:47:06 -0700 (PDT) From: cgel.zte@gmail.com X-Google-Original-From: xu.xin16@zte.com.cn To: akpm@linux-foundation.org Cc: keescook@chromium.org, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, xu xin , Yang Yang , Ran Xiaokai , wangyong , Yunkai Zhang Subject: [PATCH v3] mm/ksm: introduce ksm_force for each process Date: Sat, 7 May 2022 05:47:02 +0000 Message-Id: <20220507054702.687958-1-xu.xin16@zte.com.cn> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Rspamd-Queue-Id: 3F1E240036 X-Stat-Signature: j1hbn58t6xd6xodx1tnypzh47ek7xt6n Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=f9mfNMSz; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf04.hostedemail.com: domain of cgel.zte@gmail.com designates 209.85.216.52 as permitted sender) smtp.mailfrom=cgel.zte@gmail.com X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1651902420-719066 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin To use KSM, we must explicitly call madvise() in application code, which means installed apps on OS needs to be uninstall and source code needs to be modified. It is inconvenient. In order to change this situation, We add a new proc 'ksm_force' under /proc// to support turning on/off KSM scanning of a process's mm dynamically. If ksm_force is set as 1, force all anonymous and 'qualified' vma of this mm to be involved in KSM scanning without explicitly calling madvise to make vma MADV_MERGEABLE. But It is effctive only when the klob of '/sys/kernel/mm/ksm/run' is set as 1. If ksm_enale is set as 0, cancel the feature of ksm_force of this process and unmerge those merged pages which is not madvised as MERGEABLE of this process, but leave MERGEABLE areas merged. Signed-off-by: xu xin Reviewed-by: Yang Yang Reviewed-by: Ran Xiaokai Reviewed-by: wangyong Reviewed-by: Yunkai Zhang --- v3: - fix compile error of mm/ksm.c v2: - fix a spelling error in commit log. - remove a redundant condition check in ksm_force_write(). --- fs/proc/base.c | 99 ++++++++++++++++++++++++++++++++++++++++ include/linux/mm_types.h | 9 ++++ mm/ksm.c | 32 ++++++++++++- 3 files changed, 138 insertions(+), 2 deletions(-) diff --git a/fs/proc/base.c b/fs/proc/base.c index 8dfa36a99c74..3115ffa4c9fb 100644 --- a/fs/proc/base.c +++ b/fs/proc/base.c @@ -96,6 +96,7 @@ #include #include #include +#include #include #include "internal.h" #include "fd.h" @@ -3168,6 +3169,102 @@ static int proc_pid_ksm_merging_pages(struct seq_file *m, struct pid_namespace * return 0; } + +static ssize_t ksm_force_read(struct file *file, char __user *buf, size_t count, + loff_t *ppos) +{ + struct task_struct *task; + struct mm_struct *mm; + char buffer[PROC_NUMBUF]; + ssize_t len; + int ret; + + task = get_proc_task(file_inode(file)); + if (!task) + return -ESRCH; + + mm = get_task_mm(task); + ret = 0; + if (mm) { + len = snprintf(buffer, sizeof(buffer), "%d\n", mm->ksm_force); + ret = simple_read_from_buffer(buf, count, ppos, buffer, len); + mmput(mm); + } + + return ret; +} + +static ssize_t ksm_force_write(struct file *file, const char __user *buf, + size_t count, loff_t *ppos) +{ + struct task_struct *task; + struct mm_struct *mm; + char buffer[PROC_NUMBUF]; + int force; + int err = 0; + + memset(buffer, 0, sizeof(buffer)); + if (count > sizeof(buffer) - 1) + count = sizeof(buffer) - 1; + if (copy_from_user(buffer, buf, count)) { + err = -EFAULT; + goto out_return; + } + + err = kstrtoint(strstrip(buffer), 0, &force); + + if (err) + goto out_return; + if (force != 0 && force != 1) { + err = -EINVAL; + goto out_return; + } + + task = get_proc_task(file_inode(file)); + if (!task) { + err = -ESRCH; + goto out_return; + } + + mm = get_task_mm(task); + if (!mm) + goto out_put_task; + + if (mm->ksm_force != force) { + if (mmap_write_lock_killable(mm)) { + err = -EINTR; + goto out_mmput; + } + + if (force == 0) + mm->ksm_force = force; + else { + /* + * Force anonymous pages of this mm to be involved in KSM merging + * without explicitly calling madvise. + */ + if (!test_bit(MMF_VM_MERGEABLE, &mm->flags)) + err = __ksm_enter(mm); + if (!err) + mm->ksm_force = force; + } + + mmap_write_unlock(mm); + } + +out_mmput: + mmput(mm); +out_put_task: + put_task_struct(task); +out_return: + return err < 0 ? err : count; +} + +static const struct file_operations proc_pid_ksm_force_operations = { + .read = ksm_force_read, + .write = ksm_force_write, + .llseek = generic_file_llseek, +}; #endif /* CONFIG_KSM */ #ifdef CONFIG_STACKLEAK_METRICS @@ -3303,6 +3400,7 @@ static const struct pid_entry tgid_base_stuff[] = { #endif #ifdef CONFIG_KSM ONE("ksm_merging_pages", S_IRUSR, proc_pid_ksm_merging_pages), + REG("ksm_force", S_IRUSR|S_IWUSR, proc_pid_ksm_force_operations), #endif }; @@ -3639,6 +3737,7 @@ static const struct pid_entry tid_base_stuff[] = { #endif #ifdef CONFIG_KSM ONE("ksm_merging_pages", S_IRUSR, proc_pid_ksm_merging_pages), + REG("ksm_force", S_IRUSR|S_IWUSR, proc_pid_ksm_force_operations), #endif }; diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index b34ff2cdbc4f..1b1592c2f5cf 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -661,6 +661,15 @@ struct mm_struct { * merging. */ unsigned long ksm_merging_pages; + /* + * If true, force anonymous pages of this mm to be involved in KSM + * merging without explicitly calling madvise. It is effctive only + * when the klob of '/sys/kernel/mm/ksm/run' is set as 1. If false, + * cancel the feature of ksm_force of this process and unmerge + * those merged pages which is not madvised as MERGEABLE of this + * process, but leave MERGEABLE areas merged. + */ + bool ksm_force; #endif } __randomize_layout; diff --git a/mm/ksm.c b/mm/ksm.c index 38360285497a..c9f672dcc72e 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -334,6 +334,34 @@ static void __init ksm_slab_free(void) mm_slot_cache = NULL; } +/* Check if vma is qualified for ksmd scanning */ +static bool ksm_vma_check(struct vm_area_struct *vma) +{ + unsigned long vm_flags = vma->vm_flags; + + if (!(vma->vm_flags & VM_MERGEABLE) && !(vma->vm_mm->ksm_force)) + return false; + + if (vm_flags & (VM_SHARED | VM_MAYSHARE | + VM_PFNMAP | VM_IO | VM_DONTEXPAND | + VM_HUGETLB | VM_MIXEDMAP)) + return false; /* just ignore this vma*/ + + if (vma_is_dax(vma)) + return false; + +#ifdef VM_SAO + if (vm_flags & VM_SAO) + return false; +#endif +#ifdef VM_SPARC_ADI + if (vm_flags & VM_SPARC_ADI) + return false; +#endif + + return true; +} + static __always_inline bool is_stable_node_chain(struct stable_node *chain) { return chain->rmap_hlist_len == STABLE_NODE_CHAIN; @@ -523,7 +551,7 @@ static struct vm_area_struct *find_mergeable_vma(struct mm_struct *mm, if (ksm_test_exit(mm)) return NULL; vma = vma_lookup(mm, addr); - if (!vma || !(vma->vm_flags & VM_MERGEABLE) || !vma->anon_vma) + if (!vma || !ksm_vma_check(vma) || !vma->anon_vma) return NULL; return vma; } @@ -2297,7 +2325,7 @@ static struct rmap_item *scan_get_next_rmap_item(struct page **page) vma = find_vma(mm, ksm_scan.address); for (; vma; vma = vma->vm_next) { - if (!(vma->vm_flags & VM_MERGEABLE)) + if (!ksm_vma_check(vma)) continue; if (ksm_scan.address < vma->vm_start) ksm_scan.address = vma->vm_start;