From patchwork Sun Nov 11 21:26:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Timofey Titovets X-Patchwork-Id: 10677877 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0B6C6139B for ; Sun, 11 Nov 2018 21:27:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EE9532A09F for ; Sun, 11 Nov 2018 21:27:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E15192A24A; Sun, 11 Nov 2018 21:27:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.4 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE,RCVD_IN_SORBS_WEB autolearn=no version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2C8DB2A09F for ; Sun, 11 Nov 2018 21:27:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A40DF6B0003; Sun, 11 Nov 2018 16:27:21 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9F0126B0005; Sun, 11 Nov 2018 16:27:21 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 907216B0006; Sun, 11 Nov 2018 16:27:21 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-lf1-f71.google.com (mail-lf1-f71.google.com [209.85.167.71]) by kanga.kvack.org (Postfix) with ESMTP id 2430B6B0003 for ; Sun, 11 Nov 2018 16:27:21 -0500 (EST) Received: by mail-lf1-f71.google.com with SMTP id z10so755769lfe.21 for ; Sun, 11 Nov 2018 13:27:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:mime-version:content-transfer-encoding; bh=Fu+D4CAD9piKj/DNPVzR9+0sRl+ZMtTo3qdmHS6fCrw=; b=Zsx1qjMHax7zDk8rlP/sKKjVsmeSuUGg2n36cZ3mfGCdKYYzvmSZ0NkFNhVT/3pYMe Lz6CzPzq8+hjOKqxz2VPSTQv7PNlbKMegbYbbBwQ107jkcgs5dz8f4mQdG10yFhfGMl8 qep/iQ7andaPnCCN/cEV9Ko+X/DAsG36CMlBBTWeXnIzdVOLPgf0LXjmGFxlV3La7bhD /AjUt17fS+g+L2Zpv+go5vzqHegn+HWbCGejw15cXYonGvAu64qYCsmQcpSIPvocYtyV cHp73h/Wis0upSHEtrLSUrcVKAphdcypF8ip65PJdXdbW6UuewD/A+NbGXzqRguylnG9 /juQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of nefelim4ag@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=nefelim4ag@gmail.com X-Gm-Message-State: AGRZ1gJGNVEBYcJ1SlmTeAONukX0c6ylFcd73S2/55j2LoV8UZxB2nfZ 3X6rEpZn5w5TyfCBsYKB7VCxQhN5PE45Q7rQ3F/bNSoUTIdoZR1zZxHhrNMsI13kYznabgcy8Bg FDJ25duaijmmKenkawe/FcwkEP9J258cenaLUzcklFSchAu0mG//TACTed3u9MRx/X87ZyBfWYv oC3r3VQzhYUmpDYX747ZVKQDJwlQ1oBjTpjzkwYWiePIcuNfK44TbR+uPIBT0VAAhKpWuR6MrA6 QNtcvNiNd3I1RjzmXa+wtYmXC8QqdPlzfBMqsHZZhdlpUrCFMudQgNRTX4ESTefIthn3UCVyQ3M 5/JUETTKobQzNMaJPJ5kTJnb2jzz5rYXNRhUrY6mOS2w7ST+kEkPCwVta0J5ZmmfmKspi9XaWQ= = X-Received: by 2002:a2e:b00a:: with SMTP id y10-v6mr9963178ljk.109.1541971640439; Sun, 11 Nov 2018 13:27:20 -0800 (PST) X-Received: by 2002:a2e:b00a:: with SMTP id y10-v6mr9963160ljk.109.1541971639216; Sun, 11 Nov 2018 13:27:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1541971639; cv=none; d=google.com; s=arc-20160816; b=gexlZKYu4tUXY35nZpofnq1ox9Rkb+2S85OEA1S8qbqGSiNM4sgMVcHVMLzuvrNMG9 2zSJOaDr45RKMkRXJFVCxqBvMEYXOO7A3aJOVZhf6jZKzHkUHofoONw7mwEKqOWONV3k MKC5l8uZ3tf6sEtPm+e3Vpxt/QPhVYyAe8mI5nmUiD1fLlRuWnHOC8Dsp3gJc9rD8Mwf xMl26CB0GN2oxFMjWTxQ5yAf0yN/hm58y/2XKdlJS8V39GjcxszSQHq6DTVJVmtrVlOP yuXo/9PcuP6qaACaJRvePvGhdZk5qX6r7HxT8m/HKH6XcyABPiUcM4GFyVCA3zhFUUEt XwnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from; bh=Fu+D4CAD9piKj/DNPVzR9+0sRl+ZMtTo3qdmHS6fCrw=; b=QWZYPOnbri2gbXxuwGiAaqd7wB2LOEJIkmidiQGeLbFOB0wBdl9tMpOGIkhUJCg6NT KslWggU6ulsVD8OIzHv1DWbLKrwTI0jel0GOt9IEPrWexOquh7n0S8o57y6VkmWZWRbF vs8F+NWrGQ4WFVnmNiuVMAOe+iroFXNIUdC/Fd6yCQye7yh6No6u9bnO/RvO0HjwL03a bJizDXwHL/IWl5ACPzRNyMhBtNUFwEal4IGC9/DFTPsM1QP2N51WkdPLPYZFsVCS/a5O CPtK0a3jgi8pedxOWNy3iyq4JRYwDwGIMDmpZCb0b0kgbd5oAYRiQJUTovVbeuhdgdbR 2xsA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of nefelim4ag@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=nefelim4ag@gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id c10-v6sor7712607ljb.11.2018.11.11.13.27.18 for (Google Transport Security); Sun, 11 Nov 2018 13:27:19 -0800 (PST) Received-SPF: pass (google.com: domain of nefelim4ag@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of nefelim4ag@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=nefelim4ag@gmail.com X-Google-Smtp-Source: AJdET5dN6BHaSo3YWH9lJ6IMaAqLBQd4dc/N2BDlncHJJuorPA0MpmMI4Jie8qn2PvzNh1A4Loeq5g== X-Received: by 2002:a2e:2281:: with SMTP id i123-v6mr8998786lji.154.1541971638468; Sun, 11 Nov 2018 13:27:18 -0800 (PST) Received: from localhost.localdomain ([178.121.238.12]) by smtp.gmail.com with ESMTPSA id s14-v6sm2727176lfk.59.2018.11.11.13.27.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 11 Nov 2018 13:27:17 -0800 (PST) From: Timofey Titovets To: linux-kernel@vger.kernel.org Cc: linux-doc@vger.kernel.org, linux-mm@kvack.org, Timofey Titovets Subject: [PATCH v2] ksm: allow dedup all tasks memory Date: Mon, 12 Nov 2018 00:26:10 +0300 Message-Id: <20181111212610.25213-1-timofey.titovets@synesis.ru> X-Mailer: git-send-email 2.19.1 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Timofey Titovets ksm by default working only on memory that added by madvice(). And only way get that work on other applications: - Use LD_PRELOAD and libraries - Patch kernel Lets use kernel task list in ksm_scan_thread and add logic to allow ksm import VMA from tasks. That behaviour controlled by new attribute: mode I try mimic hugepages attribute, so mode have two states: - normal - old default behaviour - always [new] - allow ksm to get tasks vma and try working on that. To reduce CPU load & tasklist locking time, ksm try import VMAs from one task per loop. So add new attribute "mode" Two passible values: - normal [default] - ksm use only madvice - always [new] - ksm will search vma over all processes memory and add it to the dedup list v1 -> v2: - Rebase on v4.19.1 Signed-off-by: Timofey Titovets --- Documentation/admin-guide/mm/ksm.rst | 7 ++ mm/ksm.c | 149 ++++++++++++++++++++++----- 2 files changed, 128 insertions(+), 28 deletions(-) diff --git a/Documentation/admin-guide/mm/ksm.rst b/Documentation/admin-guide/mm/ksm.rst index 9303786632d1..253f94a09be8 100644 --- a/Documentation/admin-guide/mm/ksm.rst +++ b/Documentation/admin-guide/mm/ksm.rst @@ -116,6 +116,13 @@ run Default: 0 (must be changed to 1 to activate KSM, except if CONFIG_SYSFS is disabled) +mode + * set always to allow ksm deduplicate memory of every process + * set normal to use only madviced memory + + Default: normal (dedupulicate only madviced memory as in + earlier releases) + use_zero_pages specifies whether empty pages (i.e. allocated pages that only contain zeroes) should be treated specially. When set to 1, diff --git a/mm/ksm.c b/mm/ksm.c index 1a088306ef81..5097d710c466 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -295,6 +295,10 @@ static int ksm_nr_node_ids = 1; static unsigned long ksm_run = KSM_RUN_STOP; static void wait_while_offlining(void); +#define KSM_MODE_NORMAL 0 +#define KSM_MODE_ALWAYS 1 +static unsigned long ksm_mode = KSM_MODE_NORMAL; + static DECLARE_WAIT_QUEUE_HEAD(ksm_thread_wait); static DEFINE_MUTEX(ksm_thread_mutex); static DEFINE_SPINLOCK(ksm_mmlist_lock); @@ -303,6 +307,11 @@ static DEFINE_SPINLOCK(ksm_mmlist_lock); sizeof(struct __struct), __alignof__(struct __struct),\ (__flags), NULL) +static inline int ksm_mode_always(void) +{ + return (ksm_mode == KSM_MODE_ALWAYS); +} + static int __init ksm_slab_init(void) { rmap_item_cache = KSM_KMEM_CACHE(rmap_item, 0); @@ -2386,17 +2395,94 @@ static void ksm_do_scan(unsigned int scan_npages) static int ksmd_should_run(void) { - return (ksm_run & KSM_RUN_MERGE) && !list_empty(&ksm_mm_head.mm_list); + return (ksm_run & KSM_RUN_MERGE) && + (!list_empty(&ksm_mm_head.mm_list) || ksm_mode_always()); +} + + +static int ksm_enter(struct mm_struct *mm, unsigned long *vm_flags) +{ + int err; + + if (*vm_flags & (VM_MERGEABLE | VM_SHARED | VM_MAYSHARE | + VM_PFNMAP | VM_IO | VM_DONTEXPAND | + VM_HUGETLB | VM_MIXEDMAP)) + return 0; + +#ifdef VM_SAO + if (*vm_flags & VM_SAO) + return 0; +#endif +#ifdef VM_SPARC_ADI + if (*vm_flags & VM_SPARC_ADI) + return 0; +#endif + if (!test_bit(MMF_VM_MERGEABLE, &mm->flags)) { + err = __ksm_enter(mm); + if (err) + return err; + } + + *vm_flags |= VM_MERGEABLE; + + return 0; +} + +/* + * Register all vmas for all processes in the system with KSM. + * Note that every call to ksm_madvise, for a given vma, after the first + * does nothing but set flags. + */ +void ksm_import_task_vma(struct task_struct *task) +{ + struct vm_area_struct *vma; + struct mm_struct *mm; + int error; + + mm = get_task_mm(task); + if (!mm) + return; + down_write(&mm->mmap_sem); + vma = mm->mmap; + while (vma) { + error = ksm_enter(vma->vm_mm, &vma->vm_flags); + vma = vma->vm_next; + } + up_write(&mm->mmap_sem); + mmput(mm); + return; } static int ksm_scan_thread(void *nothing) { + pid_t last_pid = 1; + pid_t curr_pid; + struct task_struct *task; + set_freezable(); set_user_nice(current, 5); while (!kthread_should_stop()) { mutex_lock(&ksm_thread_mutex); wait_while_offlining(); + if (ksm_mode_always()) { + /* + * import one task's vma per run + */ + read_lock(&tasklist_lock); + + for_each_process(task) { + curr_pid = task_pid_nr(task); + if (curr_pid == last_pid) + break; + } + + task = next_task(task); + last_pid = task_pid_nr(task); + + ksm_import_task_vma(task); + read_unlock(&tasklist_lock); + } if (ksmd_should_run()) ksm_do_scan(ksm_thread_pages_to_scan); mutex_unlock(&ksm_thread_mutex); @@ -2422,33 +2508,9 @@ int ksm_madvise(struct vm_area_struct *vma, unsigned long start, switch (advice) { case MADV_MERGEABLE: - /* - * Be somewhat over-protective for now! - */ - if (*vm_flags & (VM_MERGEABLE | VM_SHARED | VM_MAYSHARE | - VM_PFNMAP | VM_IO | VM_DONTEXPAND | - VM_HUGETLB | VM_MIXEDMAP)) - return 0; /* just ignore the advice */ - - if (vma_is_dax(vma)) - return 0; - -#ifdef VM_SAO - if (*vm_flags & VM_SAO) - return 0; -#endif -#ifdef VM_SPARC_ADI - if (*vm_flags & VM_SPARC_ADI) - return 0; -#endif - - if (!test_bit(MMF_VM_MERGEABLE, &mm->flags)) { - err = __ksm_enter(mm); - if (err) - return err; - } - - *vm_flags |= VM_MERGEABLE; + err = ksm_enter(mm, vm_flags); + if (err) + return err; break; case MADV_UNMERGEABLE: @@ -2852,6 +2914,36 @@ static ssize_t pages_to_scan_store(struct kobject *kobj, } KSM_ATTR(pages_to_scan); +static ssize_t mode_show(struct kobject *kobj, struct kobj_attribute *attr, + char *buf) +{ + switch (ksm_mode) { + case KSM_MODE_NORMAL: + return sprintf(buf, "always [normal]\n"); + break; + case KSM_MODE_ALWAYS: + return sprintf(buf, "[always] normal\n"); + break; + } + + return sprintf(buf, "always [normal]\n"); +} + +static ssize_t mode_store(struct kobject *kobj, struct kobj_attribute *attr, + const char *buf, size_t count) +{ + if (!memcmp("always", buf, min(sizeof("always")-1, count))) { + ksm_mode = KSM_MODE_ALWAYS; + wake_up_interruptible(&ksm_thread_wait); + } else if (!memcmp("normal", buf, min(sizeof("normal")-1, count))) { + ksm_mode = KSM_MODE_NORMAL; + } else + return -EINVAL; + + return count; +} +KSM_ATTR(mode); + static ssize_t run_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) { @@ -3109,6 +3201,7 @@ KSM_ATTR_RO(full_scans); static struct attribute *ksm_attrs[] = { &sleep_millisecs_attr.attr, &pages_to_scan_attr.attr, + &mode_attr.attr, &run_attr.attr, &pages_shared_attr.attr, &pages_sharing_attr.attr,