From patchwork Fri Mar 21 17:37:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sourav Panda X-Patchwork-Id: 14025835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11D7DC36005 for ; Fri, 21 Mar 2025 17:37:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 19ED6280004; Fri, 21 Mar 2025 13:37:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 123F4280001; Fri, 21 Mar 2025 13:37:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E9285280004; Fri, 21 Mar 2025 13:37:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id C599D280001 for ; Fri, 21 Mar 2025 13:37:35 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 100095390B for ; Fri, 21 Mar 2025 17:37:37 +0000 (UTC) X-FDA: 83246265354.04.193C893 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf24.hostedemail.com (Postfix) with ESMTP id 23D86180011 for ; Fri, 21 Mar 2025 17:37:34 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=2fdVLCFL; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf24.hostedemail.com: domain of 33aPdZwsKCOAUQWTCXRCPFCIQQING.EQONKPWZ-OOMXCEM.QTI@flex--souravpanda.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=33aPdZwsKCOAUQWTCXRCPFCIQQING.EQONKPWZ-OOMXCEM.QTI@flex--souravpanda.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742578655; a=rsa-sha256; cv=none; b=sbofJUbYqWvqUrMERCQmPpkZayRL37zjAKctE1DglUtbroMOtcXB3ev4JMUnDl62nXxoxs NttMvCG5e8xgX0/ODhUZukeFkxPus+bePSENVoxjR56uOu1bb7UYH9B0U7wlSy0LJhJAgJ ettKhdA5cLGpzGIH9vK01WvF32RhV9s= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=2fdVLCFL; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf24.hostedemail.com: domain of 33aPdZwsKCOAUQWTCXRCPFCIQQING.EQONKPWZ-OOMXCEM.QTI@flex--souravpanda.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=33aPdZwsKCOAUQWTCXRCPFCIQQING.EQONKPWZ-OOMXCEM.QTI@flex--souravpanda.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742578655; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KD+/AzOE/8DE108iYCdGt2BwCB9Yw4lZbr5znCcmPD8=; b=3v/tDvVqOmy7mBrFnl9HGZO0LKTuXrUWyyBFIUdQjhOQhvfdR0bxQcw0/ZINDAl0cov6cW FKXZW+QEHmCEI0YIGJmpwqD/bYbuWnqW25qFlFiut4GUsAMgWY5/MRkCFV88AVFaxoQ4uc 1G+x9soS0o0hJyKLfpBmqlJYmpLGMNA= Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2ff798e8c90so3535041a91.1 for ; Fri, 21 Mar 2025 10:37:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742578654; x=1743183454; darn=kvack.org; h=to:from:subject:message-id:references:mime-version:in-reply-to:date :from:to:cc:subject:date:message-id:reply-to; bh=KD+/AzOE/8DE108iYCdGt2BwCB9Yw4lZbr5znCcmPD8=; b=2fdVLCFLUEHE6dHYXgpc1wP+RrA5OLOjLfl+ljOPyuULywTL4xmYuZ2wnIPL4s4D1v 0r9hvKqwCX8pmY2KcDCr4cqT53Z/bd71rrdAWGIUvq+8CmtQqKQG8U+ZumGcT+5bZqud BOSK2X2gm60LAyxKJXiatGhtm+A9kpG5PiblrHCEbOGwjiLEWv6oNk6UTRRR1Jmu0i31 0V/r3aa1+T0oOw7CHeVsQU6xhnM3EqkKzg+9MIJQl5df8iTwwzAKQyJORzzNektWInTD 0xYvyPK4W4noV/otR1nv0TuylSMcLpQQGb8kCXYoMzoxrH+LaxWC2vUB2pFWl6dpU8lN SSRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742578654; x=1743183454; h=to:from:subject:message-id:references:mime-version:in-reply-to:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KD+/AzOE/8DE108iYCdGt2BwCB9Yw4lZbr5znCcmPD8=; b=eUkbLf42uAk7V/2LtxNLbH21ov2FNaT3d9um6C/APfIv2SFwgIisdWLZ07WoAkbJSq NIc6HRFyDBYD/r82l7AC9Au4nJFNdfUNa+ijz92KyDfMl6hhWoxWNp0/3N2FF/i2GxIR lWOf9YgDHAp3MjY5BBF9+ylh1dKvJlL89RT4WDFIc2t65thQZdPSt59+2tMM+GkIB8SY UAsBfcqU5kNa/Jg6jLH8e5hX35eArd4FAcbDYpTeKvUgXACB70jjCgjLrEIeNoGC0A6j XaDVbnVG3Ppk4SemD5tH/IOg3gJaJOgFcUD13d14R7RSn0IO/jXutaHnW+MYhBx9h/cu 1E+w== X-Forwarded-Encrypted: i=1; AJvYcCXAowoGfqDy2iXi4s4l+3ZlD/NZUaLRFHFCi4DKXSqQ8QCe1xKVAEJUbx4j5SIOAwxbqnyL/Ms10A==@kvack.org X-Gm-Message-State: AOJu0Ywh6E7evXNWcS16oPZMU76jtrwitAZWQBEya/s5ovW4281ogb7K LcxWtZ5Zh7U14ZR393SJmrkXG2iDLNh1sfE6Fjc+ZuJPhfKoX0m7IPNRmNaIxJwUkrNjuwkj+Zi Kr+dU33mVwe9R/GDwSpTzug== X-Google-Smtp-Source: AGHT+IE3zfkzVNg9SOyEjNSGlTIDNM2s5rLvniscEkew+nzR6ifKeKx8uNXSoO7cH2YP2/p20l9xr1Homzd8hizqww== X-Received: from pjbee11.prod.google.com ([2002:a17:90a:fc4b:b0:2ea:46ed:5d3b]) (user=souravpanda job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4b10:b0:2ff:796b:4d05 with SMTP id 98e67ed59e1d1-3030fea7630mr6796934a91.11.1742578653992; Fri, 21 Mar 2025 10:37:33 -0700 (PDT) Date: Fri, 21 Mar 2025 17:37:25 +0000 In-Reply-To: <20250321173729.3175898-1-souravpanda@google.com> Mime-Version: 1.0 References: <20250321173729.3175898-1-souravpanda@google.com> X-Mailer: git-send-email 2.49.0.395.g12beb8f557-goog Message-ID: <20250321173729.3175898-3-souravpanda@google.com> Subject: [RFC PATCH 2/6] mm: make Selective KSM synchronous From: Sourav Panda To: mathieu.desnoyers@efficios.com, willy@infradead.org, david@redhat.com, pasha.tatashin@soleen.com, rientjes@google.com, akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, weixugc@google.com, gthelen@google.com, souravpanda@google.com, surenb@google.com X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 23D86180011 X-Stat-Signature: tumg9t3hrmup3g7cij9hiaankifzpopt X-Rspam-User: X-HE-Tag: 1742578654-791062 X-HE-Meta: U2FsdGVkX18qibtQV+pnmww4x4ip0QqB9+CdumKTiZpE7tLsdydBHoiaPVPtXviELrDMyx2k8Q3jDdVyOPCypnjGZnW9yNdSxIpI0XRjdUZgknyxKsjg4sJqe4UGO99x755l2nRDCMjEJQ41BMvRFTnwmso5+MIuN4tZPSrejITVnohNcGGnc/Ajh1gjn9hU6Ewkre+LquO13EciJVg06jXh7qXj3EwKMaQpn6E3ABi7sH/qPFpuk4Cc3NDm1u4BanZdh4JsIfjGlg25EbHeI9ARACSKynL64YJ8q/hZLC0uF8mkflqUZ2LrOnk/aLqEdp5qiFomtuTyfpXhZ7iR4PsMChMGkfvPTlFp+scfK76gKAquFyyrjksDn9mL5V8/eT47E2dLzy1Ndg7yOpEtymO6Lh7P/3yJuQc7hs+gTrBddSgNQriuCHg+Y7Xjm3Hg1nZrGyug/lSlcHO69cZOvo8KhVZrCRa/QeHoHRkakihN8nR2OJ5XhTVWQXx7tjQDuRFauEcIUMsvYg5zHS71jI2FMfY4xMTxAS4r/et1MdSHqFy9ghqpzfMAhjKPHGqZGu0UYZMqZnIZE/pdLBsRuUqXQeNLmxsmT7iFWtOeGtE/OGsg0h0YhvMiQ2OfQXwNKHK9/eHvfkflFVA13tbz7gMpgJIgkgsMcYUodGo8WkwQDXpZ7K6bRzHOr/Q/6gJvV8xCBWJ83jI0Y1xIpW1y/Nkw++nD3Qvh1/3BQYFoI2U3XM2ARSRI9hMHaJdI01XtyXLcpLEzoJsBVyVizu73CrfN9LCUbXJZEQ78fQY1ancdYHARVJuEObty2wHVWtlJOf94QduCPVIFl3mI6hR3/34O3cIDt6smfkS+65/Iq8rHgkyGGE6ZdTib3sXtuTbyh95u0WBs8OYl3OvE7a/t5g/+QcePqUMtpcTNCPUI6X7f1hKwHg1x1Hy1mkECjw7zSSrf8gm9aj9bpwLH8q9 F65MQC8C Pt1PKWXDkbVPGneWGgS/V4ltPhdSph5vSNhjWsUlPI6QjCulv8sh1FCcfcURwNWi8KQ06lfa0FjY7qnxDSY/kXZFAaBSItopCYITYEeEIEBUTZio61d8gJvkWng82pEk19VkuIoZmcQWWWYUUi/VI9NbYfX68+vAbYYTTx2FWRiIHR4ZxNfNr5sjsvA6riLhuBdGkIgopj/WCmp1TbDte5qd5ZhBrkBQtXohB1TiPXXUOd3RUKGuIYF4nhgoDX8y1Fyfz6JOJhJfVepFfoxf7fxbSiPWE39djf3oRAQe95mlGaCKvc33Cnn8DIaiMLRwjHld0889Dgea+rdpJ5Z88l3A1C2vpMk0HVmJJX5LpL7OZBu8K/LaHfgRc0uGasWBdYqhhOXUdeM9VO6zIEkTdp9iBTwBksQkqRPE3xWNVl5eY4/lr7k7tgcUGR3Jmq3l82Mue4thaNJTzxJOoYcqk4pcxjpNKTwyc0XVD6XEyD9aCVeEdgkmcbBcQv5X29FLdJpqHb5qwOXw/3iTRwK1Rr6ZByx6d9kabzPMHg47Rvw/wiyPzDshGCV3jr6OSaf72gHJgYJ8QHw+KwUo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Make KSM synchronous by introducing the following sysfs file, which shall carryout merging on the specified memory region synchronously and eliminates the need of ksmd running in the background. echo "pid start_addr end_addr" > /sys/kernel/mm/ksm/trigger_merge Signed-off-by: Sourav Panda --- mm/ksm.c | 317 +++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 271 insertions(+), 46 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 8be2b144fefd..b2f184557ed9 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -290,16 +290,18 @@ static unsigned int zero_checksum __read_mostly; /* Whether to merge empty (zeroed) pages with actual zero pages */ static bool ksm_use_zero_pages __read_mostly; -/* Skip pages that couldn't be de-duplicated previously */ -/* Default to true at least temporarily, for testing */ -static bool ksm_smart_scan = true; - /* The number of zero pages which is placed by KSM */ atomic_long_t ksm_zero_pages = ATOMIC_LONG_INIT(0); /* The number of pages that have been skipped due to "smart scanning" */ static unsigned long ksm_pages_skipped; +#ifndef CONFIG_SELECTIVE_KSM /* advisor immaterial if there is no scanning */ + +/* Skip pages that couldn't be de-duplicated previously */ +/* Default to true at least temporarily, for testing */ +static bool ksm_smart_scan = true; + /* Don't scan more than max pages per batch. */ static unsigned long ksm_advisor_max_pages_to_scan = 30000; @@ -465,6 +467,7 @@ static void advisor_stop_scan(void) if (ksm_advisor == KSM_ADVISOR_SCAN_TIME) scan_time_advisor(); } +#endif /* CONFIG_SELECTIVE_KSM */ #ifdef CONFIG_NUMA /* Zeroed when merging across nodes is not allowed */ @@ -957,6 +960,25 @@ static struct folio *ksm_get_folio(struct ksm_stable_node *stable_node, return NULL; } +static unsigned char get_rmap_item_age(struct ksm_rmap_item *rmap_item) +{ +#ifdef CONFIG_SELECTIVE_KSM /* age is immaterial in selective ksm */ + return 0; +#else + unsigned char age; + /* + * Usually ksmd can and must skip the rb_erase, because + * root_unstable_tree was already reset to RB_ROOT. + * But be careful when an mm is exiting: do the rb_erase + * if this rmap_item was inserted by this scan, rather + * than left over from before. + */ + age = (unsigned char)(ksm_scan.seqnr - rmap_item->address); + WARN_ON_ONCE(age > 1); + return age; +#endif /* CONFIG_SELECTIVE_KSM */ +} + /* * Removing rmap_item from stable or unstable tree. * This function will clean the information from the stable/unstable tree. @@ -991,16 +1013,7 @@ static void remove_rmap_item_from_tree(struct ksm_rmap_item *rmap_item) rmap_item->address &= PAGE_MASK; } else if (rmap_item->address & UNSTABLE_FLAG) { - unsigned char age; - /* - * Usually ksmd can and must skip the rb_erase, because - * root_unstable_tree was already reset to RB_ROOT. - * But be careful when an mm is exiting: do the rb_erase - * if this rmap_item was inserted by this scan, rather - * than left over from before. - */ - age = (unsigned char)(ksm_scan.seqnr - rmap_item->address); - BUG_ON(age > 1); + unsigned char age = get_rmap_item_age(rmap_item); if (!age) rb_erase(&rmap_item->node, root_unstable_tree + NUMA(rmap_item->nid)); @@ -2203,6 +2216,37 @@ static void stable_tree_append(struct ksm_rmap_item *rmap_item, rmap_item->mm->ksm_merging_pages++; } +#ifdef CONFIG_SELECTIVE_KSM +static int update_checksum(struct page *page, struct ksm_rmap_item *rmap_item) +{ + /* + * Typically KSM would wait for a second round to even consider + * the page for unstable tree insertion to ascertain its stability. + * Avoid this when using selective ksm. + */ + rmap_item->oldchecksum = calc_checksum(page); + return 0; +} +#else +static int update_checksum(struct page *page, struct ksm_rmap_item *rmap_item) +{ + remove_rmap_item_from_tree(rmap_item); + + /* + * If the hash value of the page has changed from the last time + * we calculated it, this page is changing frequently: therefore we + * don't want to insert it in the unstable tree, and we don't want + * to waste our time searching for something identical to it there. + */ + checksum = calc_checksum(page); + if (rmap_item->oldchecksum != checksum) { + rmap_item->oldchecksum = checksum; + return -EINVAL; + } + return 0; +} +#endif + /* * cmp_and_merge_page - first see if page can be merged into the stable tree; * if not, compare checksum to previous and if it's the same, see if page can @@ -2218,7 +2262,6 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite struct page *tree_page = NULL; struct ksm_stable_node *stable_node; struct folio *kfolio; - unsigned int checksum; int err; bool max_page_sharing_bypass = false; @@ -2241,20 +2284,8 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite if (!is_page_sharing_candidate(stable_node)) max_page_sharing_bypass = true; } else { - remove_rmap_item_from_tree(rmap_item); - - /* - * If the hash value of the page has changed from the last time - * we calculated it, this page is changing frequently: therefore we - * don't want to insert it in the unstable tree, and we don't want - * to waste our time searching for something identical to it there. - */ - checksum = calc_checksum(page); - if (rmap_item->oldchecksum != checksum) { - rmap_item->oldchecksum = checksum; + if (update_checksum(page, rmap_item)) return; - } - if (!try_to_merge_with_zero_page(rmap_item, page)) return; } @@ -2379,6 +2410,111 @@ static struct ksm_rmap_item *get_next_rmap_item(struct ksm_mm_slot *mm_slot, return rmap_item; } +#ifdef CONFIG_SELECTIVE_KSM +static struct ksm_rmap_item *retrieve_rmap_item(struct page **page, + struct mm_struct *mm, + unsigned long start, + unsigned long end) +{ + struct ksm_mm_slot *mm_slot; + struct mm_slot *slot; + struct vm_area_struct *vma; + struct ksm_rmap_item *rmap_item; + struct vma_iterator vmi; + + lru_add_drain_all(); + + if (!ksm_merge_across_nodes) { + struct ksm_stable_node *stable_node, *next; + struct folio *folio; + + list_for_each_entry_safe(stable_node, next, + &migrate_nodes, list) { + folio = ksm_get_folio(stable_node, KSM_GET_FOLIO_NOLOCK); + if (folio) + folio_put(folio); + } + } + + spin_lock(&ksm_mmlist_lock); + slot = mm_slot_lookup(mm_slots_hash, mm); + spin_unlock(&ksm_mmlist_lock); + + if (!slot) + return NULL; + mm_slot = mm_slot_entry(slot, struct ksm_mm_slot, slot); + + ksm_scan.address = 0; + ksm_scan.mm_slot = mm_slot; + ksm_scan.rmap_list = &mm_slot->rmap_list; + + vma_iter_init(&vmi, mm, ksm_scan.address); + + mmap_read_lock(mm); + for_each_vma(vmi, vma) { + if (!(vma->vm_flags & VM_MERGEABLE)) + continue; + if (ksm_scan.address < vma->vm_start) + ksm_scan.address = vma->vm_start; + if (!vma->anon_vma) + ksm_scan.address = vma->vm_end; + + while (ksm_scan.address < vma->vm_end) { + struct page *tmp_page = NULL; + struct folio_walk fw; + struct folio *folio; + + if (ksm_scan.address < start || ksm_scan.address > end) + break; + + folio = folio_walk_start(&fw, vma, ksm_scan.address, 0); + if (folio) { + if (!folio_is_zone_device(folio) && + folio_test_anon(folio)) { + folio_get(folio); + tmp_page = fw.page; + } + folio_walk_end(&fw, vma); + } + + if (tmp_page) { + flush_anon_page(vma, tmp_page, ksm_scan.address); + flush_dcache_page(tmp_page); + rmap_item = get_next_rmap_item(mm_slot, + ksm_scan.rmap_list, + ksm_scan.address); + if (rmap_item) { + ksm_scan.rmap_list = + &rmap_item->rmap_list; + ksm_scan.address += PAGE_SIZE; + *page = tmp_page; + } else { + folio_put(folio); + } + mmap_read_unlock(mm); + return rmap_item; + } + ksm_scan.address += PAGE_SIZE; + } + } + mmap_read_unlock(mm); + return NULL; +} + +static void ksm_sync_merge(struct mm_struct *mm, + unsigned long start, unsigned long end) +{ + struct ksm_rmap_item *rmap_item; + struct page *page; + + rmap_item = retrieve_rmap_item(&page, mm, start, end); + if (!rmap_item) + return; + cmp_and_merge_page(page, rmap_item); + put_page(page); +} + +#else /* CONFIG_SELECTIVE_KSM */ /* * Calculate skip age for the ksm page age. The age determines how often * de-duplicating has already been tried unsuccessfully. If the age is @@ -2688,6 +2824,7 @@ static int ksm_scan_thread(void *nothing) } return 0; } +#endif /* CONFIG_SELECTIVE_KSM */ static void __ksm_add_vma(struct vm_area_struct *vma) { @@ -3335,9 +3472,10 @@ static ssize_t pages_to_scan_store(struct kobject *kobj, unsigned int nr_pages; int err; +#ifndef CONFIG_SELECTIVE_KSM if (ksm_advisor != KSM_ADVISOR_NONE) return -EINVAL; - +#endif err = kstrtouint(buf, 10, &nr_pages); if (err) return -EINVAL; @@ -3396,6 +3534,65 @@ static ssize_t run_store(struct kobject *kobj, struct kobj_attribute *attr, } KSM_ATTR(run); +static ssize_t trigger_merge_show(struct kobject *kobj, + struct kobj_attribute *attr, + char *buf) +{ + return -EINVAL; /* Not yet implemented */ +} + +static ssize_t trigger_merge_store(struct kobject *kobj, + struct kobj_attribute *attr, + const char *buf, size_t count) +{ + unsigned long start, end; + pid_t pid; + char *input, *ptr; + int ret; + struct task_struct *task; + struct mm_struct *mm; + + input = kstrdup(buf, GFP_KERNEL); + if (!input) + return -ENOMEM; + + ptr = strim(input); + ret = sscanf(ptr, "%d %lx %lx", &pid, &start, &end); + kfree(input); + + if (ret != 3) + return -EINVAL; + + if (start >= end) + return -EINVAL; + + /* Find the mm_struct */ + rcu_read_lock(); + task = find_task_by_vpid(pid); + if (!task) { + rcu_read_unlock(); + return -ESRCH; + } + + get_task_struct(task); + + rcu_read_unlock(); + mm = get_task_mm(task); + put_task_struct(task); + + if (!mm) + return -EINVAL; + + mutex_lock(&ksm_thread_mutex); + wait_while_offlining(); + ksm_sync_merge(mm, start, end); + mutex_unlock(&ksm_thread_mutex); + + mmput(mm); + return count; +} +KSM_ATTR(trigger_merge); + #ifdef CONFIG_NUMA static ssize_t merge_across_nodes_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) @@ -3635,6 +3832,7 @@ static ssize_t full_scans_show(struct kobject *kobj, } KSM_ATTR_RO(full_scans); +#ifndef CONFIG_SELECTIVE_KSM static ssize_t smart_scan_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) { @@ -3780,11 +3978,13 @@ static ssize_t advisor_target_scan_time_store(struct kobject *kobj, return count; } KSM_ATTR(advisor_target_scan_time); +#endif /* CONFIG_SELECTIVE_KSM */ static struct attribute *ksm_attrs[] = { &sleep_millisecs_attr.attr, &pages_to_scan_attr.attr, &run_attr.attr, + &trigger_merge_attr.attr, &pages_scanned_attr.attr, &pages_shared_attr.attr, &pages_sharing_attr.attr, @@ -3802,12 +4002,14 @@ static struct attribute *ksm_attrs[] = { &stable_node_chains_prune_millisecs_attr.attr, &use_zero_pages_attr.attr, &general_profit_attr.attr, +#ifndef CONFIG_SELECTIVE_KSM &smart_scan_attr.attr, &advisor_mode_attr.attr, &advisor_max_cpu_attr.attr, &advisor_min_pages_to_scan_attr.attr, &advisor_max_pages_to_scan_attr.attr, &advisor_target_scan_time_attr.attr, +#endif NULL, }; @@ -3815,40 +4017,63 @@ static const struct attribute_group ksm_attr_group = { .attrs = ksm_attrs, .name = "ksm", }; + +static int __init ksm_sysfs_init(void) +{ + return sysfs_create_group(mm_kobj, &ksm_attr_group); +} +#else /* CONFIG_SYSFS */ +static int __init ksm_sysfs_init(void) +{ + ksm_run = KSM_RUN_MERGE; /* no way for user to start it */ + return 0; +} #endif /* CONFIG_SYSFS */ -static int __init ksm_init(void) +#ifdef CONFIG_SELECTIVE_KSM +static int __init ksm_thread_sysfs_init(void) +{ + return ksm_sysfs_init(); +} +#else /* CONFIG_SELECTIVE_KSM */ +static int __init ksm_thread_sysfs_init(void) { struct task_struct *ksm_thread; int err; - /* The correct value depends on page size and endianness */ - zero_checksum = calc_checksum(ZERO_PAGE(0)); - /* Default to false for backwards compatibility */ - ksm_use_zero_pages = false; - - err = ksm_slab_init(); - if (err) - goto out; - ksm_thread = kthread_run(ksm_scan_thread, NULL, "ksmd"); if (IS_ERR(ksm_thread)) { pr_err("ksm: creating kthread failed\n"); err = PTR_ERR(ksm_thread); - goto out_free; + return err; } -#ifdef CONFIG_SYSFS - err = sysfs_create_group(mm_kobj, &ksm_attr_group); + err = ksm_sysfs_init(); if (err) { pr_err("ksm: register sysfs failed\n"); kthread_stop(ksm_thread); - goto out_free; } -#else - ksm_run = KSM_RUN_MERGE; /* no way for user to start it */ -#endif /* CONFIG_SYSFS */ + return err; +} +#endif /* CONFIG_SELECTIVE_KSM */ + +static int __init ksm_init(void) +{ + int err; + + /* The correct value depends on page size and endianness */ + zero_checksum = calc_checksum(ZERO_PAGE(0)); + /* Default to false for backwards compatibility */ + ksm_use_zero_pages = false; + + err = ksm_slab_init(); + if (err) + goto out; + + err = ksm_thread_sysfs_init(); + if (err) + goto out_free; #ifdef CONFIG_MEMORY_HOTREMOVE /* There is no significance to this priority 100 */