From patchwork Fri Feb 10 01:15:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Yang X-Patchwork-Id: 13135328 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA4B4C05027 for ; Fri, 10 Feb 2023 01:15:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CCC4F6B00C0; Thu, 9 Feb 2023 20:15:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C54816B00C1; Thu, 9 Feb 2023 20:15:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF5596B00C2; Thu, 9 Feb 2023 20:15:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 984F16B00C0 for ; Thu, 9 Feb 2023 20:15:35 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 5D2E2AB8FC for ; Fri, 10 Feb 2023 01:15:35 +0000 (UTC) X-FDA: 80449614630.17.3051F4C Received: from mxct.zte.com.cn (mxct.zte.com.cn [183.62.165.209]) by imf06.hostedemail.com (Postfix) with ESMTP id 509EA180009 for ; Fri, 10 Feb 2023 01:15:31 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=none; spf=pass (imf06.hostedemail.com: domain of yang.yang29@zte.com.cn designates 183.62.165.209 as permitted sender) smtp.mailfrom=yang.yang29@zte.com.cn; dmarc=pass (policy=none) header.from=zte.com.cn ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675991733; a=rsa-sha256; cv=none; b=AbI+FqFk5Wstd41D5G2ceLPiw2AGHX/8EmV/RRqiRSIyi68cDCNC2q+96JJqBK+cL6uRKz 09vtxSAeKEdcUNVVaipQ1O+anKN7tmchsaai/mwJynhGOmGcUiDrPtQeJXR0/UTgpddN50 v5BtWJBoUlIfd6+q2Hfq4y7VT5uPkig= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=none; spf=pass (imf06.hostedemail.com: domain of yang.yang29@zte.com.cn designates 183.62.165.209 as permitted sender) smtp.mailfrom=yang.yang29@zte.com.cn; dmarc=pass (policy=none) header.from=zte.com.cn ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675991733; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references; bh=ZRCWa5VcbtTMd93vzJE230tM5ULf6+8N5VXoD1UdbY8=; b=P6U2o1mn2JY1Xs/aNdrMApC37oRj0HlL9k9hs0Uj9REQiERs+M+S3cWz5iWXFmEdQoKDzy G7cI46Bgcy/nIejXKDJeZC1CDMdgFkPdR44pJS5Ea2Ia6pX87TKB6dr1vAhM11dGx+aJOG NOO71gpRayQyq24H2B2cdAzl5bwkAu0= Received: from mse-fl1.zte.com.cn (unknown [10.5.228.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mxct.zte.com.cn (FangMail) with ESMTPS id 4PCbQ81jJRz501Qm; Fri, 10 Feb 2023 09:15:28 +0800 (CST) Received: from szxlzmapp01.zte.com.cn ([10.5.231.85]) by mse-fl1.zte.com.cn with SMTP id 31A1FLFN006412; Fri, 10 Feb 2023 09:15:21 +0800 (+08) (envelope-from yang.yang29@zte.com.cn) Received: from mapi (szxlzmapp01[null]) by mapi (Zmail) with MAPI id mid14; Fri, 10 Feb 2023 09:15:22 +0800 (CST) Date: Fri, 10 Feb 2023 09:15:22 +0800 (CST) X-Zmail-TransId: 2b0363e59aaafffffffff5e3bbcc X-Mailer: Zmail v1.0 Message-ID: <202302100915227721315@zte.com.cn> Mime-Version: 1.0 From: To: Cc: , , , , , , , , Subject: =?utf-8?q?=5BPATCH_v6_0/6=5D_ksm=3A_support_tracking_KSM-placed_zer?= =?utf-8?q?o-pages?= X-MAIL: mse-fl1.zte.com.cn 31A1FLFN006412 X-Fangmail-Gw-Spam-Type: 0 X-Fangmail-Anti-Spam-Filtered: true X-Fangmail-MID-QID: 63E59AB0.000/4PCbQ81jJRz501Qm X-Rspam-User: X-Rspamd-Queue-Id: 509EA180009 X-Rspamd-Server: rspam01 X-Stat-Signature: jfcoezbi1es7rgksmrog8ozbdjyft35w X-HE-Tag: 1675991731-503082 X-HE-Meta: U2FsdGVkX19tC7Jy6NCL1muJs88L8kOlBeyQNDNzSk2MCf0WC7WslzwvhLH+WcTkooyM5ndzb2PxtMB8lU/ZAJuUrU0CWRPV3jR5kHtYmmoeqLvzrVSjBcHkmP+ROb+IZzR7bmzKodrBdKU7bC5fsOx21YKZGW/pwtG6lqQgUVqliOy3Nx1FBgmDdR85KeuxlPxrH0ELEdfxS1wdNEkjOBI2k1bMs6QqXP9BolLjrpPOBEZD+ql0iGxRsb3JDY8uwKmMu7gCqMRNhDzSGYol7uG0TlF0dLSdWMpme8FHEW4PPS3y0AAub/ZlrrTJrXSEjA4A56yZKn9PsSpGV47h64F4GlxnzoMDMoTPQOViGwrUFhDi+IUkttcGu2blAHdIOFIErweXZUVhRS50EhWxpjgdMt6xbGs0upVc8pVeKB8um5Sfq7f1rKKVYd49LeyhWosQQJEfJCnyNDZhmRVPYxJ7evIYt4MebNC9+mAzG6NFjN4wlBf5Kvk/6YOxSYh8UGYfixLMuYFtxbIIoDljk3nPKSKOxPCTgkT78fAkLWXtjLZd12wM5aHyk/X35BMOBhW9itoh5wEMt2rvSz80NY1wuHhTmRNVyJmgwB94hfbAHEoA1FvOtORZ/6cB9fYJHu3ZdW55+dmpVZrXhTliKXfRb3+LQfR2AcDXIRgntZ8PlLQmy5zSHbnI5+tCtrmtilDlU7A38d8eFKB2TzaxWPmyB6mUgdmmRCMcfDI7zKTLfXI70Ys0J+oyE0iWpbui8PGU1G6brN3qH9nVyJtF218WS26M0pS6TM2xLm6wdFV83WpKZFNDQJFWVC37QzgNPoFBNKyEgXDxbs2nBy5Hww8/Krh1zWXWkgVW1QTcTEOTFFctnYrHKhuNPWgXgtuWX5shNKpSdyEhZZ5TO5XOFJWyJKkJLDzwTWIDHgCe0t/rp8xpBBl7T3i8pFX8bZ01p2P7ZaRhVxLUqA28zN8 kTkBhkf0 QEaOxm4NlwdUsb/DIDts/ewFA0uAJSqBHL1PtjE3dDC688ZrfA0/mIYJD2V3tGTNP+8fXgNcK3l8qin3QTOy6jyW6pHbN69jQpHs2h3YkmW9cX/MaLv9UZ1W7VZHf+GC1W9S6V44IEdbb2Pgo/UfogC/dWoFijCvgEzzBE66/WIhbVdUlH15dvPmvO3Up3es1SDd6JYfcHA40ZIkLK6zUhHAtSc1RelXMKXCw3kSAUeGN2mbAI41qtaUEGj6jDge9/3NdhDEVVWTWA43Wr9wT4JwxZugsS8+Ejjykc9v7dMgbzYJ/oE5GziqiM861WY1YAJXeWQEHoc4K8hcRZLb5PEioYY8M6CKeaTX2StsETSzoetDbY2n0KRx6DmeEmuMMdOObzwe085ftLpw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin The core idea of this patch set is to enable users to perceive the number of any pages merged by KSM, regardless of whether use_zero_page switch has been turned on, so that users can know how much free memory increase is really due to their madvise(MERGEABLE) actions. But the problem is, when enabling use_zero_pages, all empty pages will be merged with kernel zero pages instead of with each other as use_zero_pages is disabled, and then these zero-pages are no longer monitored by KSM. The motivations for me to do this contains three points: 1) MADV_UNMERGEABLE and other ways to trigger unsharing will *not* unshare the shared zeropage as placed by KSM (which is against the MADV_UNMERGEABLE documentation at least); see the link: https://lore.kernel.org/lkml/4a3daba6-18f9-d252-697c-197f65578c44@redhat.com/ 2) We cannot know how many pages are zero pages placed by KSM when enabling use_zero_pages, which hides the critical information about how much actual memory are really saved by KSM. Knowing how many ksm-placed zero pages are helpful for user to use the policy of madvise (MERGEABLE) better because they can see the actual profit brought by KSM. 3) The zero pages placed-by KSM are different from those initial empty page (filled with zeros) which are never touched by applications. The former is active-merged by KSM while the later have never consume actual memory. use_zero_pages is useful, not only because of cache colouring as described in doc, but also because use_zero_pages can accelerate merging empty pages when there are plenty of empty pages (full of zeros) as the time of page-by-page comparisons (unstable_tree_search_insert) is saved. So we hope to implement the support for ksm zero page tracking without affecting the feature of use_zero_pages. Zero pages may be the most common merged pages in actual environment(not only VM but also including other application like containers). Enabling use_zero_pages in the environment with plenty of empty pages(full of zeros) will be very useful. Users and app developer can also benefit from knowing the proportion of zero pages in all merged pages to optimize applications. With the patch series, we can both unshare zero-pages(KSM-placed) accurately and count ksm zero pages with enabling use_zero_pages. v5->v6: 1) In [PATCH 1/6], fix some coments as Divid's suggestions [1]. 2) In [PATCH 6/6], modify the patch as Divid's suggestions [2]. [1]: https://lore.kernel.org/lkml/0e0c90a2-d12c-f965-9cce-ecd5d28c09dd@redhat.com/ [2]: https://lore.kernel.org/lkml/3704dcf0-bd0a-8ab2-7f4f-045fc7c73171@redhat.com/ v4->v5: --- 1) fix warning: mm/ksm.c:3238:9: warning: no previous prototype for 'zero_pages_sharing_show' [-Wmissing-prototypes]. In [PATCH 3/6] (ksm: count all zero pages placed by KSM), declare the function ssize_t zero_pages_sharing_show(struct kobject *kobj...) as 'static'. 2) In [PATCH 6/6],fix error of "} while (end_scans < start_scans + 20);" to "} while (end_scans < start_scans + 2);" in wait_two_full_scans(). --- v3->v4: 1) The patch series are readjusted to adapt to these recent changes of break_ksm() from David Hildenbrand's commits: https://lore.kernel.org/all/20221021101141.84170-9-david@redhat.com/T/#u 2) Some changes of patch itself: In [patch 2/6], add a check of mm exiting in unshare_zero_pages in case of unsharing the zero pages whose process is exiting; form a new function clean_rmap_item_zero_flag(), and add it after stable_tree_search() to fix; In [patch 3/6], all decreasing actions of zero pages count are put in clean_rmap_item_zero_flag(), which is more accurate. 3) Add a selftest of unsharing and counting ksm-placed zero pages. --- v2->v3: 1) Add more descriptive information in cover letter. 2) In [patch 2/5], add more commit log for explaining reasons. 3) In [patch 2/5], fix misuse of break_ksm() in unmerge_ksm_pages(): break_ksm(vma, addr, NULL) -> break_ksm(vma, addr, false); --- v1->v2: [patch 4/5] fix build warning, mm/ksm.c:550, misleading indentation; statement 'rmap_item->mm->ksm_zero_pages_sharing--;' is not part of the previous 'if'. *** BLURB HERE *** xu xin (6): ksm: abstract the function try_to_get_old_rmap_item ksm: support unsharing zero pages placed by KSM ksm: count all zero pages placed by KSM ksm: count zero pages for each process ksm: add zero_pages_sharing documentation selftest: add testing unsharing and counting ksm zero page Documentation/admin-guide/mm/ksm.rst | 7 + fs/proc/base.c | 1 + include/linux/mm_types.h | 7 +- mm/ksm.c | 185 +++++++++++++++++----- tools/testing/selftests/vm/ksm_functional_tests.c | 96 ++++++++++- 5 files changed, 255 insertions(+), 41 deletions(-)