From patchwork Fri Feb 10 01:16:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Yang X-Patchwork-Id: 13135329 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D31C9C05027 for ; Fri, 10 Feb 2023 01:16:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 307396B00C1; Thu, 9 Feb 2023 20:16:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 290506B00C3; Thu, 9 Feb 2023 20:16:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 159E86B00C4; Thu, 9 Feb 2023 20:16:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 04CAD6B00C1 for ; Thu, 9 Feb 2023 20:16:52 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id C5F621C5D2F for ; Fri, 10 Feb 2023 01:16:51 +0000 (UTC) X-FDA: 80449617822.01.22828D7 Received: from mxhk.zte.com.cn (mxhk.zte.com.cn [63.216.63.40]) by imf26.hostedemail.com (Postfix) with ESMTP id 271B8140005 for ; Fri, 10 Feb 2023 01:16:48 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of yang.yang29@zte.com.cn designates 63.216.63.40 as permitted sender) smtp.mailfrom=yang.yang29@zte.com.cn; dmarc=pass (policy=none) header.from=zte.com.cn ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675991809; a=rsa-sha256; cv=none; b=E3sXyef24z4KHXSWjGRCut0Uy8RHK3iRFUwe6TSfU8Ka7w3Kpry2Lih9ljdwG29pXqMP+G LI8/L6jqds+VDVGNoBE5zSn2uElPETOM/31E3ZQ+PZlz/brD8Zqb+rKmG31Ki9EfZyRxU8 mrCw6L38sc+zXbBzcdyltelHuQ5FXCg= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of yang.yang29@zte.com.cn designates 63.216.63.40 as permitted sender) smtp.mailfrom=yang.yang29@zte.com.cn; dmarc=pass (policy=none) header.from=zte.com.cn ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675991809; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references; bh=Go5knzcgdTW3rle4PYI2BccmNgMMlWTFK6bplH5FMRI=; b=1jmyBcj2gwqigtO8EY8KXapTQCV27hiOtfP5ABW8zW69NxcTseofcwQcyxQFabuUStFkAS 8kX11fYP1dFwHp/PdzdpLEgrD07XxiU16e4bCuDAAkYNN1+7Qs5wvgCEp8VmISjHyPpDNn VtubIeMnJe+Ho2Dad7+TZQInuTLvwwk= Received: from mse-fl1.zte.com.cn (unknown [10.5.228.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mxhk.zte.com.cn (FangMail) with ESMTPS id 4PCbRd2v10z8R044; Fri, 10 Feb 2023 09:16:45 +0800 (CST) Received: from szxlzmapp05.zte.com.cn ([10.5.230.85]) by mse-fl1.zte.com.cn with SMTP id 31A1GfQW007483; Fri, 10 Feb 2023 09:16:41 +0800 (+08) (envelope-from yang.yang29@zte.com.cn) Received: from mapi (szxlzmapp01[null]) by mapi (Zmail) with MAPI id mid14; Fri, 10 Feb 2023 09:16:42 +0800 (CST) Date: Fri, 10 Feb 2023 09:16:42 +0800 (CST) X-Zmail-TransId: 2b0363e59afa1ea3fe53 X-Mailer: Zmail v1.0 Message-ID: <202302100916423431376@zte.com.cn> Mime-Version: 1.0 From: To: Cc: , , , , , , , , Subject: =?utf-8?q?=5BPATCH_v6_1/6=5D_ksm=3A_abstract_the_function_try=5Fto?= =?utf-8?q?=5Fget=5Fold=5Frmap=5Fitem?= X-MAIL: mse-fl1.zte.com.cn 31A1GfQW007483 X-Fangmail-Gw-Spam-Type: 0 X-FangMail-Miltered: at cgslv5.04-192.168.250.137.novalocal with ID 63E59AFD.000 by FangMail milter! X-FangMail-Envelope: 1675991805/4PCbRd2v10z8R044/63E59AFD.000/10.5.228.132/[10.5.228.132]/mse-fl1.zte.com.cn/ X-Fangmail-Anti-Spam-Filtered: true X-Fangmail-MID-QID: 63E59AFD.000/4PCbRd2v10z8R044 X-Rspam-User: X-Rspamd-Queue-Id: 271B8140005 X-Rspamd-Server: rspam01 X-Stat-Signature: 6nzj8rp48iwc1tjac8f4ezxjhwrpamsd X-HE-Tag: 1675991808-868535 X-HE-Meta: U2FsdGVkX1+WGszZUf+CqEYa3CKq4IAZS0BUsA3eb2Kvs32RFWGBMWyN1AsvVWUCT5oFqwrKtakMDkzZDfzgAgB0Hw/QO9rdum6Py3MME642sfS63nl6SiWit0jyhJq9tnNxwSNYulNZIlWv4/9Hvrx9KDIP4OxUtWj4clfvTLBnP+bOV+y364kzKqFnyU4b1iC+45KLvYGquIAiNPbEJK5pIJvR7goAeffBOZflVx0r4vCTp2xFdeXWHmzdOtx9jdhOhnUbczFBpFTVcsUqpTT0EtBCoonB684jKXieNJdJD1F8iSVhXjlxkw2KbTkAezk+chuXJCdHZctZja6g7kAQLh00zsDnygqL3YyZ61ZhZB4fXLx76vI4GIkUeJdrSmOp/fKJHZleLp58XdQTM6Km+zDqfTLcR8KOeR933wXTHgwXD3RHj/k62IPKi9MfYDtjKox/eAehQ6cNrOup3b1AkbbqiX1YCTQad4t31V4yD5Epr9rJ5479qQpcurifweOWw9C1hYsFq5QQtSIZn7sxQV4GN8k0hQnF5Zb9iyhbpJbnLD/vJTc4EY3Yq2dL+wQrysJbQFfWw0mWRYccLlCX08YNFTYhuTUW0ILb2fg+WKfwBnhjIr8eSyXVlOkNGZ/9rzTPrFAb7VU84xw+Vb2pDmpf3YwAh85C8TNCXtV9XyOluiiw9UWdkelgOAlV0IRNIuGIBHkEUBg0StaqHtCl8attKnQpCCTlF/tAQvOZ3ozF07TOw/F7UWQW+qDtD8irADPJe9J/6ZiGOquHuvo1944+1r9rUvZ3j4vwXIrkexKJC7GNw17h74fm4X4sDaLRCp6skxKiM7xQdUPMEsD/O6Ext5I8F72HKxqKx+uEhkpw0ftId3NF+v3uT8w234KuyKk8TRiTP3RtIvNY51I53Tbl6aQgPQKMjw5iNVqg8bM3cQ6oyj8PYFDepuGAf9tqUkla4kmtXdE+VN3 sUobxSac Bx+xg/icuzsboqdZlU6s9w77aMq+j5zuyQzkmJkQbJljjO3WgLwumw7LE+pJE5KBRN1I1OpdZzFCRoKKXayPSSp9V2419bE5Ga3KDYV3r+Mzw3M6ED48MVrqaToevhOHfbPJmqn0GBc9BOaSkMXx1lmnsMh6+nkAbAZvQqKeHs2MnABg0rLAIJC8hIeda++bro6OYMt5ERgS8nlhdRK8bL6Seq+3dr0hx/uu2wkdSZ9oTM4heYyZcBY66ET4OVCqEUpoh5ksp2kaAecDSKPHvBBdvJT3IRcqAbJNe2TiCYJ+yI+M2Tua5l/tUWJxSWIWXDL2JzXM6ZH42Y4cKzRn/qVMRYq5gR4QlRjl1hRp+oFcJ/+ypnC7zHdtbqSR4ioEVy+ERZAQrdQPRgPaTfGJlOlyo9A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin A new function try_to_get_old_rmap_item is abstracted from get_next_rmap_item. This function will be reused by the subsequent patches about counting ksm_zero_pages. The patch improves the readability and reusability of KSM code. Signed-off-by: xu xin Cc: David Hildenbrand Cc: Claudio Imbrenda Cc: Xuexin Jiang Reviewed-by: Xiaokai Ran Reviewed-by: Yang Yang v5->v6: Modify some comments according to David's suggestions. --- mm/ksm.c | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 83e2f74ae7da..905a79d213da 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -2214,23 +2214,38 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite } } -static struct ksm_rmap_item *get_next_rmap_item(struct ksm_mm_slot *mm_slot, - struct ksm_rmap_item **rmap_list, - unsigned long addr) +static struct ksm_rmap_item *try_to_get_old_rmap_item(unsigned long addr, + struct ksm_rmap_item **rmap_list) { - struct ksm_rmap_item *rmap_item; - while (*rmap_list) { - rmap_item = *rmap_list; + struct ksm_rmap_item *rmap_item = *rmap_list; + if ((rmap_item->address & PAGE_MASK) == addr) return rmap_item; if (rmap_item->address > addr) break; *rmap_list = rmap_item->rmap_list; + /* + * If we end up here, the VMA is MADV_UNMERGEABLE or its page + * is ineligible or discarded, e.g. MADV_DONTNEED. + */ remove_rmap_item_from_tree(rmap_item); free_rmap_item(rmap_item); } + return NULL; +} + +static struct ksm_rmap_item *get_next_rmap_item(struct ksm_mm_slot *mm_slot, + struct ksm_rmap_item **rmap_list, + unsigned long addr) +{ + struct ksm_rmap_item *rmap_item; + + rmap_item = try_to_get_old_rmap_item(addr, rmap_list); + if (rmap_item) + return rmap_item; + rmap_item = alloc_rmap_item(); if (rmap_item) { /* It has already been zeroed */ From patchwork Fri Feb 10 01:17:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Yang X-Patchwork-Id: 13135330 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D5CAC05027 for ; Fri, 10 Feb 2023 01:18:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B91B86B00C3; Thu, 9 Feb 2023 20:18:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B41D36B00C5; Thu, 9 Feb 2023 20:18:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9E2F56B00C6; Thu, 9 Feb 2023 20:18:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 8C5796B00C3 for ; Thu, 9 Feb 2023 20:18:03 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 50CDBC10E1 for ; Fri, 10 Feb 2023 01:18:03 +0000 (UTC) X-FDA: 80449620846.28.6C81CFF Received: from mxct.zte.com.cn (mxct.zte.com.cn [183.62.165.209]) by imf24.hostedemail.com (Postfix) with ESMTP id 944A6180002 for ; Fri, 10 Feb 2023 01:18:00 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=zte.com.cn; spf=pass (imf24.hostedemail.com: domain of yang.yang29@zte.com.cn designates 183.62.165.209 as permitted sender) smtp.mailfrom=yang.yang29@zte.com.cn ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675991881; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references; bh=WAeLIOqUu6S1p2iujxnPPM5NIajaPHAFmC1QUWCV04g=; b=GvnwxcFtyggA5MSK6JcMCSWKJ3yd2K84lOqwy7gLBXLDx1PzBDDE594FJOp/0c2rPJog9W iyqTn0ijye/slyzj7zr0zFcs6UsrAYJt+CZ9XW7Ivy2ZxFyVZh/O0byefHL3RqTftvXu+B XZdUCrNcGeTG1L0qmHDiuF66uUsfqOk= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=zte.com.cn; spf=pass (imf24.hostedemail.com: domain of yang.yang29@zte.com.cn designates 183.62.165.209 as permitted sender) smtp.mailfrom=yang.yang29@zte.com.cn ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675991881; a=rsa-sha256; cv=none; b=rZoGUKWUMhrB+kuJYa4hcpjQOqOyK9iL4hoUV3/iKKYECHoqqmQrYXJ70Y7YEOe5KSuehm KvGVThYBsCxna/ZTbLXsWXJcRLztw8IbX0F7ioGYIwu6p2i8IVbETC6eaRaav/KT7Wnxt+ RO72yy0l8dtkq6lmizPC1JjPtUzl3D0= Received: from mse-fl2.zte.com.cn (unknown [10.5.228.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mxct.zte.com.cn (FangMail) with ESMTPS id 4PCbT15pgxz501Ql; Fri, 10 Feb 2023 09:17:57 +0800 (CST) Received: from szxlzmapp04.zte.com.cn ([10.5.231.166]) by mse-fl2.zte.com.cn with SMTP id 31A1HojB022735; Fri, 10 Feb 2023 09:17:50 +0800 (+08) (envelope-from yang.yang29@zte.com.cn) Received: from mapi (szxlzmapp01[null]) by mapi (Zmail) with MAPI id mid14; Fri, 10 Feb 2023 09:17:51 +0800 (CST) Date: Fri, 10 Feb 2023 09:17:51 +0800 (CST) X-Zmail-TransId: 2b0363e59b3fffffffffe0043a66 X-Mailer: Zmail v1.0 Message-ID: <202302100917515661425@zte.com.cn> Mime-Version: 1.0 From: To: Cc: , , , , , , , , Subject: =?utf-8?q?=5BPATCH_v6_2/6=5D_ksm=3A_support_unsharing_zero_pages_pl?= =?utf-8?q?aced_by_KSM?= X-MAIL: mse-fl2.zte.com.cn 31A1HojB022735 X-Fangmail-Gw-Spam-Type: 0 X-Fangmail-Anti-Spam-Filtered: true X-Fangmail-MID-QID: 63E59B45.001/4PCbT15pgxz501Ql X-Rspamd-Queue-Id: 944A6180002 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: dxsm7n61q1y61nuuqjqx9h9sn9bjwfn8 X-HE-Tag: 1675991880-41736 X-HE-Meta: U2FsdGVkX18AvysXgXst4aTUKIU4wo1wLw0go++Gc0887Kntja+Wyr5d98O0k83Lf9PTXsFjTA8yygINcWGlIyJQ0iVcooW7yRGDo6Z4k9SgofCaG+9J9CebfOCAYMh5U+Lql499LtTZ7GQ52Y4g7nvrn1OUcfKlTEWhoFB2JWsKXZOLfHsNw8K0NUkytLrxGG2jfYDTdNUC1id9kMPvKSdEz8vtD0mR13F01WPL4beGfNN6JiqnkBAstCllk74FxY/rwzY6zewUaWmZYAxIx5NFpeKOSZj1ju5OqdEzPiXIzFPA0l7KxN1P1dxywa7kgKm1h8+tuFia+OWfgB+aHM8XOCP8NJsoOkzquCjBWqxtSXHSITbxIu5gJDpYe6xBO9Vzu5k7Waphpl3xRNXL3BElzBM68Rx62Bao2YV91NbUw2PpuN3bJYjGpKmoZ4EkW29kb9g+0JIxCUKOTf4/B8rcOYER8zIRsf9ws7rj0Rm2LQ/Gv861uExgp7OT8Mu2j73LP/be3d7EIHtWRXYH+t2v4sWkL/GLTsz0LCSNUMT5oZg3uv7ntSXDXEG7do7Nl8A8EoZVigFAAjBg0OkuBJm2G404kUpxO3jV2yPQt1JHIx3M1/t99q5IiMj19KTUzf0LHdd07nqVQljRkArGWEfwTO8BYmBhfUUHvyHwCIkSwc/9pLSKuaH307d08zQgS0tHAGHskcUBXrKb53pXO0Cdfe+7MPnRBsXjF3ecRvx4FSubHdTMATWJEkx5WVp8bBfS6pMzjWkZfUAIS/u5OUKtMMjWYFjYNgrTkoSmdbKJ3Tt2Ql/qMYZC7jSVzb8718v3PBq2qArTAlvzZnJoe72fcmtACV/sJ/o7I+MsURhAmlqahkHj2TRl+qAOMWCJqVnSRl3Vjb15fMBhPU6QqTzsfpyK9l81XKFfYschxlrj9kBLTPwFHxRUKGYQ6hQzvp3zaoFqY1h93g9hGCu BcLljyWQ zOuwrMBlyfRS8MeQ3iedCYaBAgztE77lVeCR/Qzlt67sKwFfcM0qBqinUf+lSX7HwbBdJMHa7KjEj4bRUeSUS3WmH+4tE3KZNWglIkbASKA4e7PCcXsEaxYGIF71TtcwVviSL6ez2TWtT6tYGcTDqZa3i9xYx2dGlL5RBgd5gkj7JuT413uTHnbg3H+cbMIfjIx39EOy1Z9RDKeZAvTkLUfUsyu5sgF0W4bnLHkiLqu88H5L4iDc/8jmfDr/UFgLfAPpmHkbifQWPYHFlSvkj7mKW1mrLEDzyTn/hsANRxYmF4y3NOlcS552tFiTrFOsMLBOC6wTYgfeFmbeT+dxMUBY/tlDN6d0msLZ3 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin use_zero_pages may be very useful, not just because of cache colouring as described in doc, but also because use_zero_pages can accelerate merging empty pages when there are plenty of empty pages (full of zeros) as the time of page-by-page comparisons (unstable_tree_search_insert) is saved. But when enabling use_zero_pages, madvise(addr, len, MADV_UNMERGEABLE) and other ways (like write 2 to /sys/kernel/mm/ksm/run) to trigger unsharing will *not* actually unshare the shared zeropage as placed by KSM (which is against the MADV_UNMERGEABLE documentation). As these KSM-placed zero pages are out of the control of KSM, the related counts of ksm pages don't expose how many zero pages are placed by KSM (these special zero pages are different from those initially mapped zero pages, because the zero pages mapped to MADV_UNMERGEABLE areas are expected to be a complete and unshared page) To not blindly unshare all shared zero_pages in applicable VMAs, the patch introduces a dedicated flag ZERO_PAGE_FLAG to mark the rmap_items of those shared zero_pages. and guarantee that these rmap_items will be not freed during the time of zero_pages not being writing, so we can only unshare the *KSM-placed* zero_pages. The patch will not degrade the performance of use_zero_pages as it doesn't change the way of merging empty pages in use_zero_pages's feature. Signed-off-by: xu xin Reported-by: David Hildenbrand Cc: Claudio Imbrenda Cc: Xuexin Jiang Reviewed-by: Xiaokai Ran Reviewed-by: Yang Yang --- mm/ksm.c | 141 +++++++++++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 111 insertions(+), 30 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 905a79d213da..ab04b44679c8 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -214,6 +214,7 @@ struct ksm_rmap_item { #define SEQNR_MASK 0x0ff /* low bits of unstable tree seqnr */ #define UNSTABLE_FLAG 0x100 /* is a node of the unstable tree */ #define STABLE_FLAG 0x200 /* is listed from the stable tree */ +#define ZERO_PAGE_FLAG 0x400 /* is zero page placed by KSM */ /* The stable and unstable tree heads */ static struct rb_root one_stable_tree[1] = { RB_ROOT }; @@ -420,6 +421,11 @@ static inline bool ksm_test_exit(struct mm_struct *mm) return atomic_read(&mm->mm_users) == 0; } +enum break_ksm_pmd_entry_return_flag { + HAVE_KSM_PAGE = 1, + HAVE_ZERO_PAGE +}; + static int break_ksm_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long next, struct mm_walk *walk) { @@ -427,6 +433,7 @@ static int break_ksm_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long nex spinlock_t *ptl; pte_t *pte; int ret; + bool is_zero_page = false; if (pmd_leaf(*pmd) || !pmd_present(*pmd)) return 0; @@ -434,6 +441,8 @@ static int break_ksm_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long nex pte = pte_offset_map_lock(walk->mm, pmd, addr, &ptl); if (pte_present(*pte)) { page = vm_normal_page(walk->vma, addr, *pte); + if (!page) + is_zero_page = is_zero_pfn(pte_pfn(*pte)); } else if (!pte_none(*pte)) { swp_entry_t entry = pte_to_swp_entry(*pte); @@ -444,7 +453,14 @@ static int break_ksm_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long nex if (is_migration_entry(entry)) page = pfn_swap_entry_to_page(entry); } - ret = page && PageKsm(page); + + if (page && PageKsm(page)) + ret = HAVE_KSM_PAGE; + else if (is_zero_page) + ret = HAVE_ZERO_PAGE; + else + ret = 0; + pte_unmap_unlock(pte, ptl); return ret; } @@ -466,19 +482,22 @@ static const struct mm_walk_ops break_ksm_ops = { * of the process that owns 'vma'. We also do not want to enforce * protection keys here anyway. */ -static int break_ksm(struct vm_area_struct *vma, unsigned long addr) +static int break_ksm(struct vm_area_struct *vma, unsigned long addr, + bool unshare_zero_page) { vm_fault_t ret = 0; do { - int ksm_page; + int walk_result; cond_resched(); - ksm_page = walk_page_range_vma(vma, addr, addr + 1, + walk_result = walk_page_range_vma(vma, addr, addr + 1, &break_ksm_ops, NULL); - if (WARN_ON_ONCE(ksm_page < 0)) - return ksm_page; - if (!ksm_page) + if (WARN_ON_ONCE(walk_result < 0)) + return walk_result; + if (!walk_result) + return 0; + if (walk_result == HAVE_ZERO_PAGE && !unshare_zero_page) return 0; ret = handle_mm_fault(vma, addr, FAULT_FLAG_UNSHARE | FAULT_FLAG_REMOTE, @@ -539,7 +558,7 @@ static void break_cow(struct ksm_rmap_item *rmap_item) mmap_read_lock(mm); vma = find_mergeable_vma(mm, addr); if (vma) - break_ksm(vma, addr); + break_ksm(vma, addr, false); mmap_read_unlock(mm); } @@ -764,6 +783,30 @@ static struct page *get_ksm_page(struct ksm_stable_node *stable_node, return NULL; } +/* + * Cleaning the rmap_item's ZERO_PAGE_FLAG + * This function will be called when unshare or writing on zero pages. + */ +static inline void clean_rmap_item_zero_flag(struct ksm_rmap_item *rmap_item) +{ + if (rmap_item->address & ZERO_PAGE_FLAG) + rmap_item->address &= PAGE_MASK; +} + +/* Only called when rmap_item is going to be freed */ +static inline void unshare_zero_pages(struct ksm_rmap_item *rmap_item) +{ + struct vm_area_struct *vma; + + if (rmap_item->address & ZERO_PAGE_FLAG) { + vma = vma_lookup(rmap_item->mm, rmap_item->address); + if (vma && !ksm_test_exit(rmap_item->mm)) + break_ksm(vma, rmap_item->address, true); + } + /* Put at last. */ + clean_rmap_item_zero_flag(rmap_item); +} + /* * Removing rmap_item from stable or unstable tree. * This function will clean the information from the stable/unstable tree. @@ -824,6 +867,7 @@ static void remove_trailing_rmap_items(struct ksm_rmap_item **rmap_list) struct ksm_rmap_item *rmap_item = *rmap_list; *rmap_list = rmap_item->rmap_list; remove_rmap_item_from_tree(rmap_item); + unshare_zero_pages(rmap_item); free_rmap_item(rmap_item); } } @@ -853,7 +897,7 @@ static int unmerge_ksm_pages(struct vm_area_struct *vma, if (signal_pending(current)) err = -ERESTARTSYS; else - err = break_ksm(vma, addr); + err = break_ksm(vma, addr, false); } return err; } @@ -2044,6 +2088,39 @@ static void stable_tree_append(struct ksm_rmap_item *rmap_item, rmap_item->mm->ksm_merging_pages++; } +static int try_to_merge_with_kernel_zero_page(struct ksm_rmap_item *rmap_item, + struct page *page) +{ + struct mm_struct *mm = rmap_item->mm; + int err = 0; + + /* + * It should not take ZERO_PAGE_FLAG because on one hand, + * get_next_rmap_item don't return zero pages' rmap_item. + * On the other hand, even if zero page was writen as + * anonymous page, rmap_item has been cleaned after + * stable_tree_search + */ + if (!WARN_ON_ONCE(rmap_item->address & ZERO_PAGE_FLAG)) { + struct vm_area_struct *vma; + + mmap_read_lock(mm); + vma = find_mergeable_vma(mm, rmap_item->address); + if (vma) { + err = try_to_merge_one_page(vma, page, + ZERO_PAGE(rmap_item->address)); + if (!err) + rmap_item->address |= ZERO_PAGE_FLAG; + } else { + /* If the vma is out of date, we do not need to continue. */ + err = 0; + } + mmap_read_unlock(mm); + } + + return err; +} + /* * cmp_and_merge_page - first see if page can be merged into the stable tree; * if not, compare checksum to previous and if it's the same, see if page can @@ -2055,7 +2132,6 @@ static void stable_tree_append(struct ksm_rmap_item *rmap_item, */ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_item) { - struct mm_struct *mm = rmap_item->mm; struct ksm_rmap_item *tree_rmap_item; struct page *tree_page = NULL; struct ksm_stable_node *stable_node; @@ -2092,6 +2168,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite } remove_rmap_item_from_tree(rmap_item); + clean_rmap_item_zero_flag(rmap_item); if (kpage) { if (PTR_ERR(kpage) == -EBUSY) @@ -2128,29 +2205,16 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite * Same checksum as an empty page. We attempt to merge it with the * appropriate zero page if the user enabled this via sysfs. */ - if (ksm_use_zero_pages && (checksum == zero_checksum)) { - struct vm_area_struct *vma; - - mmap_read_lock(mm); - vma = find_mergeable_vma(mm, rmap_item->address); - if (vma) { - err = try_to_merge_one_page(vma, page, - ZERO_PAGE(rmap_item->address)); - } else { + if (ksm_use_zero_pages) { + if (checksum == zero_checksum) /* - * If the vma is out of date, we do not need to - * continue. + * In case of failure, the page was not really empty, so we + * need to continue. Otherwise we're done. */ - err = 0; - } - mmap_read_unlock(mm); - /* - * In case of failure, the page was not really empty, so we - * need to continue. Otherwise we're done. - */ - if (!err) - return; + if (!try_to_merge_with_kernel_zero_page(rmap_item, page)) + return; } + tree_rmap_item = unstable_tree_search_insert(rmap_item, page, &tree_page); if (tree_rmap_item) { @@ -2230,6 +2294,7 @@ static struct ksm_rmap_item *try_to_get_old_rmap_item(unsigned long addr, * is ineligible or discarded, e.g. MADV_DONTNEED. */ remove_rmap_item_from_tree(rmap_item); + unshare_zero_pages(rmap_item); free_rmap_item(rmap_item); } @@ -2352,6 +2417,22 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) } if (is_zone_device_page(*page)) goto next_page; + if (is_zero_pfn(page_to_pfn(*page))) { + /* + * To monitor ksm zero pages which becomes non-anonymous, + * we have to save each rmap_item of zero pages by + * try_to_get_old_rmap_item() walking on + * ksm_scan.rmap_list, otherwise their rmap_items will be + * freed by the next turn of get_next_rmap_item(). The + * function get_next_rmap_item() will free all "skipped" + * rmap_items because it thinks its areas as UNMERGEABLE. + */ + rmap_item = try_to_get_old_rmap_item(ksm_scan.address, + ksm_scan.rmap_list); + if (rmap_item && (rmap_item->address & ZERO_PAGE_FLAG)) + ksm_scan.rmap_list = &rmap_item->rmap_list; + goto next_page; + } if (PageAnon(*page)) { flush_anon_page(vma, *page, ksm_scan.address); flush_dcache_page(*page); From patchwork Fri Feb 10 01:18:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Yang X-Patchwork-Id: 13135331 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BEB9C05027 for ; Fri, 10 Feb 2023 01:19:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6AEEA6B00C5; Thu, 9 Feb 2023 20:19:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 65EDD6B00C7; Thu, 9 Feb 2023 20:19:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 527FF6B00C8; Thu, 9 Feb 2023 20:19:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 407336B00C5 for ; Thu, 9 Feb 2023 20:19:06 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 121061A10B1 for ; Fri, 10 Feb 2023 01:19:06 +0000 (UTC) X-FDA: 80449623492.14.DCF38A5 Received: from mxhk.zte.com.cn (mxhk.zte.com.cn [63.216.63.40]) by imf17.hostedemail.com (Postfix) with ESMTP id 3EA5240007 for ; Fri, 10 Feb 2023 01:19:02 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=zte.com.cn; spf=pass (imf17.hostedemail.com: domain of yang.yang29@zte.com.cn designates 63.216.63.40 as permitted sender) smtp.mailfrom=yang.yang29@zte.com.cn ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675991944; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references; bh=v0XSjLI+rO5Kq2Y1Eem8ZQwbVe/7axGQVmO2VxpMLp4=; b=a3WUFSxU2lNRePBaTSBB8/WAjSDfRLgKrTMQiHtBrNObkQGs0XaZR1gUIGkXusGY27kpoM cD5ULnMiIs93P8onNh0oB5ROdTiMgWwFzdasQuyWf9Eu4NrvfIcumQgpwFOqKdFvVYJZEC q7R8NxRvGAkT7b0mRCjNmRn+3Nym0oM= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=zte.com.cn; spf=pass (imf17.hostedemail.com: domain of yang.yang29@zte.com.cn designates 63.216.63.40 as permitted sender) smtp.mailfrom=yang.yang29@zte.com.cn ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675991944; a=rsa-sha256; cv=none; b=ccPoWus0Xcpd5RsbeEpf5cOzwqXY3T1CNzssruP2qvk/2xoKSCsddpMChoRq353eUQjXGZ WjB4kYOxAGVuSU73LE/oWJ+XQ3tB66O7kAZ+CdXa9dCUfUo0LYk8b5OWvGybo0O4isDGXo FrdvbwhQk8u/KBI8/hFna/ls7FLuv3I= Received: from mse-fl1.zte.com.cn (unknown [10.5.228.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mxhk.zte.com.cn (FangMail) with ESMTPS id 4PCbVD3jTNz8R043; Fri, 10 Feb 2023 09:19:00 +0800 (CST) Received: from szxlzmapp07.zte.com.cn ([10.5.230.251]) by mse-fl1.zte.com.cn with SMTP id 31A1IqXc009217; Fri, 10 Feb 2023 09:18:52 +0800 (+08) (envelope-from yang.yang29@zte.com.cn) Received: from mapi (szxlzmapp01[null]) by mapi (Zmail) with MAPI id mid14; Fri, 10 Feb 2023 09:18:52 +0800 (CST) Date: Fri, 10 Feb 2023 09:18:52 +0800 (CST) X-Zmail-TransId: 2b0363e59b7c1e046e42 X-Mailer: Zmail v1.0 Message-ID: <202302100918524481474@zte.com.cn> Mime-Version: 1.0 From: To: Cc: , , , , , , , , Subject: =?utf-8?q?=5BPATCH_v6_3/6=5D_ksm=3A_count_all_zero_pages_placed_by_?= =?utf-8?q?KSM?= X-MAIL: mse-fl1.zte.com.cn 31A1IqXc009217 X-Fangmail-Gw-Spam-Type: 0 X-FangMail-Miltered: at cgslv5.04-192.168.250.137.novalocal with ID 63E59B84.000 by FangMail milter! X-FangMail-Envelope: 1675991940/4PCbVD3jTNz8R043/63E59B84.000/10.5.228.132/[10.5.228.132]/mse-fl1.zte.com.cn/ X-Fangmail-Anti-Spam-Filtered: true X-Fangmail-MID-QID: 63E59B84.000/4PCbVD3jTNz8R043 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 3EA5240007 X-Stat-Signature: aw1qajn4p4h3yegeyq41pds9gmjpjfyx X-Rspam-User: X-HE-Tag: 1675991942-162749 X-HE-Meta: U2FsdGVkX19UN0I8nUmHx5RvyZHlMRboaQuv4vq/7dtkJN+DtPzpe4GQE9X6WiObBEl8zwuHfzAjY0N/dA4UqkYrhmcqYqanEBFs9fDgV3caa41sp6YDBxk2udU0CI9+KY4ETTAEZbbS0udi8OShnOmwp+bC2PzByTbB2Q/3PpkstvKWnKtRx/J5d0i12FGtBQYapZCzUxIPJAEO1bPui1Rxz6P/j6YIcBTRnA5iyT2E03d+1Mne0N4Ynhva24FQPYhFb/FMLXqpe/pNH2P2bfIiYbmgQVU7wrN8ofzA+RYZuJsPAqjSyksA6wCdmQugz/AJXagkKoRZXMOlhoxxb9Cqrl4W0ddZzuk8hBR1QaN7mAYrGLmtqbUPK7AQhTU2mQTGv+wqvqs2hN66pwN3MfT98yowfUZKUA1P7MSv97f6lwoOApoqJjARRtqrBqcBuv7/gr2O96wAHTBcrxMWd5Y+BHkg5Z/j5RYROW6VMv/Y2u2LroEc8lZDDEJ0nzK6hkQATg+L45byvgr5K0K38sUIcDFGB50ACnR9W29LFUoq5p/KGJXpdZD/RtFHZ2KfFqwvPUCMriBy6DMEgR6oLxCnlKZLLPl4urvESJt/1cVw9S3Wyrwh2/Nhvu86l4AQmDTEls0bDKgeH7STT8nsvQk7REZopZH+3VFeT2TqtUvPycZca94khlCI2h8cssWRFGEkHTlcBLh9v4qnbiW5IvTbI1mFY+6E/BHIpto37WIxgP4xc52UGGtIf03b9x+dxc32voLh91llKZ3x5Nw6P1ywIsP+RIkbA60HTOnZldLIusRH9VCHHlSqoSFgOsezIU9hFnnTuNBOfubPetuzWHxN+HPp6FVP6VXi2R19JEht5RNGXnaop/CPCxLrWDJurHjIZAyWKca2HIsT5PQzrFpQVjhWvc7MunO7y3bFE0iUWrFKhO0kZUJZhIL2IME3melSfhJFeNYFU8Ejgtb Bh5HTKpG mKRM94EPWKu2DOZvXcU4Rhm2Kv+BZTE12bEY7K3nwsen2k2GFste+BRt8c1L4QQbbG2YEpyyE18mOSCG5NtZNQflqcuFisUnS6TxL3fMwcuXe6tcltr4CpJjxLM5tVTmOFVmZoUEUf6UHaJsqnAd+XC0x08IhQ2sSfQz39JhyPp9v1RLYyPFwRqQhfPZz57mFGqMPuHZZ7F0WFu8Gh5HclBU/0rD2D4+8UASobEz/2TDSfKrlp9U7BamYQvxX793FKbO1dv9lKCOCvX7dvKqIzvMEhFcyvvegoDGY0oU1w0+LJb76KfKcwluL6zA0kfRhglGol66PchnYSkCDb7oNrtfpRHSN3Lx7Mc1c X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin As pages_sharing and pages_shared don't include the number of zero pages merged by KSM, we cannot know how many pages are zero pages placed by KSM when enabling use_zero_pages, which leads to KSM not being transparent with all actual merged pages by KSM. In the early days of use_zero_pages, zero-pages was unable to get unshared by the ways like MADV_UNMERGEABLE so it's hard to count how many times one of those zeropages was then unmerged. But now, unsharing KSM-placed zero page accurately has been achieved, so we can easily count both how many times a page full of zeroes was merged with zero-page and how many times one of those pages was then unmerged. and so, it helps to estimate memory demands when each and every shared page could get unshared. So we add zero_pages_sharing under /sys/kernel/mm/ksm/ to show the number of all zero pages placed by KSM. Signed-off-by: xu xin Cc: Claudio Imbrenda Cc: David Hildenbrand Cc: Xuexin Jiang Reviewed-by: Xiaokai Ran Reviewed-by: Yang Yang v4->v5: fix warning mm/ksm.c:3238:9: warning: no previous prototype for 'zero_pages_sharing_show' [-Wmissing-prototypes]. Reviewed-by: Claudio Imbrenda --- mm/ksm.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index ab04b44679c8..1fa668e1fe82 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -276,6 +276,9 @@ static unsigned int zero_checksum __read_mostly; /* Whether to merge empty (zeroed) pages with actual zero pages */ static bool ksm_use_zero_pages __read_mostly; +/* The number of zero pages placed by KSM use_zero_pages */ +static unsigned long ksm_zero_pages_sharing; + #ifdef CONFIG_NUMA /* Zeroed when merging across nodes is not allowed */ static unsigned int ksm_merge_across_nodes = 1; @@ -789,8 +792,10 @@ static struct page *get_ksm_page(struct ksm_stable_node *stable_node, */ static inline void clean_rmap_item_zero_flag(struct ksm_rmap_item *rmap_item) { - if (rmap_item->address & ZERO_PAGE_FLAG) + if (rmap_item->address & ZERO_PAGE_FLAG) { + ksm_zero_pages_sharing--; rmap_item->address &= PAGE_MASK; + } } /* Only called when rmap_item is going to be freed */ @@ -2109,8 +2114,10 @@ static int try_to_merge_with_kernel_zero_page(struct ksm_rmap_item *rmap_item, if (vma) { err = try_to_merge_one_page(vma, page, ZERO_PAGE(rmap_item->address)); - if (!err) + if (!err) { rmap_item->address |= ZERO_PAGE_FLAG; + ksm_zero_pages_sharing++; + } } else { /* If the vma is out of date, we do not need to continue. */ err = 0; @@ -3230,6 +3237,13 @@ static ssize_t pages_volatile_show(struct kobject *kobj, } KSM_ATTR_RO(pages_volatile); +static ssize_t zero_pages_sharing_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + return sysfs_emit(buf, "%ld\n", ksm_zero_pages_sharing); +} +KSM_ATTR_RO(zero_pages_sharing); + static ssize_t stable_node_dups_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) { @@ -3285,6 +3299,7 @@ static struct attribute *ksm_attrs[] = { &pages_sharing_attr.attr, &pages_unshared_attr.attr, &pages_volatile_attr.attr, + &zero_pages_sharing_attr.attr, &full_scans_attr.attr, #ifdef CONFIG_NUMA &merge_across_nodes_attr.attr, From patchwork Fri Feb 10 01:19:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Yang X-Patchwork-Id: 13135332 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38FBCC636D4 for ; Fri, 10 Feb 2023 01:20:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CA1CA6B00C8; Thu, 9 Feb 2023 20:20:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C78AE6B00C9; Thu, 9 Feb 2023 20:20:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B68786B00CA; Thu, 9 Feb 2023 20:20:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A563D6B00C8 for ; Thu, 9 Feb 2023 20:20:09 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 77A371209EE for ; Fri, 10 Feb 2023 01:20:09 +0000 (UTC) X-FDA: 80449626138.26.04D6B40 Received: from mxhk.zte.com.cn (mxhk.zte.com.cn [63.216.63.40]) by imf18.hostedemail.com (Postfix) with ESMTP id 9B8A21C000F for ; Fri, 10 Feb 2023 01:20:06 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of yang.yang29@zte.com.cn designates 63.216.63.40 as permitted sender) smtp.mailfrom=yang.yang29@zte.com.cn; dmarc=pass (policy=none) header.from=zte.com.cn ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675992007; a=rsa-sha256; cv=none; b=FsfSzx3wJi/ttUSEkysETq7CjSHmWr8zz7Y6Pt0Fuyb5fB7+4rVFoJTc9vMbQNkKT67wBv 3LbgXADRBptFPLUpgpfUH8IqmYm2n3BzEmY2lfGhFeirdhcukKRaajr2LY9k2b9LTYJbCO Tv2dSpSDlrduW/IRfOEmsDnmqTKijEM= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of yang.yang29@zte.com.cn designates 63.216.63.40 as permitted sender) smtp.mailfrom=yang.yang29@zte.com.cn; dmarc=pass (policy=none) header.from=zte.com.cn ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675992007; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references; bh=kYr5WVQqOlA4UGoUEmtACqMpk4wQYhe20dijyYWJccU=; b=cMqN6xFoc+0ZhPUraMQykJLlmXvHciRbY2tUezBkD2xXkFVhyxZVemIgGM2kv/N1aNdZfT QijcxR4Sr7Wg279xddW18ko0CVp2aYIs4v6H5X9efVGpIDqY61qgUlKHw1H/r5T3qvcCZP SzmtpCcDUwxfxb1M1zewuzA/UecMW9c= Received: from mse-fl2.zte.com.cn (unknown [10.5.228.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mxhk.zte.com.cn (FangMail) with ESMTPS id 4PCbWR6Qvjz8R042; Fri, 10 Feb 2023 09:20:03 +0800 (CST) Received: from szxlzmapp02.zte.com.cn ([10.5.231.79]) by mse-fl2.zte.com.cn with SMTP id 31A1JmBv024052; Fri, 10 Feb 2023 09:19:48 +0800 (+08) (envelope-from yang.yang29@zte.com.cn) Received: from mapi (szxlzmapp01[null]) by mapi (Zmail) with MAPI id mid14; Fri, 10 Feb 2023 09:19:49 +0800 (CST) Date: Fri, 10 Feb 2023 09:19:49 +0800 (CST) X-Zmail-TransId: 2b0363e59bb508349ce2 X-Mailer: Zmail v1.0 Message-ID: <202302100919492571517@zte.com.cn> Mime-Version: 1.0 From: To: Cc: , , , , , , , , Subject: =?utf-8?q?=5BPATCH_v6_4/6=5D_ksm=3A_count_zero_pages_for_each_proce?= =?utf-8?q?ss?= X-MAIL: mse-fl2.zte.com.cn 31A1JmBv024052 X-Fangmail-Gw-Spam-Type: 0 X-FangMail-Miltered: at cgslv5.04-192.168.250.137.novalocal with ID 63E59BC3.000 by FangMail milter! X-FangMail-Envelope: 1675992003/4PCbWR6Qvjz8R042/63E59BC3.000/10.5.228.133/[10.5.228.133]/mse-fl2.zte.com.cn/ X-Fangmail-Anti-Spam-Filtered: true X-Fangmail-MID-QID: 63E59BC3.000/4PCbWR6Qvjz8R042 X-Rspam-User: X-Rspamd-Queue-Id: 9B8A21C000F X-Rspamd-Server: rspam01 X-Stat-Signature: a1or17qa1djohhdwu4wqodqugtxfufhq X-HE-Tag: 1675992006-960831 X-HE-Meta: U2FsdGVkX19lDVbDo9ol9V21oHfduNgNJCKnz3vg8XZpntXjUZXol76Sbn0fh1fJhig1MOEuHHMlMccMTgVAx4Tj4f9HLxxHC4nlGB2FmsqHFxH7aSQplIxW6NH2kUSynEWSHTIM+wLm/fyU+oq5vNLc5qOGslOD9BFyfGzuv63eznM0oZbmf1T8ixZjotnR5Pr+s/5eXBlgMQ0jqH/mZPCSIO9iU5wtOjyR2LzhLlq7ReJuf4j+ERmHvik3My7IJAdD14hTgdu5OFXpFduJ0tePVZ9qUa2nw/i4kYeS/3tN5jHA5OwLuc4WRkQ7PS0CIH+oheSnCld8WkgCp/GZAg4NYRQsPQbq2kwsUui+q1TS53ILLKGJqtXS9Xi6Y6c8DKw/U3TL+fg79IiQo2epGGkGQnY97jcEx6Xc+7sRGmzg1LGwAg42zTuYrWI+fFwn/bgP8a2rXoG0LLnVeUH61ZRqPUBC6cCnXfdK3pW5mapajrLlT8E5zW291U6bOQ76l4as9z12/FsdH245VnAhqe/09p1RDqgb/XGawhlc+zWK28zynMhA4kq4Yc0oejsiWuHBbbj3/Fz5WaybrbCEYu6Lo0Hkxc2BC6JBYG0/W5IG9R6XF9KYn02BTPcOX0znsajUohxvdaFGLBfY5P/oQcH/q3japPN3RoA1Lvvz+ceNamp0XsS1+BCTA7ZVOmOLY7oTY5/iHBZdoh3I5/lxhvWbXn/HoVnby6YMEwVkH3Mh9BZ2uAl51FbZThG7y2eQLqsCmUr4L2rtUOJm+B2AHOKwIS/RW1KXR1dTB3wXJ4HjRhz+MyatkTQLBIv+oD6gyc7uJRkJP6u2kkrq12UUDmHF/86oT+FMKGPiT/+a5KkgiV5q32ZEhj3ww74aoz+j8EeTx1o4yOGfstzx/AtpVNm+qoHkAwuni8+kZ6WJmlC7SQFd9kQNvegLKaNAJFnHLIEmDXcjh+Q7kaxBolE NjUUlUuy uSWDWD27qZgQyEQPjFEKGGpJqJzSojYWyuN1wClvfuEJoKufT7n47n3hPNR6EoNYSSBUiD3C+aIyR8YkreqNhUkbAoh25FeA0qE049CdcCGAE8gkbTcBsZ4PaUS1I4sOtIa5RXVWw4MpUHy7TWcKf6sGoWJBp/j2ZI3m4KYEBSX5XRfe4H+rGgq8UAkg7l6jYhLg70gXHYrRRlM/REXVglspvmPYwfsO/gW0w9YqdMelRMzx5cYOxKrXDIE24pN/d9H9S0gaFd0BZx7XS9RLs2d0Yk2T3IdetLx4IZ9dlj8ddTY1fn0I7SbMdVOPWVFRqwOG8ff+HXwjz+4VFBRAqRZ78qQ4tedac5l2F X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin As the number of ksm zero pages is not included in ksm_merging_pages per process when enabling use_zero_pages, it's unclear of how many actual pages are merged by KSM. To let users accurately estimate their memory demands when unsharing KSM zero-pages, it's necessary to show KSM zero- pages per process. since unsharing zero pages placed by KSM accurately is achieved, then tracking empty pages merging and unmerging is not a difficult thing any longer. Since we already have /proc//ksm_stat, just add the information of zero_pages_sharing in it. Cc: Claudio Imbrenda Cc: David Hildenbrand Cc: Xuexin Jiang Cc: Xiaokai Ran Cc: Yang Yang Signed-off-by: xu xin Reviewed-by: Claudio Imbrenda --- fs/proc/base.c | 1 + include/linux/mm_types.h | 7 ++++++- mm/ksm.c | 2 ++ 3 files changed, 9 insertions(+), 1 deletion(-) diff --git a/fs/proc/base.c b/fs/proc/base.c index 9e479d7d202b..ac9ebe972be0 100644 --- a/fs/proc/base.c +++ b/fs/proc/base.c @@ -3207,6 +3207,7 @@ static int proc_pid_ksm_stat(struct seq_file *m, struct pid_namespace *ns, mm = get_task_mm(task); if (mm) { seq_printf(m, "ksm_rmap_items %lu\n", mm->ksm_rmap_items); + seq_printf(m, "zero_pages_sharing %lu\n", mm->ksm_zero_pages_sharing); mmput(mm); } diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 4e1031626403..5c734ebc1890 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -776,7 +776,7 @@ struct mm_struct { #ifdef CONFIG_KSM /* * Represent how many pages of this process are involved in KSM - * merging. + * merging (not including ksm_zero_pages_sharing). */ unsigned long ksm_merging_pages; /* @@ -784,6 +784,11 @@ struct mm_struct { * including merged and not merged. */ unsigned long ksm_rmap_items; + /* + * Represent how many empty pages are merged with kernel zero + * pages when enabling KSM use_zero_pages. + */ + unsigned long ksm_zero_pages_sharing; #endif #ifdef CONFIG_LRU_GEN struct { diff --git a/mm/ksm.c b/mm/ksm.c index 1fa668e1fe82..42dbcc3ec90d 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -794,6 +794,7 @@ static inline void clean_rmap_item_zero_flag(struct ksm_rmap_item *rmap_item) { if (rmap_item->address & ZERO_PAGE_FLAG) { ksm_zero_pages_sharing--; + rmap_item->mm->ksm_zero_pages_sharing--; rmap_item->address &= PAGE_MASK; } } @@ -2117,6 +2118,7 @@ static int try_to_merge_with_kernel_zero_page(struct ksm_rmap_item *rmap_item, if (!err) { rmap_item->address |= ZERO_PAGE_FLAG; ksm_zero_pages_sharing++; + rmap_item->mm->ksm_zero_pages_sharing++; } } else { /* If the vma is out of date, we do not need to continue. */ From patchwork Fri Feb 10 01:20:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Yang X-Patchwork-Id: 13135333 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96C68C636D4 for ; Fri, 10 Feb 2023 01:20:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3795E6B00CA; Thu, 9 Feb 2023 20:20:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 328F26B00CB; Thu, 9 Feb 2023 20:20:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2181D6B00CC; Thu, 9 Feb 2023 20:20:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 12CA96B00CA for ; Thu, 9 Feb 2023 20:20:53 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A34D841191 for ; Fri, 10 Feb 2023 01:20:52 +0000 (UTC) X-FDA: 80449627944.07.8DAB35D Received: from mxct.zte.com.cn (mxct.zte.com.cn [183.62.165.209]) by imf15.hostedemail.com (Postfix) with ESMTP id 340C0A0006 for ; Fri, 10 Feb 2023 01:20:49 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=zte.com.cn; spf=pass (imf15.hostedemail.com: domain of yang.yang29@zte.com.cn designates 183.62.165.209 as permitted sender) smtp.mailfrom=yang.yang29@zte.com.cn ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675992050; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references; bh=XU9FheZT57GPBRP073pAhzfqBkSRwOOpOqW3eL8idB8=; b=5qhfzB0AVKtHxF8CClFgM3nvBxpl/C2O366A6TrIXbhHNZhpU6wOWjxIdx0ro6Mew0eV9v r9ceH7CR7JB/FSFiwjqGmW2cNC/W2xQRCpHsrTjKx6M6ecHwFbixZGYk/rCsLe4t5R5rGr o+z6xHCgGaSfa1GEI6N9QQDrKwwsu0w= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=zte.com.cn; spf=pass (imf15.hostedemail.com: domain of yang.yang29@zte.com.cn designates 183.62.165.209 as permitted sender) smtp.mailfrom=yang.yang29@zte.com.cn ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675992050; a=rsa-sha256; cv=none; b=pYeumIheGLTnxgigTp6jWBhLPUOnorRaqPP31vXSOA+2kP/a0s4K7IEhaEKLxtiDBFN/UF DGvGkPK0RohX44UUI+3ilSBkUobhrnHktkeEavDszlasyeu7PL/rTnmZte7nPm8IdGuFZS 8jiGt7J+bIKP9le15WdyqL20KAvvleE= Received: from mse-fl2.zte.com.cn (unknown [10.5.228.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mxct.zte.com.cn (FangMail) with ESMTPS id 4PCbXG5ZPmz501Qh; Fri, 10 Feb 2023 09:20:46 +0800 (CST) Received: from szxlzmapp01.zte.com.cn ([10.5.231.85]) by mse-fl2.zte.com.cn with SMTP id 31A1Kgft025577; Fri, 10 Feb 2023 09:20:42 +0800 (+08) (envelope-from yang.yang29@zte.com.cn) Received: from mapi (szxlzmapp01[null]) by mapi (Zmail) with MAPI id mid14; Fri, 10 Feb 2023 09:20:42 +0800 (CST) Date: Fri, 10 Feb 2023 09:20:42 +0800 (CST) X-Zmail-TransId: 2b0363e59beaffffffffcc54c922 X-Mailer: Zmail v1.0 Message-ID: <202302100920429071565@zte.com.cn> Mime-Version: 1.0 From: To: Cc: , , , , , , , , Subject: =?utf-8?q?=5BPATCH_v6_5/6=5D_ksm=3A_add_zero=5Fpages=5Fsharing_docu?= =?utf-8?q?mentation?= X-MAIL: mse-fl2.zte.com.cn 31A1Kgft025577 X-Fangmail-Gw-Spam-Type: 0 X-Fangmail-Anti-Spam-Filtered: true X-Fangmail-MID-QID: 63E59BEE.002/4PCbXG5ZPmz501Qh X-Rspamd-Queue-Id: 340C0A0006 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: qia81ashawg6dbbdfcpkjmwtafu4e4zs X-HE-Tag: 1675992049-591663 X-HE-Meta: U2FsdGVkX18DuqEMyDO7LYqdUWOOlv7VfrVrMqtHHm7IJD86ZpN/XtuUv4o6FRZJql/6gIefzKhrWwLD9VcgHWeeBQI9JqXxZTRgi9+3o5zZkD5H7lwgbNQ/jlvAjmGmkSCnSTCvf7ukmSP8b1oeTmR1VpMwChWLYS9ewuyoTUilPUV7Tv3xKrv2KQR1EUmNE89evhK/lRRNvUFdTvISZ9JR6WBkVK0uJq0IUCXM49n4mk9mJUZXEf0M+0xhVZnfCWXVFgq53Ugo2dboRQ/0C6TFNkuA0azVafqw97tU/XBn+8BuAyU+dECM9MIA1kySC2ZDUDBfqYwgXjbxjB2uDtdlK3iKg0XDB1yveJ8IEM8FZDnKPIbV+EK84BPMxcPdkjbB7xUG0lOns1qUs4j6Z+tpOijt20PJ8etCFOuGlxoV6wA02uA6L6xYcRl02sdeXHKK+Nzz6f6rI0hm0OBTTGTyCTrgfsJlSHwrY1YEFCcrspbhzDXO1XrMyqbwQ0R9ruT4/9IFIqlvsydM5atE3RJZMu1ta/Lw2qLftBtlKXUBRmohvkXhqPXhNE/iddRt8+O0M3UFknroyA07TetIn9ZAHFhfSqPNGUzYIYCqsb0t2MCdfYUic0C1J+X+gPWXED3mLdhKfAxc1uOVL1gndRidABvdR9m/6oFws8Ly/e64me8N7xLbchDr8ZkT8Rnmd+xjfAMG1TZOSLdWmz+S69yamege/u284lVCa3DjyLPHeXCpCFCvtrkM74ft8ubhF7P9J9kujN+y+aDglmuZUlffDnJ8TfpG21iklHfpVxccTqeyJ3qj0kceEAznoeP39Ag7fgofeVHQinClmrKSfVD9FxuJcM7gapRIQLyLa3cVBKh+txpgcIibqpuiIjdwof0hfUu+GLQsvtn9qCjZl1FefZ80SlifCemIKNEhYNgUYWmUsNy3BM6o9KK50zGDHmy/KyFo+W8A9Gbx7YY Di7r8MnW 1bKzlMbioN+AQRO/C4d1rXtEKuPCY6UYehcnLYV4eheWCAOu+CiMUrwIqNaUhdS0vrOSpuLHm5S3Oek0XaqqFOq8xjEIiqZ77Ny2dxxcG2hMgWAtzApJ4awvtwmusH9ZLyNjRXVK1TFx3lrnBjjsfx98GkWlRXdh0DzeMGPzd7FTcrvrKIejvy73Qyi5rG9q4nGrQxN8I/+0nUenGpY1oPhqOA4EQpPv7+5BvnATaum55XJNM3RM/YLQshCGmYa3DRgEMhEQdyfnnP9vc0wIAjU+BwTXNjJ2hr57g/qcOOqXZM6MYTUnXZPRtrPRL5hzQthwlIZYb/xaNGOeY+g/0qWzT6gNEJygzgiGMnng/CZnHxfACHng2k+DQ+DRAyvjQ0GD6dI1Py3tcNA2Tp6t5txUPNQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000063, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin When enabling use_zero_pages, pages_sharing cannot represent how much memory saved indeed. zero_pages_sharing + pages_sharing does. add the description of zero_pages_sharing. Cc: Xiaokai Ran Cc: Yang Yang Cc: Jiang Xuexin Cc: Claudio Imbrenda Cc: David Hildenbrand Signed-off-by: xu xin --- Documentation/admin-guide/mm/ksm.rst | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/Documentation/admin-guide/mm/ksm.rst b/Documentation/admin-guide/mm/ksm.rst index fb6ba2002a4b..f160f9487a90 100644 --- a/Documentation/admin-guide/mm/ksm.rst +++ b/Documentation/admin-guide/mm/ksm.rst @@ -173,6 +173,13 @@ stable_node_chains the number of KSM pages that hit the ``max_page_sharing`` limit stable_node_dups number of duplicated KSM pages +zero_pages_sharing + how many empty pages are sharing kernel zero page(s) instead of + with each other as it would happen normally. Only effective when + enabling ``use_zero_pages`` knob. + +When enabling ``use_zero_pages``, the sum of ``pages_sharing`` + +``zero_pages_sharing`` represents how much really saved by KSM. A high ratio of ``pages_sharing`` to ``pages_shared`` indicates good sharing, but a high ratio of ``pages_unshared`` to ``pages_sharing`` From patchwork Fri Feb 10 01:21:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Yang X-Patchwork-Id: 13135334 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00465C61DA4 for ; Fri, 10 Feb 2023 01:22:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 54C7A6B00CC; Thu, 9 Feb 2023 20:22:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4FC656B00CD; Thu, 9 Feb 2023 20:22:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3EC716B00CE; Thu, 9 Feb 2023 20:22:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 2FA0B6B00CC for ; Thu, 9 Feb 2023 20:22:08 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id EF5B7161151 for ; Fri, 10 Feb 2023 01:22:07 +0000 (UTC) X-FDA: 80449631094.15.57A6A79 Received: from mxct.zte.com.cn (mxct.zte.com.cn [183.62.165.209]) by imf05.hostedemail.com (Postfix) with ESMTP id 7484E100010 for ; Fri, 10 Feb 2023 01:22:05 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=zte.com.cn; spf=pass (imf05.hostedemail.com: domain of yang.yang29@zte.com.cn designates 183.62.165.209 as permitted sender) smtp.mailfrom=yang.yang29@zte.com.cn ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675992126; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references; bh=q8aT+UKs3RageA/xYVixFXiFtP0UQSUwN/I2j9zzRKw=; b=cyO5y5oNIAdOZz89+d5K3VKkCIAEh7OVWjwYxxUJZ6W1h2FiWpTjODdqeBbzrIBFNUUO3V ++MpFmLKlFCawOweJXui1yehQ/MOl990MJEC68wduWMTLUT9vBHiwrTneU+39VN2sidHQC LdfixHTNxdo+/HqhMfenVTBYKxwSG4U= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=zte.com.cn; spf=pass (imf05.hostedemail.com: domain of yang.yang29@zte.com.cn designates 183.62.165.209 as permitted sender) smtp.mailfrom=yang.yang29@zte.com.cn ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675992126; a=rsa-sha256; cv=none; b=yWi6bBSa9lm/kZ2jQ0LWxhfOODmt0qT3Kn1dpj3IE6WMhmsPr0QNA68xGnWyWkf7vw0INj KqSW1YUz/tOfW6CkZGJ24HUVd0m63JHSeY9exjCBsoJKJEv3ktET45x2cIRiH3mz516mah 4ZqQtVmo5cpCAON4PXS0SxgyCiC0QW0= Received: from mse-fl2.zte.com.cn (unknown [10.5.228.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mxct.zte.com.cn (FangMail) with ESMTPS id 4PCbYk6QnNz501Qf; Fri, 10 Feb 2023 09:22:02 +0800 (CST) Received: from szxlzmapp05.zte.com.cn ([10.5.230.85]) by mse-fl2.zte.com.cn with SMTP id 31A1LuBR026416; Fri, 10 Feb 2023 09:21:56 +0800 (+08) (envelope-from yang.yang29@zte.com.cn) Received: from mapi (szxlzmapp01[null]) by mapi (Zmail) with MAPI id mid14; Fri, 10 Feb 2023 09:21:57 +0800 (CST) Date: Fri, 10 Feb 2023 09:21:57 +0800 (CST) X-Zmail-TransId: 2b0363e59c35ffffffffa055065e X-Mailer: Zmail v1.0 Message-ID: <202302100921574141612@zte.com.cn> Mime-Version: 1.0 From: To: Cc: , , , , , , , , Subject: =?utf-8?q?=5BPATCH_v6_6/6=5D_selftest=3A_add_testing_unsharing_and_?= =?utf-8?q?counting_ksm_zero?= =?utf-8?q?_page?= X-MAIL: mse-fl2.zte.com.cn 31A1LuBR026416 X-Fangmail-Gw-Spam-Type: 0 X-Fangmail-Anti-Spam-Filtered: true X-Fangmail-MID-QID: 63E59C3A.001/4PCbYk6QnNz501Qf X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7484E100010 X-Stat-Signature: ufj614gfabmw6pxmscs9q33zpruh4geg X-HE-Tag: 1675992125-365246 X-HE-Meta: U2FsdGVkX1/WrsCeYD1e1vnYo+PnbxTSj3Qcmmx/VROzKh3oyZGKmMcLGojgtl7ipxIQwJYA41w9MSiMqw+Crw4Hc+nPOm35F4Bp+p3XstvFXd1Fw9gDUF90KsaQxQ3XdZ/+aU4pVUN71d+Zg5vkLBzhuAW7UZYbdTud5Cgi0M9i150lfAj556tyfj8uKdllO5hvXxfaWM2RmtefKMphdqEmJ0orWTj+RkWurvw6goZ/xBge29F9E7zr6Yvo4nN2ufYQTPdooSl9ZfgPHZIt8+cVNA9VzLkM8njudazhMaeinqzs7H5NSCbxwoglJ0UVwRos1qBuTs0a1ADCb8riB6vE1Hqs8BS7pwKnk01rxL1IMPfoC4BZM2zujBJ4BFQrSZBasR/RFFBOIAj6k5sUKWmuqKeZAnJI0q+wOU8DieS8AfsHNpgzGxLGJ4x+0BjGplaaV3H2J8k4gmIiAlok7BkiW85o3FBx2JgyAyFWxs7l1yaeIb0rdHCxBNFw/cb6BCx15MwDkn+ZhTcJSTy0B1ORNIvqZYEyEUOt5K68zmvnfufaRkLOe2NW1UXENM9Xp5LSp6l9bFdGkHTkGKA6hooyPgdanFD1vDQY7E8lK5m+J0UIFbIIdSs48Kj7iRZ8hInXLAORHN6ZDRD0PgI4lucl50pvEM1lIGbBjg/Esw3lw1BqcQhAdud7Vkq3nK+jju9RrCiSj4hvQC9bnT/dCQaWTO8zMRwzMjSIv60tI+OMocd1Okbww6IKuQ96B1n4iflLFluo61dovWTZx8xUsE0WvTC5vPqQv+ukL9wIvqgYb4M7VO25yJ5pSpHCe+bsaNWSo37pRuVLgYp8Vb5g42cao+YrglouDdAoYtvXRfwvhMcM+hZqfHG1iMkgzOqjhVCHzwswrdBh7DXVCqtHMA5t8J7xR3BUqIVaNGkydoLBGWQa17C7hPjXpHxM1HRxIOm1h6gaYNiLImxWpng ehDRoo3I hnr3q06LvzQLFEgnHLtu4Lu9qIaCagFWbxGrVxrbRdK0BanFU8LtgVG53C59wylcci8bB4+5Eb+f6ZRclketj4i0MMisMiM80XXnOcEQv27KighHhMPGzypJytD4l3z7SoNEWFNBGV8gT/2+rDSxPLwF1mSHLw5o0//LGsH66t3aXNDsWh7Bn7wzpZGRYZjeeS8o37C57XsIjErwM4wbXzU1MC0hQQVAh45rcg5h46CQnaeqwGkAv110ROETkF0OO2gZ3rOw1PYMcLGL3zKuwSBnRTXh+9roxF7AoGST05h5hVIqmJ/YoYqeIP9ULl+/Z3lIJFMdwlz1sooUYdJzrMyEvTHK0ou6MLVK/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin Add a function test_unmerge_zero_page() to test the functionality on unsharing and counting ksm-placed zero pages and counting of this patch series. test_unmerge_zero_page() actually contains three subjct test objects: 1) whether the count of ksm zero page can react correctly to cow (copy on write); 2) whether the count of ksm zero page can react correctly to unmerge; 3) whether ksm zero pages are really unmerged. Signed-off-by: xu xin Cc: Claudio Imbrenda Cc: David Hildenbrand Cc: Xuexin Jiang Reviewed-by: Xiaokai Ran Reviewed-by: Yang Yang v5->v6: According to David's suggestions, the following changes are made: 1) Rename check_ksm_zero_pages_count() -> ksm_get_zero_pages(), and do the comparison outside. 2) Open all global fd from main() rather than the test case. 3) Remove COW-related test codes and focus on explicit unmerging here. 4) Add some coments to explain why wait_two_full_scans is required. 5) Clean up some unneed changes. v4->v5: fix error of "} while (end_scans < start_scans + 20);" to "} while (end_scans < start_scans + 2);" in wait_two_full_scans(). --- tools/testing/selftests/vm/ksm_functional_tests.c | 96 ++++++++++++++++++++++- 1 file changed, 92 insertions(+), 4 deletions(-) diff --git a/tools/testing/selftests/vm/ksm_functional_tests.c b/tools/testing/selftests/vm/ksm_functional_tests.c index b11b7e5115dc..3033cd6ed3b4 100644 --- a/tools/testing/selftests/vm/ksm_functional_tests.c +++ b/tools/testing/selftests/vm/ksm_functional_tests.c @@ -24,9 +24,12 @@ #define KiB 1024u #define MiB (1024 * KiB) +#define PageSize (4 * KiB) static int ksm_fd; static int ksm_full_scans_fd; +static int ksm_zero_pages_fd; +static int ksm_use_zero_pages_fd; static int pagemap_fd; static size_t pagesize; @@ -57,6 +60,21 @@ static bool range_maps_duplicates(char *addr, unsigned long size) return false; } +static long ksm_get_zero_pages(void) +{ + char buf[20]; + ssize_t read_size; + unsigned long ksm_zero_pages; + + read_size = pread(ksm_zero_pages_fd, buf, sizeof(buf) - 1, 0); + if (read_size < 0) + return -errno; + buf[read_size] = 0; + ksm_zero_pages = strtol(buf, NULL, 10); + + return ksm_zero_pages; +} + static long ksm_get_full_scans(void) { char buf[10]; @@ -70,15 +88,12 @@ static long ksm_get_full_scans(void) return strtol(buf, NULL, 10); } -static int ksm_merge(void) +static int wait_two_full_scans(void) { long start_scans, end_scans; - /* Wait for two full scans such that any possible merging happened. */ start_scans = ksm_get_full_scans(); if (start_scans < 0) - return start_scans; - if (write(ksm_fd, "1", 1) != 1) return -errno; do { end_scans = ksm_get_full_scans(); @@ -89,6 +104,34 @@ static int ksm_merge(void) return 0; } +static inline int ksm_merge(void) +{ + /* Wait for two full scans such that any possible merging happened. */ + if (write(ksm_fd, "1", 1) != 1) + return -errno; + + return wait_two_full_scans(); +} + +static int unmerge_zero_page(char *start, unsigned long size) +{ + int ret; + + ret = madvise(start, size, MADV_UNMERGEABLE); + if (ret) { + ksft_test_result_fail("MADV_UNMERGEABLE failed\n"); + return ret; + } + + /* + * Wait for two full scans such that any possible unmerging of zero + * pages happened. Why? Because the unmerge action of zero pages is not + * done in the context of madvise(), but in the context of + * unshare_zero_pages() of the ksmd thread. + */ + return wait_two_full_scans(); +} + static char *mmap_and_merge_range(char val, unsigned long size) { char *map; @@ -146,6 +189,48 @@ static void test_unmerge(void) munmap(map, size); } +static void test_unmerge_zero_pages(void) +{ + const unsigned int size = 2 * MiB; + char *map; + unsigned long pages_expected; + + ksft_print_msg("[RUN] %s\n", __func__); + + /* Confirm the interfaces*/ + if (ksm_zero_pages_fd < 0) { + ksft_test_result_skip("open(\"/sys/kernel/mm/ksm/zero_pages_sharing\") failed\n"); + return; + } + if (ksm_use_zero_pages_fd < 0) { + ksft_test_result_skip("open \"/sys/kernel/mm/ksm/use_zero_pages\" failed\n"); + return; + } + if (write(ksm_use_zero_pages_fd, "1", 1) != 1) { + ksft_test_result_skip("write \"/sys/kernel/mm/ksm/use_zero_pages\" failed\n"); + return; + } + + /* Mmap zero pages*/ + map = mmap_and_merge_range(0x00, size); + if (map == MAP_FAILED) + return; + + if (unmerge_zero_page(map + size / 2, size / 2)) + goto unmap; + + /* Check if zero_pages_sharing can be update correctly when unmerge */ + pages_expected = (size / 2) / PageSize; + ksft_test_result(pages_expected == ksm_get_zero_pages(), + "zero page count react to unmerge\n"); + + /* Check if ksm zero pages are really unmerged */ + ksft_test_result(!range_maps_duplicates(map + size / 2, size / 2), + "KSM zero pages were unmerged\n"); +unmap: + munmap(map, size); +} + static void test_unmerge_discarded(void) { const unsigned int size = 2 * MiB; @@ -264,8 +349,11 @@ int main(int argc, char **argv) pagemap_fd = open("/proc/self/pagemap", O_RDONLY); if (pagemap_fd < 0) ksft_exit_skip("open(\"/proc/self/pagemap\") failed\n"); + ksm_zero_pages_fd = open("/sys/kernel/mm/ksm/zero_pages_sharing", O_RDONLY); + ksm_use_zero_pages_fd = open("/sys/kernel/mm/ksm/use_zero_pages", O_RDWR); test_unmerge(); + test_unmerge_zero_pages(); test_unmerge_discarded(); #ifdef __NR_userfaultfd test_unmerge_uffd_wp();