From patchwork Sun Oct 9 02:19:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: xu xin X-Patchwork-Id: 13001878 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57965C433F5 for ; Sun, 9 Oct 2022 02:20:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CC2E16B0071; Sat, 8 Oct 2022 22:19:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C72C16B0073; Sat, 8 Oct 2022 22:19:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B164A6B0074; Sat, 8 Oct 2022 22:19:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9E5DD6B0071 for ; Sat, 8 Oct 2022 22:19:59 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 6B8ECABCA0 for ; Sun, 9 Oct 2022 02:19:59 +0000 (UTC) X-FDA: 79999805718.21.67E8F31 Received: from mail-pl1-f193.google.com (mail-pl1-f193.google.com [209.85.214.193]) by imf03.hostedemail.com (Postfix) with ESMTP id 0BB1520014 for ; Sun, 9 Oct 2022 02:19:57 +0000 (UTC) Received: by mail-pl1-f193.google.com with SMTP id d24so7688606pls.4 for ; Sat, 08 Oct 2022 19:19:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jsbh45Fv58HEehSJMVvF1reAzOnYH1CPjnbjNiOuTbI=; b=IY6o+u/pLMZi4wFmcw5uqJB+plYJlGXOYXMKUIx/I6RempySqHXX4p8H6FrdpKXqaY l+TJqDQ6k81QwdzrhfHZC1XS9uG/K+ZOBuByePD6fGnX0gesRFBP6ejOcXwvZMfSpvHv 4/ip8yokki7OdNv7mE7FVla8cC5EjnDnIFkn+4FjWsvVxUt/q7oeOmYxV9yK3HoEsrr2 QMNddGvJ41qITpkE4W4hGrma43Y50VfVHoGv/dJK6Fck/N9/V4EmxYTzPZY2iYbHT65H B+Jf4usU0Akcf4NJ90Hj7EDMltRqDJtYOfPwZB5g7Gs0BdCe2CbXcEAQK0VPN88B+5U5 fNug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jsbh45Fv58HEehSJMVvF1reAzOnYH1CPjnbjNiOuTbI=; b=xNtAHVFyg3nhSVKGdOPZQLriMrMGi+f18twcwESm+zZKRKoeEUmfEsEVjKmuSD1+Kg cW5kS3fbmx1BxUcL/ryjB6AMLEJ4hLsu9XMfxERiQjR7Ge5yAPk2kaj5j2SxmBq/oxEp C3Q+lvZMBSc8lrEydqDhaJqg/FwMS4voCBH13cqJNZPq3TLCEoI9VC8kvlRTIi8MLr+L p219YwrLlT9MJfXF/iDJqCakXJIDlvL2eqZU3Z/R+VGFZd8YySH8v4vyIbE2uNJ/QZRs CsrdOT59OAG6zOxKkQGua66UQnnLLMQgqI/gvcA18aOiTrXFC67aCuAUkF1b35uaetr0 5gNQ== X-Gm-Message-State: ACrzQf3j2wQhX/v0CsL4LIC+Uf17R+6MbF/4W6bJnQ8tUjjMZboHNEKo VKw2lDuqAHseqMNrS+aEn2s= X-Google-Smtp-Source: AMsMyM4AdT30BWpujVN2SmQYYAaqiC5g0djHrVWOrn0yrmMCpf0aIs13MpEVvw54B513wvnFdGFNaw== X-Received: by 2002:a17:902:9048:b0:17f:93a5:4638 with SMTP id w8-20020a170902904800b0017f93a54638mr12217181plz.108.1665281997076; Sat, 08 Oct 2022 19:19:57 -0700 (PDT) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id jb12-20020a170903258c00b0017e9b820a1asm3967799plb.100.2022.10.08.19.19.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Oct 2022 19:19:56 -0700 (PDT) From: xu.xin.sc@gmail.com X-Google-Original-From: xu.xin16@zte.com.cn To: akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, xu xin , Claudio Imbrenda , David Hildenbrand , Xuexin Jiang , Xiaokai Ran , Yang Yang Subject: [PATCH v2 1/5] ksm: abstract the function try_to_get_old_rmap_item Date: Sun, 9 Oct 2022 02:19:50 +0000 Message-Id: <20221009021950.315263-1-xu.xin16@zte.com.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221009021816.315205-1-xu.xin16@zte.com.cn> References: <20221009021816.315205-1-xu.xin16@zte.com.cn> MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665281998; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jsbh45Fv58HEehSJMVvF1reAzOnYH1CPjnbjNiOuTbI=; b=aNdkLhb807Cz7B3QpMbZoT+8KjuHp6ZzihPEGzma5n2VI2YFukKVBmBJrhVKR9P4nRXdyr iuQEYjZabZn5U86zEvow628n/UMnxXWFw8R8RWxPV0leYJUkLJ6q2JcfZaVcUxtqRCw9Cz ZgZeIOYTfnUcRDlWCeG1o9HdAvB822Q= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="IY6o+u/p"; spf=pass (imf03.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.214.193 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1665281998; a=rsa-sha256; cv=none; b=x1WoCmh0marHa8i/co4+cvboaXhACbA4nA5VZh7L1ZaZgJDlm7pjfJHBOq3gEyxmj4pS0T PO/wzML5U+FkaKfSdHrCOpn/hXmTwLZhJraTNYkMSiptizC96LKXy8ovlPXwMmAQPGgBdh xQckauz8F7s+QKen5lt31XAgj7GiPI8= X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 0BB1520014 X-Stat-Signature: 8itmyh7tt5p64ehsh48ctkw8byyyp3ek Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="IY6o+u/p"; spf=pass (imf03.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.214.193 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1665281997-620715 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin A new function try_to_get_old_rmap_item is abstracted from get_next_rmap_item. This function will be reused by the subsequent patches about counting ksm_zero_pages. The patch improves the readability and reusability of KSM code. Signed-off-by: xu xin Cc: Claudio Imbrenda Cc: David Hildenbrand Cc: Xuexin Jiang Cc: Xiaokai Ran Cc: Yang Yang --- mm/ksm.c | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index c19fcca9bc03..13c60f1071d8 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -2187,23 +2187,36 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite } } -static struct ksm_rmap_item *get_next_rmap_item(struct ksm_mm_slot *mm_slot, - struct ksm_rmap_item **rmap_list, - unsigned long addr) +static struct ksm_rmap_item *try_to_get_old_rmap_item(unsigned long addr, + struct ksm_rmap_item **rmap_list) { - struct ksm_rmap_item *rmap_item; - while (*rmap_list) { - rmap_item = *rmap_list; + struct ksm_rmap_item *rmap_item = *rmap_list; if ((rmap_item->address & PAGE_MASK) == addr) return rmap_item; if (rmap_item->address > addr) break; *rmap_list = rmap_item->rmap_list; + /* Running here indicates it's vma has been UNMERGEABLE */ remove_rmap_item_from_tree(rmap_item); free_rmap_item(rmap_item); } + return NULL; +} + +static struct ksm_rmap_item *get_next_rmap_item(struct ksm_mm_slot *mm_slot, + struct ksm_rmap_item **rmap_list, + unsigned long addr) +{ + struct ksm_rmap_item *rmap_item; + + /* lookup if we have a old rmap_item matching the addr*/ + rmap_item = try_to_get_old_rmap_item(addr, rmap_list); + if (rmap_item) + return rmap_item; + + /* Need to allocate a new rmap_item */ rmap_item = alloc_rmap_item(); if (rmap_item) { /* It has already been zeroed */ From patchwork Sun Oct 9 02:23:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: xu xin X-Patchwork-Id: 13001879 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84128C433FE for ; Sun, 9 Oct 2022 02:23:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 03DF96B0071; Sat, 8 Oct 2022 22:23:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F08206B0073; Sat, 8 Oct 2022 22:23:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D5AAD6B0074; Sat, 8 Oct 2022 22:23:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id BBCEA6B0071 for ; Sat, 8 Oct 2022 22:23:11 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id CA2E9A0168 for ; Sun, 9 Oct 2022 02:23:10 +0000 (UTC) X-FDA: 79999813740.01.2D2D6F3 Received: from mail-pj1-f66.google.com (mail-pj1-f66.google.com [209.85.216.66]) by imf09.hostedemail.com (Postfix) with ESMTP id 6FC2A140021 for ; Sun, 9 Oct 2022 02:23:10 +0000 (UTC) Received: by mail-pj1-f66.google.com with SMTP id 70so7287923pjo.4 for ; Sat, 08 Oct 2022 19:23:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pZ2+yi+8Cy6ea/BvAsMEkhQ4pii2mF27mTTm96t3VBo=; b=hmVN/VbmXsVIca+O9lSyXl7yVnjrBsRWpjgJh7tztjuDDFNe79jgL/7ENKZgE6W75Q zAFpLmIpf/sc9MHxVNyaHdXqpmgPaUGswY+T1o0dV9lZ0kBpbhhJqMtYPZzBehSurJch 9w80NOemig3XcODDC3RziJnf9GYfixChHHAhTNNBKvO6bAuNjC9uMAqliARaj3VERlxO ApTFtUQzITehlx0mqkd9eFETXnIHCFRSPcsVM2Ak7BM+UkvgHi9gWn+nFOKWgVc6Qhsb blleOhlmtqyLcN9c9HdU6QDuIziWV8DmLQKPU0LblzZaCTaxip4SHbEkXiAuW3PpjRnj 7dGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pZ2+yi+8Cy6ea/BvAsMEkhQ4pii2mF27mTTm96t3VBo=; b=CRyyKYcbs1kBfcI/oQp86M7NX2M1zw63lYWNb0Yfl5vhkxkePBLrSSzvODLgE76keg On3mUbhfBO9gySFF1RzS5BJ7dlzGPmbL3E7absswuO8dkZ7Z0Gig9ewSBaLUvmfCBbSp hb0zS04XGglbcTQ9YZ6QcXK0hWiZBPAOT78Jsq0RU0EuWouZK6qvunz+wWM7aemdD+lm gKENYz1mdhAbdPLgbXix2KN64huUaUn1i9trfjqntrdmPQeieN578LE3reUhnDKX0CKp RgrzEOsxAADpGdx3PrB+gtiDeDRcF1S8KQ0ZgciEwAzpkH0N4UIY/8TrsaP16/+nJa6S jQMw== X-Gm-Message-State: ACrzQf2INECoOAf8faX8dzyzC1nyN83eJF8Hi3LK4gk46GZX98CwX4C9 Lo9FME/W9yBXcCbXMSgpVHU= X-Google-Smtp-Source: AMsMyM5ftPPoFEo1quN0Z6hL33p1YRq2w6VQdOOo9gynaqnyRjqAWa2xBVxLRplXTS+62wc57oO8Fw== X-Received: by 2002:a17:90a:4607:b0:202:e22d:489c with SMTP id w7-20020a17090a460700b00202e22d489cmr24665392pjg.80.1665282189349; Sat, 08 Oct 2022 19:23:09 -0700 (PDT) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id b10-20020a170902d50a00b0016dc6279ab7sm1737835plg.149.2022.10.08.19.23.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Oct 2022 19:23:08 -0700 (PDT) From: xu.xin.sc@gmail.com X-Google-Original-From: xu.xin16@zte.com.cn To: akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, xu xin , David Hildenbrand , Claudio Imbrenda , Xuexin Jiang , Xiaokai Ran , Yang Yang Subject: [PATCH v2 2/5] ksm: support unsharing zero pages placed by KSM Date: Sun, 9 Oct 2022 02:23:02 +0000 Message-Id: <20221009022302.315318-1-xu.xin16@zte.com.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221009021816.315205-1-xu.xin16@zte.com.cn> References: <20221009021816.315205-1-xu.xin16@zte.com.cn> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1665282190; a=rsa-sha256; cv=none; b=dpSnVC8zjYC0peTG5FrfPeGiASOM1CiAwhV/1vQojQgx7pQaKTOcblxi1snnKF3GcXrw1i jDeTcHJlfaZx4od3VKcEzU5EfdUE16V3o2514qAUy5N9NbmjKTt84qmZyycCj7kBarjgxE 2ncVQpwACmqWRPVos1XGLAnswMhtQBQ= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="hmVN/Vbm"; spf=pass (imf09.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.216.66 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665282190; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pZ2+yi+8Cy6ea/BvAsMEkhQ4pii2mF27mTTm96t3VBo=; b=l9JS2fb0Lghtz3JIL3tGfRQapRRCFHInYYPOXoBxSaRLgT/TARlq8spqsBioOQAxFXFrRt YpxMMYwYn/1UP6zgcNGGA9u/B7uH02o/SGPsRBpvflxI3C9e/0dtnE/YO52S11wEcJd6e7 qtSYclgZ2yS1nvn+fnsrf+O9d7Ri8ps= Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="hmVN/Vbm"; spf=pass (imf09.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.216.66 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: e69z6k5urenbzk6phgehgccwziojsue4 X-Rspamd-Queue-Id: 6FC2A140021 X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1665282190-843545 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin After the commit e86c59b1b12d ("mm/ksm: improve deduplication of zero pages with colouring"), madvise(addr, len, MADV_UNMERGEABLE) and other ways (like write 2 to /sys/kernel/mm/ksm/run) to trigger unsharing will **not** unshare the shared zeropage as placed by KSM (which is against the MADV_UNMERGEABLE documentation at least). To not blindly unshare all shared zero_pages in applicable VMAs, the patch introduces a dedicated flag ZERO_PAGE_FLAG to mark the rmap_items of those shared zero_pages. and guarantee that these rmap_items will be not freed during the time of zero_pages not being writing, so we can only unshare the *KSM-placed* zero_pages. The patch will not degrade the performance of use_zero_pages as it doesn't change the way of merging empty pages in use_zero_pages's feature. Fixes: e86c59b1b12d ("mm/ksm: improve deduplication of zero pages with colouring") Reported-by: David Hildenbrand Cc: Claudio Imbrenda Cc: Xuexin Jiang Signed-off-by: xu xin Co-developed-by: Xiaokai Ran Signed-off-by: Xiaokai Ran Co-developed-by: Yang Yang Signed-off-by: Yang Yang Signed-off-by: xu xin --- mm/ksm.c | 133 ++++++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 103 insertions(+), 30 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 13c60f1071d8..e988a17b837e 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -213,6 +213,7 @@ struct ksm_rmap_item { #define SEQNR_MASK 0x0ff /* low bits of unstable tree seqnr */ #define UNSTABLE_FLAG 0x100 /* is a node of the unstable tree */ #define STABLE_FLAG 0x200 /* is listed from the stable tree */ +#define ZERO_PAGE_FLAG 0x400 /* is zero page placed by KSM */ /* The stable and unstable tree heads */ static struct rb_root one_stable_tree[1] = { RB_ROOT }; @@ -381,14 +382,6 @@ static inline struct ksm_rmap_item *alloc_rmap_item(void) return rmap_item; } -static inline void free_rmap_item(struct ksm_rmap_item *rmap_item) -{ - ksm_rmap_items--; - rmap_item->mm->ksm_rmap_items--; - rmap_item->mm = NULL; /* debug safety */ - kmem_cache_free(rmap_item_cache, rmap_item); -} - static inline struct ksm_stable_node *alloc_stable_node(void) { /* @@ -434,7 +427,8 @@ static inline bool ksm_test_exit(struct mm_struct *mm) * of the process that owns 'vma'. We also do not want to enforce * protection keys here anyway. */ -static int break_ksm(struct vm_area_struct *vma, unsigned long addr) +static int break_ksm(struct vm_area_struct *vma, unsigned long addr, + bool ksm_check_bypass) { struct page *page; vm_fault_t ret = 0; @@ -449,6 +443,16 @@ static int break_ksm(struct vm_area_struct *vma, unsigned long addr) ret = handle_mm_fault(vma, addr, FAULT_FLAG_WRITE | FAULT_FLAG_REMOTE, NULL); + else if (ksm_check_bypass && is_zero_pfn(page_to_pfn(page))) { + /* + * Although it's not ksm page, it's zero page as placed by + * KSM use_zero_page, so we should unshare it when + * ksm_check_bypass is true. + */ + ret = handle_mm_fault(vma, addr, + FAULT_FLAG_WRITE | FAULT_FLAG_REMOTE, + NULL); + } else ret = VM_FAULT_WRITE; put_page(page); @@ -496,6 +500,11 @@ static struct vm_area_struct *find_mergeable_vma(struct mm_struct *mm, return vma; } +/* + * Note: Don't call break_cow() in the context protected by + * mmap_read_lock(), which may cause dead lock because inside + * break_cow mmap_read_lock exists. + */ static void break_cow(struct ksm_rmap_item *rmap_item) { struct mm_struct *mm = rmap_item->mm; @@ -511,10 +520,35 @@ static void break_cow(struct ksm_rmap_item *rmap_item) mmap_read_lock(mm); vma = find_mergeable_vma(mm, addr); if (vma) - break_ksm(vma, addr); + break_ksm(vma, addr, false); mmap_read_unlock(mm); } +/* Only called when rmap_item->address is with ZERO_PAGE_FLAG */ +static inline int unshare_zero_pages(struct ksm_rmap_item *rmap_item) +{ + struct mm_struct *mm = rmap_item->mm; + struct vm_area_struct *vma; + unsigned long addr = rmap_item->address; + int err = -EFAULT; + + vma = vma_lookup(mm, addr); + if (vma) + err = break_ksm(vma, addr, true); + + return err; +} + +static inline void free_rmap_item(struct ksm_rmap_item *rmap_item) +{ + if (rmap_item->address & ZERO_PAGE_FLAG) + unshare_zero_pages(rmap_item); + ksm_rmap_items--; + rmap_item->mm->ksm_rmap_items--; + rmap_item->mm = NULL; /* debug safety */ + kmem_cache_free(rmap_item_cache, rmap_item); +} + static struct page *get_mergeable_page(struct ksm_rmap_item *rmap_item) { struct mm_struct *mm = rmap_item->mm; @@ -825,7 +859,7 @@ static int unmerge_ksm_pages(struct vm_area_struct *vma, if (signal_pending(current)) err = -ERESTARTSYS; else - err = break_ksm(vma, addr); + err = break_ksm(vma, addr, NULL); } return err; } @@ -2017,6 +2051,36 @@ static void stable_tree_append(struct ksm_rmap_item *rmap_item, rmap_item->mm->ksm_merging_pages++; } +static int try_to_merge_with_kernel_zero_page(struct mm_struct *mm, + struct ksm_rmap_item *rmap_item, + struct page *page) +{ + int err = 0; + + if (!(rmap_item->address & ZERO_PAGE_FLAG)) { + struct vm_area_struct *vma; + + mmap_read_lock(mm); + vma = find_mergeable_vma(mm, rmap_item->address); + if (vma) { + err = try_to_merge_one_page(vma, page, + ZERO_PAGE(rmap_item->address)); + } else { + /* If the vma is out of date, we do not need to continue. */ + err = 0; + } + mmap_read_unlock(mm); + /* + * In case of failure, the page was not really empty, so we + * need to continue. Otherwise we're done. + */ + if (!err) + rmap_item->address |= ZERO_PAGE_FLAG; + } + + return err; +} + /* * cmp_and_merge_page - first see if page can be merged into the stable tree; * if not, compare checksum to previous and if it's the same, see if page can @@ -2101,29 +2165,21 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite * Same checksum as an empty page. We attempt to merge it with the * appropriate zero page if the user enabled this via sysfs. */ - if (ksm_use_zero_pages && (checksum == zero_checksum)) { - struct vm_area_struct *vma; - - mmap_read_lock(mm); - vma = find_mergeable_vma(mm, rmap_item->address); - if (vma) { - err = try_to_merge_one_page(vma, page, - ZERO_PAGE(rmap_item->address)); - } else { + if (ksm_use_zero_pages) { + if (checksum == zero_checksum) { + /* If success, just return. Otherwise, continue */ + if (!try_to_merge_with_kernel_zero_page(mm, rmap_item, page)) + return; + } else if (rmap_item->address & ZERO_PAGE_FLAG) { /* - * If the vma is out of date, we do not need to - * continue. + * The page now is not kernel zero page (COW happens to it) + * but the flag of its rmap_item is still zero-page, so need + * to reset the flag and update the corresponding count. */ - err = 0; + rmap_item->address &= PAGE_MASK; } - mmap_read_unlock(mm); - /* - * In case of failure, the page was not really empty, so we - * need to continue. Otherwise we're done. - */ - if (!err) - return; } + tree_rmap_item = unstable_tree_search_insert(rmap_item, page, &tree_page); if (tree_rmap_item) { @@ -2337,6 +2393,23 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) mmap_read_unlock(mm); return rmap_item; } + /* + * Because we want to monitor ksm zero pages which is + * non-anonymous, we must try to return the rmap_items + * of those kernel zero pages which replaces its + * original anonymous empty page due to use_zero_pages's + * feature. + */ + if (is_zero_pfn(page_to_pfn(*page))) { + rmap_item = try_to_get_old_rmap_item(ksm_scan.address, + ksm_scan.rmap_list); + if (rmap_item && (rmap_item->address & ZERO_PAGE_FLAG)) { + ksm_scan.rmap_list = &rmap_item->rmap_list; + ksm_scan.address += PAGE_SIZE; + mmap_read_unlock(mm); + return rmap_item; + } + } next_page: put_page(*page); ksm_scan.address += PAGE_SIZE; From patchwork Sun Oct 9 02:23:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: xu xin X-Patchwork-Id: 13001880 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCEFEC433F5 for ; Sun, 9 Oct 2022 02:23:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 594FD6B0071; Sat, 8 Oct 2022 22:23:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 51DBC6B0073; Sat, 8 Oct 2022 22:23:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 36FFE6B0074; Sat, 8 Oct 2022 22:23:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 1CFDB6B0071 for ; Sat, 8 Oct 2022 22:23:29 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id D19C441222 for ; Sun, 9 Oct 2022 02:23:28 +0000 (UTC) X-FDA: 79999814496.20.29BB705 Received: from mail-pf1-f195.google.com (mail-pf1-f195.google.com [209.85.210.195]) by imf21.hostedemail.com (Postfix) with ESMTP id 720A51C000E for ; Sun, 9 Oct 2022 02:23:28 +0000 (UTC) Received: by mail-pf1-f195.google.com with SMTP id y8so8029822pfp.13 for ; Sat, 08 Oct 2022 19:23:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IBN6huPdGpKqKhIAKLLkaRuR3gOgGOPhsRbsJyWYIRI=; b=XAM200tElyyWyP4GFtgU07tj8DP41cBE6Jr/D2kKGZQS47KOTs70HLvvrDFZ0vQYmE XWUAAYh0ItKz+yYjLgzElI4Isid9D9OBcUb0fVZTdTnb/r7gOBi9b1qZcIWvDN++Xb06 4bz8nkLMLMMTW2yxu3Bd59NlsdTWOgzrigJNJB3cTfGO1gmHwxSJ+OvxyOuKiHs0EKT2 Rx9fX6/B6me420ELRJ1oLiIpvI9P54g/+uGrPrRYdjINHIsPKMteAyWOD16wiFaRI/4W 1FmDlBiAvtpeDGFbLJs7IuP7j9JZGMZG53OtYOdT+RUfEnlGReZ1QSA1qcN6GpmtI31r RFbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IBN6huPdGpKqKhIAKLLkaRuR3gOgGOPhsRbsJyWYIRI=; b=7HxUCWidYi7ckRZCiSU5kxzjEqWVayB5IQobJQMpDyxOzp7+xhsba7uaA84quPxWXo 6MXc6MUU0RzHKgv+CpcnkiXJLDkUmP4ZwBgsXVJ/LNyLwlAc6VRdAGqfflKLPOqR32ZI NvFbBGORjIAsFU5r4MK1s0luOWH3SbyI2tdLQGRxLTaNVQlogsOfr0K/Ct9xKed1y+9R QUM2cWzzc4Tmix4M/UT+m3F1D85lT9zGX5NF1DngK//F6CSbKLgS3j5r6SJCSsnldgh4 Dm0HR67pQ+N3lA/TtcdtJMZ3AafqYJswQOGH1rPJoSdOnDB7LzXggqV3n5K9NdYHYlNF HBng== X-Gm-Message-State: ACrzQf24HWNVN1M5hfR8+iVcLpOcdlTx/wkMManP3E/ZFvs2JgIK1V4H b0dTR0GkhLmh6bKydUscQp8= X-Google-Smtp-Source: AMsMyM6y79bKyw1UWc6p/gUgr0Ws8By8/oW6O0JhPjmAErRjz/PWawklzQiLVCrXJdD+Jqf/+FLiTQ== X-Received: by 2002:a63:450c:0:b0:443:94a1:3703 with SMTP id s12-20020a63450c000000b0044394a13703mr10648210pga.565.1665282207477; Sat, 08 Oct 2022 19:23:27 -0700 (PDT) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id i14-20020a170902cf0e00b0017cc29a5536sm3976353plg.17.2022.10.08.19.23.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Oct 2022 19:23:26 -0700 (PDT) From: xu.xin.sc@gmail.com X-Google-Original-From: xu.xin16@zte.com.cn To: akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, xu xin , Claudio Imbrenda , David Hildenbrand , Xuexin Jiang , Xiaokai Ran , Yang Yang Subject: [PATCH v2 3/5] ksm: count all zero pages placed by KSM Date: Sun, 9 Oct 2022 02:23:21 +0000 Message-Id: <20221009022321.315374-1-xu.xin16@zte.com.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221009021816.315205-1-xu.xin16@zte.com.cn> References: <20221009021816.315205-1-xu.xin16@zte.com.cn> MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665282208; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IBN6huPdGpKqKhIAKLLkaRuR3gOgGOPhsRbsJyWYIRI=; b=SeKLKlwq+RX6FmfQPVlVVykN2E6fBfKDqidJImygSIxvltp3I9smSWPGPMY1HY0zsYzEik q1NbY6kgT6ISN6ObLSnjisxYj8OBgn6HrPSOn8gUu7JhznH93oWaQye4E0ByFnnSIPDEWO QFbv0nBnY7tqjhg/KBFRh1QcHTfIFqQ= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=XAM200tE; spf=pass (imf21.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.210.195 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1665282208; a=rsa-sha256; cv=none; b=fWWeUHLkzbYOQxG/h4C5wnAmoMCqU4RCc1FGYsnzHBhju7FzR1s1aup1dNS8zCNUvtVcbF iK0KsT1UCpCCTEz3KOjixRsZaYkq5mlI6IoDnj5S5vU8sDEu1TyNoWvZjAPamyaETmW7PL j+ifqmP6SspmAoVYZ52YW0wvjZ9JRSg= Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=XAM200tE; spf=pass (imf21.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.210.195 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Stat-Signature: jrn6yhzij9asugwcxhq5gid75ht4ngdd X-Rspamd-Queue-Id: 720A51C000E X-Rspamd-Server: rspam01 X-HE-Tag: 1665282208-838318 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin As pages_sharing and pages_shared don't include the number of zero pages merged by KSM, we cannot know how many pages are zero pages placed by KSM when enabling use_zero_pages, which leads to KSM not being transparent with all actual merged pages by KSM. In the early days of use_zero_pages, zero-pages was unable to get unshared by the ways like MADV_UNMERGEABLE so it's hard to count how many times one of those zeropages was then unmerged. But now, unsharing KSM-placed zero page accurately has been achieved, so we can easily count both how many times a page full of zeroes was merged with zero-page and how many times one of those pages was then unmerged. and so, it helps to estimate memory demands when each and every shared page could get unshared. So we add zero_pages_sharing under /sys/kernel/mm/ksm/ to show the number of all zero pages placed by KSM. Cc: Claudio Imbrenda Cc: David Hildenbrand Cc: Xuexin Jiang Cc: Xiaokai Ran Cc: Yang Yang Signed-off-by: xu xin --- mm/ksm.c | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index e988a17b837e..80672325f179 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -275,6 +275,9 @@ static unsigned int zero_checksum __read_mostly; /* Whether to merge empty (zeroed) pages with actual zero pages */ static bool ksm_use_zero_pages __read_mostly; +/* The number of zero pages placed by KSM use_zero_pages */ +static unsigned long ksm_zero_pages_sharing; + #ifdef CONFIG_NUMA /* Zeroed when merging across nodes is not allowed */ static unsigned int ksm_merge_across_nodes = 1; @@ -541,8 +544,10 @@ static inline int unshare_zero_pages(struct ksm_rmap_item *rmap_item) static inline void free_rmap_item(struct ksm_rmap_item *rmap_item) { - if (rmap_item->address & ZERO_PAGE_FLAG) - unshare_zero_pages(rmap_item); + if (rmap_item->address & ZERO_PAGE_FLAG) { + if (!unshare_zero_pages(rmap_item)) + ksm_zero_pages_sharing--; + } ksm_rmap_items--; rmap_item->mm->ksm_rmap_items--; rmap_item->mm = NULL; /* debug safety */ @@ -2074,8 +2079,10 @@ static int try_to_merge_with_kernel_zero_page(struct mm_struct *mm, * In case of failure, the page was not really empty, so we * need to continue. Otherwise we're done. */ - if (!err) + if (!err) { rmap_item->address |= ZERO_PAGE_FLAG; + ksm_zero_pages_sharing++; + } } return err; @@ -2177,6 +2184,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite * to reset the flag and update the corresponding count. */ rmap_item->address &= PAGE_MASK; + ksm_zero_pages_sharing--; } } @@ -3189,6 +3197,13 @@ static ssize_t pages_volatile_show(struct kobject *kobj, } KSM_ATTR_RO(pages_volatile); +static ssize_t zero_pages_sharing_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + return sysfs_emit(buf, "%ld\n", ksm_zero_pages_sharing); +} +KSM_ATTR_RO(zero_pages_sharing); + static ssize_t stable_node_dups_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) { @@ -3249,6 +3264,7 @@ static struct attribute *ksm_attrs[] = { &merge_across_nodes_attr.attr, #endif &max_page_sharing_attr.attr, + &zero_pages_sharing_attr.attr, &stable_node_chains_attr.attr, &stable_node_dups_attr.attr, &stable_node_chains_prune_millisecs_attr.attr, From patchwork Sun Oct 9 02:23:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: xu xin X-Patchwork-Id: 13001881 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC618C433FE for ; Sun, 9 Oct 2022 02:23:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 436B98E0001; Sat, 8 Oct 2022 22:23:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3BD9A6B0074; Sat, 8 Oct 2022 22:23:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 236F28E0001; Sat, 8 Oct 2022 22:23:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 0EED46B0073 for ; Sat, 8 Oct 2022 22:23:46 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D214481363 for ; Sun, 9 Oct 2022 02:23:45 +0000 (UTC) X-FDA: 79999815210.27.64D7C50 Received: from mail-pj1-f68.google.com (mail-pj1-f68.google.com [209.85.216.68]) by imf22.hostedemail.com (Postfix) with ESMTP id 5D4DDC002B for ; Sun, 9 Oct 2022 02:23:45 +0000 (UTC) Received: by mail-pj1-f68.google.com with SMTP id t12-20020a17090a3b4c00b0020b04251529so7711340pjf.5 for ; Sat, 08 Oct 2022 19:23:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RoDnBFDHVA7LU5T0eht7qIcxVf5BHA7MxEN+oTzlcX0=; b=ofCcVhUEDdNXNttt6XZggV43bk1949OnzTBSrXtuZcJcps0cu2XwVVz6ksK9cV74m+ Ta2GaV5wrtZWb5wpawvNbHLTiT95mPnnX/yBm3ErBO+AVXBdWFhQDeoxvUSH59k1PU3u IGtk3nC1csD38ozpWW24RtpPRl35romwjkmD+tZpABFsdK2ACdW/6wMqagcJxl9rWkW6 vhCCFF2S0ucKwCd7HOE8EscU085BQf4QpHm9C+PYDiV9ITerSaIZtzJSswOnAjU8vBjy kYGBVBbJSu3aznrKX5wbXkUvRNWGUMmKqWKdwYD405pTUoqT7J8uMHj84NJ1JcX1M2D2 fw3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RoDnBFDHVA7LU5T0eht7qIcxVf5BHA7MxEN+oTzlcX0=; b=qrMHwUbG+53VjKxolmJ8goTwIG68/LsjpvvzwJLEnVBhidstG81Mh1z9cVjpIAQKGz GPAlASGrlO+rFoHl/pZIJATciJGY0ULzVB6CN7Mjh90mYj/7EbKkhMz71s03gg0o/d5/ rYKCQnvr3CtDlMRaa0MUfx1HmU83VU1Gsl4jMR3QLgJx9vzyHOMVysE/BttkTTuQsLy+ /9cTxWPvGlzMq4HobMhA7qIN9cRxNa/M3oQLNz0wibjJv5Pa7SlB4PWy+kkXzWTmD41z gsgBdAC16FhgorfGyWNwEtSyJxGodT78CqaRB3Y0r4aFpbeb1Tt9dhJJrHkvz5i3TXJj pQUA== X-Gm-Message-State: ACrzQf0QSm9sVEgnlxTF9CP2o7per6Kh2OqQAirkkiAiGi2mv7AWoa9N mcIZo+ei0yPd9P0yipY4+aE= X-Google-Smtp-Source: AMsMyM7SvcN37Jq7m3iuPnNHgqj3NFAaHIJhjx0KT7J46cKR6I1fV87kmBd2iajvV+IMG3jvXDdU1w== X-Received: by 2002:a17:90a:1b43:b0:203:70c0:9f60 with SMTP id q61-20020a17090a1b4300b0020370c09f60mr23779368pjq.184.1665282224445; Sat, 08 Oct 2022 19:23:44 -0700 (PDT) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id u64-20020a17090a51c600b00200a85fa777sm6826951pjh.1.2022.10.08.19.23.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Oct 2022 19:23:43 -0700 (PDT) From: xu.xin.sc@gmail.com X-Google-Original-From: xu.xin16@zte.com.cn To: akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, xu xin , Claudio Imbrenda , David Hildenbrand , Xuexin Jiang , Xiaokai Ran , Yang Yang Subject: [PATCH v2 4/5] ksm: count zero pages for each process Date: Sun, 9 Oct 2022 02:23:38 +0000 Message-Id: <20221009022338.315428-1-xu.xin16@zte.com.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221009021816.315205-1-xu.xin16@zte.com.cn> References: <20221009021816.315205-1-xu.xin16@zte.com.cn> MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665282225; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RoDnBFDHVA7LU5T0eht7qIcxVf5BHA7MxEN+oTzlcX0=; b=5Dl3ROgQdM4C0uY6jxhKWQcJpipLnc7fApkWBND+NueW+mpXo8T9mNk8BHKEAgpmrTBo3c F94veL/hE57K6EDgfMhoBz5EwYrRPREn+3mb7X6sD4hrhdtxcWrx4N4ockuGPEt92BQKs9 57vdaRZxsF4Ye+KXHbyOece7KZkDooM= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=ofCcVhUE; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.216.68 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1665282225; a=rsa-sha256; cv=none; b=eTJnBzhWwDIWqBo0ncWDUW9qT916CmR+rxsKPwB9eUYNgrQKk5UFDC/m6DzAZ8FXpAxBI/ nXT5aVcKnoMNCyX9b7tJ2Oa6WN0fs0elvEPeNdpqEZqLNa9xVQSY2f+XRiJNBLgAiXguTO RH6D93/s3lz1hNZq5oGRxvQDFbxccnU= Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=ofCcVhUE; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.216.68 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com X-Stat-Signature: 3hrt7e388ra587ogmitsxkrmitm3abwn X-Rspamd-Queue-Id: 5D4DDC002B X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1665282225-678006 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin As the number of ksm zero pages is not included in ksm_merging_pages per process when enabling use_zero_pages, it's unclear of how many actual pages are merged by KSM. To let users accurately estimate their memory demands when unsharing KSM zero-pages, it's necessary to show KSM zero- pages per process. since unsharing zero pages placed by KSM accurately is achieved, then tracking empty pages merging and unmerging is not a difficult thing any longer. Since we already have /proc//ksm_stat, just add the information of zero_pages_sharing in it. Cc: Claudio Imbrenda Cc: David Hildenbrand Cc: Xuexin Jiang Cc: Xiaokai Ran Cc: Yang Yang Signed-off-by: xu xin --- fs/proc/base.c | 1 + include/linux/mm_types.h | 7 ++++++- mm/ksm.c | 6 +++++- 3 files changed, 12 insertions(+), 2 deletions(-) diff --git a/fs/proc/base.c b/fs/proc/base.c index 9e479d7d202b..ac9ebe972be0 100644 --- a/fs/proc/base.c +++ b/fs/proc/base.c @@ -3207,6 +3207,7 @@ static int proc_pid_ksm_stat(struct seq_file *m, struct pid_namespace *ns, mm = get_task_mm(task); if (mm) { seq_printf(m, "ksm_rmap_items %lu\n", mm->ksm_rmap_items); + seq_printf(m, "zero_pages_sharing %lu\n", mm->ksm_zero_pages_sharing); mmput(mm); } diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 500e536796ca..78a4ee264645 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -691,7 +691,7 @@ struct mm_struct { #ifdef CONFIG_KSM /* * Represent how many pages of this process are involved in KSM - * merging. + * merging (not including ksm_zero_pages_sharing). */ unsigned long ksm_merging_pages; /* @@ -699,6 +699,11 @@ struct mm_struct { * including merged and not merged. */ unsigned long ksm_rmap_items; + /* + * Represent how many empty pages are merged with kernel zero + * pages when enabling KSM use_zero_pages. + */ + unsigned long ksm_zero_pages_sharing; #endif #ifdef CONFIG_LRU_GEN struct { diff --git a/mm/ksm.c b/mm/ksm.c index 80672325f179..c1cc3cd370aa 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -545,8 +545,10 @@ static inline int unshare_zero_pages(struct ksm_rmap_item *rmap_item) static inline void free_rmap_item(struct ksm_rmap_item *rmap_item) { if (rmap_item->address & ZERO_PAGE_FLAG) { - if (!unshare_zero_pages(rmap_item)) + if (!unshare_zero_pages(rmap_item)) { ksm_zero_pages_sharing--; + rmap_item->mm->ksm_zero_pages_sharing--; + } } ksm_rmap_items--; rmap_item->mm->ksm_rmap_items--; @@ -2082,6 +2084,7 @@ static int try_to_merge_with_kernel_zero_page(struct mm_struct *mm, if (!err) { rmap_item->address |= ZERO_PAGE_FLAG; ksm_zero_pages_sharing++; + rmap_item->mm->ksm_zero_pages_sharing++; } } @@ -2185,6 +2188,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite */ rmap_item->address &= PAGE_MASK; ksm_zero_pages_sharing--; + rmap_item->mm->ksm_zero_pages_sharing--; } } From patchwork Sun Oct 9 02:23:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: xu xin X-Patchwork-Id: 13001882 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90D27C433FE for ; Sun, 9 Oct 2022 02:23:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 175426B0073; Sat, 8 Oct 2022 22:23:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1005D6B0074; Sat, 8 Oct 2022 22:23:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E93F08E0002; Sat, 8 Oct 2022 22:23:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D73D96B0073 for ; Sat, 8 Oct 2022 22:23:56 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id A41D3C0D86 for ; Sun, 9 Oct 2022 02:23:56 +0000 (UTC) X-FDA: 79999815672.07.7EC817F Received: from mail-pf1-f196.google.com (mail-pf1-f196.google.com [209.85.210.196]) by imf11.hostedemail.com (Postfix) with ESMTP id 4CBD54001A for ; Sun, 9 Oct 2022 02:23:56 +0000 (UTC) Received: by mail-pf1-f196.google.com with SMTP id 3so6521383pfw.4 for ; Sat, 08 Oct 2022 19:23:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=egU+pR1mu0fbwNOlCXxgUzq7eQbUYmzREvrHjyDBDLQ=; b=FdWJE/6vRDAn8HvbajJRjlWuKGPVVZTR6WsylqPO2APZ8f1RcKp5K9kqCiARH3Xq9U V5EnxnOTuyjLZ7nraOZ4hJL9HFQLuXXTypF7DuSg78Hcl7eB2DQNVtm+VEsqbLqrScXj bqfp0HFPy7pjoofF2hiOk6+zjPrkxK2WWrVo08cy6Kso9D7Jpt84VbaHa4i/ZFRw28hr 4bswpnvPHNEPsOAIFef/2tH3zUYktDeBjOuFnxLHWuduMLfLYK5iTU6Gvd7Ycf7Gt80x cKmq/IlFz6SJSzpOfSDUEsLvV/45SjvobCKXVcHSPntydwzKr+D82DPMsZtbETHh1mbN 5Acw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=egU+pR1mu0fbwNOlCXxgUzq7eQbUYmzREvrHjyDBDLQ=; b=T5aLmD1PV2DQpys7cUFaCC/sCKOTF0QnG+YfmCr/33dGuEzDYSsNKiMpqKuDzUN4D7 BfR+1FCHM7pP6ZXcHHjBwcLITQW2j38n8H0OxQAMxIribaDJCq+K29xCBu0zzAxR1wMb tB9Slk9c19hn6UeO8kxAxk77DB1PSsOZd0zJBIZ6SviNCbp0a9ju+ths1XWcASnEBU/U bMdx/MYflu06IX94+QfKzcq/LFrQWzZnQDqUXJyNkXyRYdUDjO2+Ahx0bO6D3A7ZxWA1 0v+n6q9bpuMN0on+5tKgdmskyiCRHXGh+vufgMW7oh/2Ghy1BQ02Q0AcXqoY0ydLaUky Y2lQ== X-Gm-Message-State: ACrzQf2R941EMST8qpCPK0EjXQ6BzOETehmE9svf+glj85urs/TseHHI yZKbaRzezt2+WTMEgBVOnw4= X-Google-Smtp-Source: AMsMyM7o7JzcVNgFATm6Jdoddk7BfGlH243B6v/ltjwvoqVHJMMH9h0GiNCq1I8FeHbulnNpJtrPyA== X-Received: by 2002:a63:ed0a:0:b0:442:2514:95f5 with SMTP id d10-20020a63ed0a000000b00442251495f5mr11113497pgi.402.1665282235332; Sat, 08 Oct 2022 19:23:55 -0700 (PDT) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id e17-20020a17090301d100b00177f4ef7970sm4018521plh.11.2022.10.08.19.23.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Oct 2022 19:23:54 -0700 (PDT) From: xu.xin.sc@gmail.com X-Google-Original-From: xu.xin16@zte.com.cn To: akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, xu xin , Xiaokai Ran , Yang Yang , Jiang Xuexin , Claudio Imbrenda , David Hildenbrand Subject: [PATCH v2 5/5] ksm: add zero_pages_sharing documentation Date: Sun, 9 Oct 2022 02:23:49 +0000 Message-Id: <20221009022349.315482-1-xu.xin16@zte.com.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221009021816.315205-1-xu.xin16@zte.com.cn> References: <20221009021816.315205-1-xu.xin16@zte.com.cn> MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665282236; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=egU+pR1mu0fbwNOlCXxgUzq7eQbUYmzREvrHjyDBDLQ=; b=lTtq5slpVDspAKhAMUA/+tOLGefVMf49fqBuIkkCQNBXa2f0IHgdBoYrkJNObuMMhFnkF2 nFaQSVCMha/rKc4SqHM8LjbSDt8OZqIs3JC6w4TNwKIdx9ZRtU/D4gx2MTizyrRktUvrtj FQvAcmo1Z4J8+UceLNqfVVkgU+nGMHc= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="FdWJE/6v"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf11.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.210.196 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1665282236; a=rsa-sha256; cv=none; b=LO1OKz8Y6T5Cee9zCjjJ1IsgRJjdqlH90NqnEqhIchiSFwqq6h2QNTjCP4UrwKSKDPtCxV 0HFcpZSONk+lacCB9EJQPovj1gsMMW+pzyNkuK+nyMRWfH/31OJIhxXqAJxD7CLxObt0QI efWylAPAmIzOk9K545O1OsxdFx/jEOM= X-Stat-Signature: quyjdc9r31xk4k4o99m9cxc3rjgyas6t X-Rspamd-Server: rspam09 X-Rspam-User: Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="FdWJE/6v"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf11.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.210.196 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com X-Rspamd-Queue-Id: 4CBD54001A X-HE-Tag: 1665282236-257835 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000484, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin When enabling use_zero_pages, pages_sharing cannot represent how much memory saved indeed. zero_pages_sharing + pages_sharing does. add the description of zero_pages_sharing. Cc: Xiaokai Ran Cc: Yang Yang Cc: Jiang Xuexin Cc: Claudio Imbrenda Cc: David Hildenbrand Signed-off-by: xu xin --- Documentation/admin-guide/mm/ksm.rst | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/mm/ksm.rst b/Documentation/admin-guide/mm/ksm.rst index fb6ba2002a4b..484665aa7418 100644 --- a/Documentation/admin-guide/mm/ksm.rst +++ b/Documentation/admin-guide/mm/ksm.rst @@ -162,7 +162,7 @@ The effectiveness of KSM and MADV_MERGEABLE is shown in ``/sys/kernel/mm/ksm/``: pages_shared how many shared pages are being used pages_sharing - how many more sites are sharing them i.e. how much saved + how many more sites are sharing them pages_unshared how many pages unique but repeatedly checked for merging pages_volatile @@ -173,6 +173,14 @@ stable_node_chains the number of KSM pages that hit the ``max_page_sharing`` limit stable_node_dups number of duplicated KSM pages +zero_pages_sharing + how many empty pages are sharing kernel zero page(s) instead of + with each other as it would happen normally. only effective when + enabling ``use_zero_pages`` knob. + +If ``use_zero_pages`` is 0, only ``pages_sharing`` can represents how +much saved. Otherwise, ``pages_sharing`` + ``zero_pages_sharing`` +represents how much saved actually. A high ratio of ``pages_sharing`` to ``pages_shared`` indicates good sharing, but a high ratio of ``pages_unshared`` to ``pages_sharing``