From patchwork Sat Oct 8 07:08:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: xu xin X-Patchwork-Id: 13001711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D64F9C433F5 for ; Sat, 8 Oct 2022 07:09:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6DE016B0072; Sat, 8 Oct 2022 03:09:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 68D8A6B0073; Sat, 8 Oct 2022 03:09:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 52E326B0074; Sat, 8 Oct 2022 03:09:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 433696B0072 for ; Sat, 8 Oct 2022 03:09:04 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 00E2C1A0886 for ; Sat, 8 Oct 2022 07:09:03 +0000 (UTC) X-FDA: 79996905408.07.B5EB7CA Received: from mail-pg1-f196.google.com (mail-pg1-f196.google.com [209.85.215.196]) by imf02.hostedemail.com (Postfix) with ESMTP id 7900280023 for ; Sat, 8 Oct 2022 07:09:03 +0000 (UTC) Received: by mail-pg1-f196.google.com with SMTP id b5so6417384pgb.6 for ; Sat, 08 Oct 2022 00:09:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vwgE5luutAnRT+PkK5v8g5jEsmfaMaq0PpMbhLvP0lE=; b=KJH9tcBDp0sap3/a+uZB7MoB+le+Hj9E/RTZw8cXEhbev7uOO9m7VY6/LSqyjHSoWi vXFWcFDRVI4HnN2FfCyepXKx7Mm91gWvEe+682mOpaoxrqv2PHpP81rIZORMZACTcD2J 88NwHLa/ZTvZQcaMOoR4VA70825G9cVEGkPTfXkSf659eBsxTRT7WO0kYuEEd2PsBDUB hIJSykq3f1G2O0UB8I3QSl7do5Os+9lJAnvlS+GdJqi6yrk/pm3SLlFDeyugOo+MEngk s7/ahbIa17kdYFWZdMuq0F7DRE82lOWFnfoEBNSX3HpZBeLG1uTSPYPEAsOYCF9WKTsu OCrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vwgE5luutAnRT+PkK5v8g5jEsmfaMaq0PpMbhLvP0lE=; b=mFCCGuxu0RL6IgXUERlhZaQPYFvbMx+Y4Yvr4nEjNOsw+SCSOTPSYSCwse9SIm+Hq2 a7OU6UhVRav6pVsXiffhNgVUpxyuejuJ9zVMSxX2Og76J8tB1k0l2N8aqkJFYT0w7SH/ 3V2G7JsMynj1uxfk1KUwdydg8O0VwpEah1lJo0tiBxaPYZ2XHh/lwvsBVtnE6aCnu3Kq jpQS5kgkjc2zoaB/S8g+SFEjFtthTZCb5OjN3Tkuyv7ytWapWVFo3F5s0/OID2T/ADPT TibZJsm4KXhtmXmBzB0G55CYNdUE180QvL3N4TeHOx8IVvU/nFD8rIvlG74FT9NFsMkT cDLA== X-Gm-Message-State: ACrzQf0pnre173iTPnFq2IxYxZtTmGqG0IyPM8v8wHPlG4fK2EvOPpya jcUguKVbV0K7A41cumwWr88= X-Google-Smtp-Source: AMsMyM7XNpyJYi8+xXQI7TAFwE3wq3DKHhs27o0T2vzkkVChvR32huCVWy4yr1IJOdVca1qkQz1HwQ== X-Received: by 2002:a05:6a00:2c98:b0:553:e926:c47a with SMTP id ef24-20020a056a002c9800b00553e926c47amr8991912pfb.14.1665212942508; Sat, 08 Oct 2022 00:09:02 -0700 (PDT) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id ij24-20020a170902ab5800b0017f49b41c12sm2593722plb.173.2022.10.08.00.08.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Oct 2022 00:09:01 -0700 (PDT) From: xu.xin.sc@gmail.com X-Google-Original-From: xu.xin16@zte.com.cn To: akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, xu xin , Claudio Imbrenda , David Hildenbrand , Xuexin Jiang , Xiaokai Ran , Yang Yang Subject: [PATCH 1/5] ksm: abstract the function try_to_get_old_rmap_item Date: Sat, 8 Oct 2022 07:08:53 +0000 Message-Id: <20221008070854.308851-1-xu.xin16@zte.com.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221008070156.308465-1-xu.xin16@zte.com.cn> References: <20221008070156.308465-1-xu.xin16@zte.com.cn> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1665212943; a=rsa-sha256; cv=none; b=jcl2vkhkWLgpDEI+B9l3gKON7uCMs1bHVK/uGZe7Q3w4JZiuskpNmfuQwBr2fOm7afyPns PeAJRBXCsL2JCAFiZG5iYuxcIHrElPbJ0+kXNst7LX48z1RCybP6MW9gyp2NihSMxueA1l z/jySVXCczmijVvPUgbLH6vTxYvFehM= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=KJH9tcBD; spf=pass (imf02.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.215.196 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665212943; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vwgE5luutAnRT+PkK5v8g5jEsmfaMaq0PpMbhLvP0lE=; b=hdFqGsZDLaM1FAbvzCsgPo58bCCaGpyqlACRytdiSO+85bAVoHxMrqEArCIUHTNo/uQVlP TMDD1+UFEWj6EU6kbe122jsuADS0KzqDwQyYb4YvB8nZhXXWGeN+1cYSyhoVbSIfXqhoTL 8pKJTC4poBT+byBTCUaOZ1p+X3qF50M= Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=KJH9tcBD; spf=pass (imf02.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.215.196 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: yw9y5hbe9axyb3obwhtcrgmwzmkguaca X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 7900280023 X-Rspam-User: X-HE-Tag: 1665212943-699142 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin A new function try_to_get_old_rmap_item is abstracted from get_next_rmap_item. This function will be reused by the subsequent patches about counting ksm_zero_pages. The patch improves the readability and reusability of KSM code. Signed-off-by: xu xin Cc: Claudio Imbrenda Cc: David Hildenbrand Cc: Xuexin Jiang Cc: Xiaokai Ran Cc: Yang Yang --- mm/ksm.c | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index c19fcca9bc03..5b68482d2b3b 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -2187,14 +2187,11 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite } } -static struct ksm_rmap_item *get_next_rmap_item(struct ksm_mm_slot *mm_slot, - struct ksm_rmap_item **rmap_list, - unsigned long addr) +static struct ksm_rmap_item *try_to_get_old_rmap_item(unsigned long addr, + struct ksm_rmap_item **rmap_list) { - struct ksm_rmap_item *rmap_item; - while (*rmap_list) { - rmap_item = *rmap_list; + struct ksm_rmap_item *rmap_item = *rmap_list; if ((rmap_item->address & PAGE_MASK) == addr) return rmap_item; if (rmap_item->address > addr) @@ -2204,6 +2201,21 @@ static struct ksm_rmap_item *get_next_rmap_item(struct ksm_mm_slot *mm_slot, free_rmap_item(rmap_item); } + return NULL; +} + +static struct ksm_rmap_item *get_next_rmap_item(struct ksm_mm_slot *mm_slot, + struct ksm_rmap_item **rmap_list, + unsigned long addr) +{ + struct ksm_rmap_item *rmap_item; + + /* lookup if we have a old rmap_item matching the addr*/ + rmap_item = try_to_get_old_rmap_item(addr, rmap_list); + if (rmap_item) + return rmap_item; + + /* Need to allocate a new rmap_item */ rmap_item = alloc_rmap_item(); if (rmap_item) { /* It has already been zeroed */ From patchwork Sat Oct 8 07:05:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: xu xin X-Patchwork-Id: 13001706 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DD6DC433F5 for ; Sat, 8 Oct 2022 07:05:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E5C1C6B0072; Sat, 8 Oct 2022 03:05:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E0EB16B0073; Sat, 8 Oct 2022 03:05:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CAB666B0074; Sat, 8 Oct 2022 03:05:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B2AD76B0072 for ; Sat, 8 Oct 2022 03:05:23 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 79608A045B for ; Sat, 8 Oct 2022 07:05:23 +0000 (UTC) X-FDA: 79996896126.05.372001A Received: from mail-pf1-f193.google.com (mail-pf1-f193.google.com [209.85.210.193]) by imf14.hostedemail.com (Postfix) with ESMTP id 11A08100017 for ; Sat, 8 Oct 2022 07:05:21 +0000 (UTC) Received: by mail-pf1-f193.google.com with SMTP id y8so6655017pfp.13 for ; Sat, 08 Oct 2022 00:05:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+TTOF9N9iR/s34f2B+vKzoMurrZM7tfnNofX5SBhUeU=; b=TKxT76iUIyp2PdMUPm342LyyWfmbAtq+yidjutwticBt2ujhaJ1f0TLf2ypF/kfBWQ qHIuSZY6cE+vjahnKM8hYO5ZCmyNbtNB+ZfHMMpuaQD9kzZAfXyJinED5JEhnUraRpuM l4F+BozbWKphh14ws5PArG5QhHIx/4OOZYJsfy1hn5Ni2vTWpGLXUqubIsmE6Vs5D160 Fa3EmC2Bq8EHZkxX0Z86TPE23d2t+mGWFGmdnmLyC89SM07tMZz+/1eY2Hy4ji5Nl8lX RSzkpJazS1MM5PtI0L44aC1YmZ+K7tDSyb0S1CNVeoMoH1AjLuk/IsOIHrOkSp8LrEax hTIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+TTOF9N9iR/s34f2B+vKzoMurrZM7tfnNofX5SBhUeU=; b=kbzw76LvpqdvP0hGinwyckZGFSmQS3Agl+xFdirRa4QUNYBw8NUJNTrYh0xNGxIdVT miOTHnasVSfZxrJrhdqmXFs7mDvFikwcGuQs4HseO48VrYTzsVw1LDjXs4Y0Jj3HyqKi gZMJgw46hWUPBa7dDLT+szV5hswUmEXE4f6mvSFOdx6FhG2ATGfyD6a55TRUFR/ptnkU HdgGE3rSbYY3/Qx1Jsq66ciU8Csndz+Z5/9EQxhNU4MoLoSAx+Yqhv+mv+0slz60o7Rr u4hn9Aytb5HAO0wQphiMcsfNaNSMHO9eUoswittn58HxhRCUXa/bzDHO+Ju/Nq2PcDNQ ON8Q== X-Gm-Message-State: ACrzQf38HWkKYDzUJWSd5QH6yFFVdCzctwm0iJ5Rl/MHajVwTcK3APcf q56x2wJC106IHSM9BWzyvKQ= X-Google-Smtp-Source: AMsMyM4HcJKDqVB5Z4cZhIadTZQlGre0ug56Zj60bcBqI2lL1UKGlbciEIhI9K8zvwfqB4nf21XIAQ== X-Received: by 2002:a63:2c8b:0:b0:41c:5f9e:a1d6 with SMTP id s133-20020a632c8b000000b0041c5f9ea1d6mr7685355pgs.601.1665212721029; Sat, 08 Oct 2022 00:05:21 -0700 (PDT) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id b8-20020a170903228800b00176ea6ce0efsm2665411plh.109.2022.10.08.00.05.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Oct 2022 00:05:20 -0700 (PDT) From: xu.xin.sc@gmail.com X-Google-Original-From: xu.xin16@zte.com.cn To: akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, xu xin , David Hildenbrand , Claudio Imbrenda , Xuexin Jiang , Xiaokai Ran , Yang Yang Subject: [PATCH 2/5] ksm: support unsharing zero pages placed by KSM Date: Sat, 8 Oct 2022 07:05:05 +0000 Message-Id: <20221008070505.308580-1-xu.xin16@zte.com.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221008070156.308465-1-xu.xin16@zte.com.cn> References: <20221008070156.308465-1-xu.xin16@zte.com.cn> MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665212722; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+TTOF9N9iR/s34f2B+vKzoMurrZM7tfnNofX5SBhUeU=; b=Cna7E7Qh4MDWSyyCfttXccjhGpUYTgEU5aXBFOYY6W+XR5j/mL+CNyMmqSbPfqRlShfGVa 3eXpXWBrgMAFXKugJmFbArm+jpIkATKyfcZFXyushSXbqbr6tPKXhn3x84RcbrnNESFE+V V0PC4P1QFZ4p4t8+CnLOrgnZJhHr6FE= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=TKxT76iU; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf14.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.210.193 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1665212722; a=rsa-sha256; cv=none; b=AaibX9iN8vAtPD3dJAgpAp6vD1Wt/TD6ZWA/G2SMU2+NxjnNhOx4fQ36p0gQY73/Gf6D9g LajzJFNg6staUaLLTCQW674Bf1eIyqM0gJBJfD3rHDup7pu9myeqQmAN7d6HUzzCtARrcj Hm9D/t8M6dyPK/TEZnboGNDkr8Ys8Jk= Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=TKxT76iU; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf14.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.210.193 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com X-Stat-Signature: 84nskwso1wiugrjs3gbynmn9c6dtkah3 X-Rspamd-Queue-Id: 11A08100017 X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1665212721-712408 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin After the commit e86c59b1b12d ("mm/ksm: improve deduplication of zero pages with colouring"), madvise(addr, len, MADV_UNMERGEABLE) and other ways (like write 2 to /sys/kernel/mm/ksm/run) to trigger unsharing will **not** unshare the shared zeropage as placed by KSM (which is against the MADV_UNMERGEABLE documentation at least). To not blindly unshare all shared zero_pages in applicable VMAs, the patch introduces a dedicated flag ZERO_PAGE_FLAG to mark the rmap_items of those shared zero_pages. and guarantee that these rmap_items will be not freed during the time of zero_pages not being writing, so we can only unshare the *KSM-placed* zero_pages. The patch will not degrade the performance of use_zero_pages as it doesn't change the way of merging empty pages in use_zero_pages's feature. Fixes: e86c59b1b12d ("mm/ksm: improve deduplication of zero pages with colouring") Reported-by: David Hildenbrand Cc: Claudio Imbrenda Cc: Xuexin Jiang Signed-off-by: xu xin Co-developed-by: Xiaokai Ran Signed-off-by: Xiaokai Ran Co-developed-by: Yang Yang Signed-off-by: Yang Yang Signed-off-by: xu xin --- mm/ksm.c | 134 ++++++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 104 insertions(+), 30 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 5b68482d2b3b..75978f7eeed1 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -213,6 +213,7 @@ struct ksm_rmap_item { #define SEQNR_MASK 0x0ff /* low bits of unstable tree seqnr */ #define UNSTABLE_FLAG 0x100 /* is a node of the unstable tree */ #define STABLE_FLAG 0x200 /* is listed from the stable tree */ +#define ZERO_PAGE_FLAG 0x400 /* is zero page placed by KSM */ /* The stable and unstable tree heads */ static struct rb_root one_stable_tree[1] = { RB_ROOT }; @@ -381,14 +382,6 @@ static inline struct ksm_rmap_item *alloc_rmap_item(void) return rmap_item; } -static inline void free_rmap_item(struct ksm_rmap_item *rmap_item) -{ - ksm_rmap_items--; - rmap_item->mm->ksm_rmap_items--; - rmap_item->mm = NULL; /* debug safety */ - kmem_cache_free(rmap_item_cache, rmap_item); -} - static inline struct ksm_stable_node *alloc_stable_node(void) { /* @@ -434,7 +427,8 @@ static inline bool ksm_test_exit(struct mm_struct *mm) * of the process that owns 'vma'. We also do not want to enforce * protection keys here anyway. */ -static int break_ksm(struct vm_area_struct *vma, unsigned long addr) +static int break_ksm(struct vm_area_struct *vma, unsigned long addr, + bool ksm_check_bypass) { struct page *page; vm_fault_t ret = 0; @@ -449,6 +443,16 @@ static int break_ksm(struct vm_area_struct *vma, unsigned long addr) ret = handle_mm_fault(vma, addr, FAULT_FLAG_WRITE | FAULT_FLAG_REMOTE, NULL); + else if (ksm_check_bypass && is_zero_pfn(page_to_pfn(page))) { + /* + * Although it's not ksm page, it's zero page as placed by + * KSM use_zero_page, so we should unshare it when + * ksm_check_bypass is true. + */ + ret = handle_mm_fault(vma, addr, + FAULT_FLAG_WRITE | FAULT_FLAG_REMOTE, + NULL); + } else ret = VM_FAULT_WRITE; put_page(page); @@ -496,6 +500,11 @@ static struct vm_area_struct *find_mergeable_vma(struct mm_struct *mm, return vma; } +/* + * Note: Don't call break_cow() in the context protected by + * mmap_read_lock(), which may cause dead lock because inside + * break_cow mmap_read_lock exists. + */ static void break_cow(struct ksm_rmap_item *rmap_item) { struct mm_struct *mm = rmap_item->mm; @@ -511,10 +520,35 @@ static void break_cow(struct ksm_rmap_item *rmap_item) mmap_read_lock(mm); vma = find_mergeable_vma(mm, addr); if (vma) - break_ksm(vma, addr); + break_ksm(vma, addr, false); mmap_read_unlock(mm); } +/* Only called when rmap_item->address is with ZERO_PAGE_FLAG */ +static inline int unshare_zero_pages(struct ksm_rmap_item *rmap_item) +{ + struct mm_struct *mm = rmap_item->mm; + struct vm_area_struct *vma; + unsigned long addr = rmap_item->address; + int err = -EFAULT; + + vma = vma_lookup(mm, addr); + if (vma) + err = break_ksm(vma, addr, true); + + return err; +} + +static inline void free_rmap_item(struct ksm_rmap_item *rmap_item) +{ + if (rmap_item->address & ZERO_PAGE_FLAG) + unshare_zero_pages(rmap_item); + ksm_rmap_items--; + rmap_item->mm->ksm_rmap_items--; + rmap_item->mm = NULL; /* debug safety */ + kmem_cache_free(rmap_item_cache, rmap_item); +} + static struct page *get_mergeable_page(struct ksm_rmap_item *rmap_item) { struct mm_struct *mm = rmap_item->mm; @@ -825,7 +859,7 @@ static int unmerge_ksm_pages(struct vm_area_struct *vma, if (signal_pending(current)) err = -ERESTARTSYS; else - err = break_ksm(vma, addr); + err = break_ksm(vma, addr, NULL); } return err; } @@ -2017,6 +2051,36 @@ static void stable_tree_append(struct ksm_rmap_item *rmap_item, rmap_item->mm->ksm_merging_pages++; } +static int try_to_merge_with_kernel_zero_page(struct mm_struct *mm, + struct ksm_rmap_item *rmap_item, + struct page *page) +{ + int err = 0; + + if (!(rmap_item->address & ZERO_PAGE_FLAG)) { + struct vm_area_struct *vma; + + mmap_read_lock(mm); + vma = find_mergeable_vma(mm, rmap_item->address); + if (vma) { + err = try_to_merge_one_page(vma, page, + ZERO_PAGE(rmap_item->address)); + } else { + /* If the vma is out of date, we do not need to continue. */ + err = 0; + } + mmap_read_unlock(mm); + /* + * In case of failure, the page was not really empty, so we + * need to continue. Otherwise we're done. + */ + if (!err) + rmap_item->address |= ZERO_PAGE_FLAG; + } + + return err; +} + /* * cmp_and_merge_page - first see if page can be merged into the stable tree; * if not, compare checksum to previous and if it's the same, see if page can @@ -2101,29 +2165,21 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite * Same checksum as an empty page. We attempt to merge it with the * appropriate zero page if the user enabled this via sysfs. */ - if (ksm_use_zero_pages && (checksum == zero_checksum)) { - struct vm_area_struct *vma; - - mmap_read_lock(mm); - vma = find_mergeable_vma(mm, rmap_item->address); - if (vma) { - err = try_to_merge_one_page(vma, page, - ZERO_PAGE(rmap_item->address)); - } else { + if (ksm_use_zero_pages) { + if (checksum == zero_checksum) { + /* If success, just return. Otherwise, continue */ + if (!try_to_merge_with_kernel_zero_page(mm, rmap_item, page)) + return; + } else if (rmap_item->address & ZERO_PAGE_FLAG) { /* - * If the vma is out of date, we do not need to - * continue. + * The page now is not kernel zero page (COW happens to it) + * but the flag of its rmap_item is still zero-page, so need + * to reset the flag and update the corresponding count. */ - err = 0; + rmap_item->address &= PAGE_MASK; } - mmap_read_unlock(mm); - /* - * In case of failure, the page was not really empty, so we - * need to continue. Otherwise we're done. - */ - if (!err) - return; } + tree_rmap_item = unstable_tree_search_insert(rmap_item, page, &tree_page); if (tree_rmap_item) { @@ -2197,6 +2253,7 @@ static struct ksm_rmap_item *try_to_get_old_rmap_item(unsigned long addr, if (rmap_item->address > addr) break; *rmap_list = rmap_item->rmap_list; + /* running here indicates these pages have been unmerged */ remove_rmap_item_from_tree(rmap_item); free_rmap_item(rmap_item); } @@ -2336,6 +2393,23 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) mmap_read_unlock(mm); return rmap_item; } + /* + * Because we want to monitor ksm zero pages which is + * non-anonymous, we must try to return the rmap_items + * of those kernel zero pages which replaces its + * original anonymous empty page due to use_zero_pages's + * feature. + */ + if (is_zero_pfn(page_to_pfn(*page))) { + rmap_item = try_to_get_old_rmap_item(ksm_scan.address, + ksm_scan.rmap_list); + if (rmap_item && (rmap_item->address & ZERO_PAGE_FLAG)) { + ksm_scan.rmap_list = &rmap_item->rmap_list; + ksm_scan.address += PAGE_SIZE; + mmap_read_unlock(mm); + return rmap_item; + } + } next_page: put_page(*page); ksm_scan.address += PAGE_SIZE; From patchwork Sat Oct 8 07:06:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: xu xin X-Patchwork-Id: 13001707 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F001CC433F5 for ; Sat, 8 Oct 2022 07:06:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 311286B0073; Sat, 8 Oct 2022 03:06:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2BFC06B0074; Sat, 8 Oct 2022 03:06:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 161306B0075; Sat, 8 Oct 2022 03:06:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 02B806B0073 for ; Sat, 8 Oct 2022 03:06:26 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id A74061C5C26 for ; Sat, 8 Oct 2022 07:06:25 +0000 (UTC) X-FDA: 79996898730.28.5464944 Received: from mail-pj1-f65.google.com (mail-pj1-f65.google.com [209.85.216.65]) by imf22.hostedemail.com (Postfix) with ESMTP id 469ECC0023 for ; Sat, 8 Oct 2022 07:06:25 +0000 (UTC) Received: by mail-pj1-f65.google.com with SMTP id x1-20020a17090ab00100b001fda21bbc90so9027916pjq.3 for ; Sat, 08 Oct 2022 00:06:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rrPOD+KhqOFJAEX56XxLC5jHOX5mIwwsCLX+qV5KxT4=; b=LKDDwupQjCXTZCVBlTOqwhB0ZHmLddxOJkaQI9iqpkAm4GBejemWj47pXzHJZBE5Lc 0kZ/v/vwQymfmd/+TkJJVFJtMeMkKl/ULS2YAYwgePcUhsmNUrl9ZgpX1cbeJSnaJwU7 Dw3yNw+wUw+i15xdQ1Pl1Ic5rS8WSgUeA01eGMhN+WNe97QRrGTnY6QLVNe2RgXLVjO9 el8zjjlkjzMHc8ytX9dgKZnz3BWj5/r03iIFgEhnR0kKILmiYW1Y6RkZuKhvl6BHxHG9 34p4HIHximpsSckGEyvvIOaAJ+xecU+iCKiGqP3oQc5TZqpjPxi7Fu2b74fhPq8vydcc 65GA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rrPOD+KhqOFJAEX56XxLC5jHOX5mIwwsCLX+qV5KxT4=; b=1iyruBIHrH8LG+Kyb7avQoMX4ohUhk/TZPbJTgzJzwfxkVCSgFaFEkaZMMvQcq/I3S k76tYhhUANPaKwvvWA4z0M/D9/DT1i3JfwL3IeLtofM69C1zr3F8Dd7IZQrOlIHWBo/Q uGPaMLxg2tB2l2MqARp4cZn7kgHUq/gb4Q3zdUl34kOm970Agi1nioPXVl8T+PTBTrmT YImehL6esUJddwrJViZXOfX9uP3FdBIOECrIs4JsYGRec1RhvNEPf4/7QE9Ctz7X/sqc 1viqHkdbSp8RcX0yZV5SwnkMyTzu4VLkI9cyqxnpXbDXBwt/1H0An8X6Z6JFfBgnEDcy xe1A== X-Gm-Message-State: ACrzQf1aeNUillb1ESWOvHpJt36zQVKI8uWgt99k7I+qrBDx8NR0QfVf b+8xEvbSEeI5V0iMLb/DE0d24z0ulMs= X-Google-Smtp-Source: AMsMyM7NGewJHu/y94jUlrNlEopVSi5jWxeOZAwuJpFctyaF2a3hIMtPMHYAHu62m2I9qFp6SMx9Mg== X-Received: by 2002:a17:90b:2651:b0:20a:daaf:75f0 with SMTP id pa17-20020a17090b265100b0020adaaf75f0mr9004581pjb.142.1665212784299; Sat, 08 Oct 2022 00:06:24 -0700 (PDT) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id o1-20020a17090a678100b002004380686bsm5557157pjj.46.2022.10.08.00.06.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Oct 2022 00:06:22 -0700 (PDT) From: xu.xin.sc@gmail.com X-Google-Original-From: xu.xin16@zte.com.cn To: akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, xu xin , Claudio Imbrenda , David Hildenbrand , Xuexin Jiang , Xiaokai Ran , Yang Yang Subject: [PATCH 3/5] ksm: count all zero pages placed by KSM Date: Sat, 8 Oct 2022 07:06:13 +0000 Message-Id: <20221008070613.308637-1-xu.xin16@zte.com.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221008070156.308465-1-xu.xin16@zte.com.cn> References: <20221008070156.308465-1-xu.xin16@zte.com.cn> MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665212785; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rrPOD+KhqOFJAEX56XxLC5jHOX5mIwwsCLX+qV5KxT4=; b=iMhpVIsRqcz67dnyx2DYANQPwJbyuDObRk3tSPO1ByPrTH0ixmIfN9UTEXg/0bj/DHK0OP /Jjd0WH4wrIpI8gFw5DK+ihUZXIoRRsmdqdwCCFBl0nrZVEzMLOesXhrH54iK2eg4FYo3V Q+hz7sK9dIED/fyPWIICHQgMO5AfYGU= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=LKDDwupQ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.216.65 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1665212785; a=rsa-sha256; cv=none; b=OSU72R2GCD3OlhxTLMdyNZZIweR4PzUlaFMFrPCzl/B2AKYuQw5JJLPzPLuDZO6JzFOg1C 9DRRamd0RBNY5Y8assAkmS1o623Tf9AcajciuEi3CkMks/jntuXa+uUWUhQMnr6hmdFJSW ahlYqKzt7ZjpzWTCQ2aqEUQ+1a4VbEc= Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=LKDDwupQ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf22.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.216.65 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com X-Stat-Signature: yfcecgba1qbry5iczmke9thapbsqs8sp X-Rspamd-Queue-Id: 469ECC0023 X-Rspam-User: X-Rspamd-Server: rspam06 X-HE-Tag: 1665212785-453140 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin As pages_sharing and pages_shared don't include the number of zero pages merged by KSM, we cannot know how many pages are zero pages placed by KSM when enabling use_zero_pages, which leads to KSM not being transparent with all actual merged pages by KSM. In the early days of use_zero_pages, zero-pages was unable to get unshared by the ways like MADV_UNMERGEABLE so it's hard to count how many times one of those zeropages was then unmerged. But now, unsharing KSM-placed zero page accurately has been achieved, so we can easily count both how many times a page full of zeroes was merged with zero-page and how many times one of those pages was then unmerged. and so, it helps to estimate memory demands when each and every shared page could get unshared. So we add zero_pages_sharing under /sys/kernel/mm/ksm/ to show the number of all zero pages placed by KSM. Cc: Claudio Imbrenda Cc: David Hildenbrand Cc: Xuexin Jiang Cc: Xiaokai Ran Cc: Yang Yang Signed-off-by: xu xin --- mm/ksm.c | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 75978f7eeed1..9b7c28abfc89 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -275,6 +275,9 @@ static unsigned int zero_checksum __read_mostly; /* Whether to merge empty (zeroed) pages with actual zero pages */ static bool ksm_use_zero_pages __read_mostly; +/* The number of zero pages placed by KSM use_zero_pages */ +static unsigned long ksm_zero_pages_sharing; + #ifdef CONFIG_NUMA /* Zeroed when merging across nodes is not allowed */ static unsigned int ksm_merge_across_nodes = 1; @@ -541,8 +544,10 @@ static inline int unshare_zero_pages(struct ksm_rmap_item *rmap_item) static inline void free_rmap_item(struct ksm_rmap_item *rmap_item) { - if (rmap_item->address & ZERO_PAGE_FLAG) - unshare_zero_pages(rmap_item); + if (rmap_item->address & ZERO_PAGE_FLAG) { + if (!unshare_zero_pages(rmap_item)) + ksm_zero_pages_sharing--; + } ksm_rmap_items--; rmap_item->mm->ksm_rmap_items--; rmap_item->mm = NULL; /* debug safety */ @@ -2074,8 +2079,10 @@ static int try_to_merge_with_kernel_zero_page(struct mm_struct *mm, * In case of failure, the page was not really empty, so we * need to continue. Otherwise we're done. */ - if (!err) + if (!err) { rmap_item->address |= ZERO_PAGE_FLAG; + ksm_zero_pages_sharing++; + } } return err; @@ -2177,6 +2184,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite * to reset the flag and update the corresponding count. */ rmap_item->address &= PAGE_MASK; + ksm_zero_pages_sharing--; } } @@ -3189,6 +3197,13 @@ static ssize_t pages_volatile_show(struct kobject *kobj, } KSM_ATTR_RO(pages_volatile); +static ssize_t zero_pages_sharing_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + return sysfs_emit(buf, "%ld\n", ksm_zero_pages_sharing); +} +KSM_ATTR_RO(zero_pages_sharing); + static ssize_t stable_node_dups_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) { @@ -3249,6 +3264,7 @@ static struct attribute *ksm_attrs[] = { &merge_across_nodes_attr.attr, #endif &max_page_sharing_attr.attr, + &zero_pages_sharing_attr.attr, &stable_node_chains_attr.attr, &stable_node_dups_attr.attr, &stable_node_chains_prune_millisecs_attr.attr, From patchwork Sat Oct 8 07:06:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: xu xin X-Patchwork-Id: 13001708 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 391DEC433F5 for ; Sat, 8 Oct 2022 07:06:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C38216B0074; Sat, 8 Oct 2022 03:06:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BE7C26B0075; Sat, 8 Oct 2022 03:06:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A61B76B0078; Sat, 8 Oct 2022 03:06:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 90BE66B0074 for ; Sat, 8 Oct 2022 03:06:51 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 68B6A16015D for ; Sat, 8 Oct 2022 07:06:51 +0000 (UTC) X-FDA: 79996899822.04.D4B2AF4 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) by imf03.hostedemail.com (Postfix) with ESMTP id 0AD642000E for ; Sat, 8 Oct 2022 07:06:50 +0000 (UTC) Received: by mail-pf1-f194.google.com with SMTP id i3so6672729pfk.9 for ; Sat, 08 Oct 2022 00:06:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=U+7cGrSCaB8toCY9LzJTV6eREW3WTAoZDgmk6SweSp0=; b=NI+szN5GnNITjU+LpGlHOTnVh2BNC7vF7YjihBuKzgQvD13g7J70KAt8n01JOrPCGE pfjhUSYusC/G0Rj7vJSaSLmiou6u2wtpZFAj1ygAIjlpggrW7H/0ADcIQM+2DCw11/Tl eM5igZHSw+2qV2dqucWmz5HaEv1wTvni+/GCpHOgRN6oxpsoNJqMmEQOfR65/8AFnqPM 77uv6tGOZYwqebSlCFSjeOWOlK449c1myqh4id/BZLKFXjA3Kd64Cf69h70BkmHFoPL6 F4mERXoSD7EgxjZAVhhAkfR9ipQXWDPZiRaWAN0WpfzMb7XTy+xLNCIR4Xhf6WuBnyfz Jvpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U+7cGrSCaB8toCY9LzJTV6eREW3WTAoZDgmk6SweSp0=; b=LIUiE2g/Hfqco9d88RoT52K4UMhFmQ4jt/UMv5wLlJHiV+sTdUtFBGHtY4MIfi3TkC iJjefTMN8PnqTSjv/AoXCCjJINqsf297D0QGmJzVLl4iY+btN046KGwWToQtN3rusBlU ZbRly+xF3zORC/zRqhinBtr9RvVwuXiS/pe24f4TGZwWicnvzrnZITGHnA5naKblEiep 9LBdWEdefMv3kNoi+nrxZkhxxrM5aTB3/HC413waZppv9ziuYJFUHQM1+sEh3/S71039 M5uiDKBARMsyHEGocK+S8v0P4g9MumJIVkWfuVimxVGGJAI42N/yXxmkSJtFtJYGH7QM OCWA== X-Gm-Message-State: ACrzQf0Ku9z4KJEnAe6/8Z+U5ewFNxD5z3+18vlaNqbyyyHWdetfMQsC NANRslMK3iymKaa7IGQxAPA= X-Google-Smtp-Source: AMsMyM5y3td7v+CEksjuK/XyY+otk4kTI3C1PTEjt8Qr6W7VoZaJT/xWMQBDRZnZZMRIN+xcHpeLqw== X-Received: by 2002:a63:4283:0:b0:457:dced:8ba3 with SMTP id p125-20020a634283000000b00457dced8ba3mr7736754pga.220.1665212810118; Sat, 08 Oct 2022 00:06:50 -0700 (PDT) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id pc14-20020a17090b3b8e00b0020a28156e11sm4281098pjb.26.2022.10.08.00.06.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Oct 2022 00:06:49 -0700 (PDT) From: xu.xin.sc@gmail.com X-Google-Original-From: xu.xin16@zte.com.cn To: akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, xu xin , Claudio Imbrenda , David Hildenbrand , Xuexin Jiang , Xiaokai Ran , Yang Yang Subject: [PATCH 4/5] ksm: count zero pages for each process Date: Sat, 8 Oct 2022 07:06:42 +0000 Message-Id: <20221008070642.308693-1-xu.xin16@zte.com.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221008070156.308465-1-xu.xin16@zte.com.cn> References: <20221008070156.308465-1-xu.xin16@zte.com.cn> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1665212811; a=rsa-sha256; cv=none; b=E2k6LRGRXPQJg4oQCNXAIVVJmt24bHNbH8s8ukBmt8RUSyzmSij1ejXkmWlwHetGMRy6Dg FuZhZUICs5vLjNPp8PZVGTMIVw2G2wFO2T80qlpzMuZr5VhEJKgX/N7Vpaw0d0B2GhPZkZ aUHvkXAycLI8jKW8xeqb6IVMwYro5uk= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=NI+szN5G; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf03.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.210.194 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665212811; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=U+7cGrSCaB8toCY9LzJTV6eREW3WTAoZDgmk6SweSp0=; b=slIibdk/XmhTtREFarLHPPbUMLIIcGSbsmjNBx068JII+4MvUylaiYIkROewlNK2Hx2lal IdIdq5ux+vYGtf9wsmQMOG+wtugh2N3Ehoc5xpcL7Zx0OZxr62AzCTbxNImpIhLgL+2SwV RghMvQo3ek53xFhrivPeo+KadKs9Pmg= X-Rspam-User: X-Stat-Signature: rpf34znh4x4xjjsbc86xmesnedi8awug X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 0AD642000E Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=NI+szN5G; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf03.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.210.194 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com X-HE-Tag: 1665212810-737743 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin As the number of ksm zero pages is not included in ksm_merging_pages per process when enabling use_zero_pages, it's unclear of how many actual pages are merged by KSM. To let users accurately estimate their memory demands when unsharing KSM zero-pages, it's necessary to show KSM zero- pages per process. since unsharing zero pages placed by KSM accurately is achieved, then tracking empty pages merging and unmerging is not a difficult thing any longer. Since we already have /proc//ksm_stat, just add the information of zero_pages_sharing in it. Cc: Claudio Imbrenda Cc: David Hildenbrand Cc: Xuexin Jiang Cc: Xiaokai Ran Cc: Yang Yang Signed-off-by: xu xin --- fs/proc/base.c | 1 + include/linux/mm_types.h | 7 ++++++- mm/ksm.c | 3 +++ 3 files changed, 10 insertions(+), 1 deletion(-) diff --git a/fs/proc/base.c b/fs/proc/base.c index 9e479d7d202b..ac9ebe972be0 100644 --- a/fs/proc/base.c +++ b/fs/proc/base.c @@ -3207,6 +3207,7 @@ static int proc_pid_ksm_stat(struct seq_file *m, struct pid_namespace *ns, mm = get_task_mm(task); if (mm) { seq_printf(m, "ksm_rmap_items %lu\n", mm->ksm_rmap_items); + seq_printf(m, "zero_pages_sharing %lu\n", mm->ksm_zero_pages_sharing); mmput(mm); } diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 500e536796ca..78a4ee264645 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -691,7 +691,7 @@ struct mm_struct { #ifdef CONFIG_KSM /* * Represent how many pages of this process are involved in KSM - * merging. + * merging (not including ksm_zero_pages_sharing). */ unsigned long ksm_merging_pages; /* @@ -699,6 +699,11 @@ struct mm_struct { * including merged and not merged. */ unsigned long ksm_rmap_items; + /* + * Represent how many empty pages are merged with kernel zero + * pages when enabling KSM use_zero_pages. + */ + unsigned long ksm_zero_pages_sharing; #endif #ifdef CONFIG_LRU_GEN struct { diff --git a/mm/ksm.c b/mm/ksm.c index 9b7c28abfc89..a889b1925f51 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -547,6 +547,7 @@ static inline void free_rmap_item(struct ksm_rmap_item *rmap_item) if (rmap_item->address & ZERO_PAGE_FLAG) { if (!unshare_zero_pages(rmap_item)) ksm_zero_pages_sharing--; + rmap_item->mm->ksm_zero_pages_sharing--; } ksm_rmap_items--; rmap_item->mm->ksm_rmap_items--; @@ -2082,6 +2083,7 @@ static int try_to_merge_with_kernel_zero_page(struct mm_struct *mm, if (!err) { rmap_item->address |= ZERO_PAGE_FLAG; ksm_zero_pages_sharing++; + rmap_item->mm->ksm_zero_pages_sharing++; } } @@ -2185,6 +2187,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite */ rmap_item->address &= PAGE_MASK; ksm_zero_pages_sharing--; + rmap_item->mm->ksm_zero_pages_sharing--; } } From patchwork Sat Oct 8 07:07:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: xu xin X-Patchwork-Id: 13001710 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60FC5C433F5 for ; Sat, 8 Oct 2022 07:07:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E5A696B0072; Sat, 8 Oct 2022 03:07:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E09BB6B0074; Sat, 8 Oct 2022 03:07:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CAAA56B0075; Sat, 8 Oct 2022 03:07:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B813B6B0072 for ; Sat, 8 Oct 2022 03:07:08 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 7E99A1A025B for ; Sat, 8 Oct 2022 07:07:08 +0000 (UTC) X-FDA: 79996900536.01.01D1894 Received: from mail-pj1-f67.google.com (mail-pj1-f67.google.com [209.85.216.67]) by imf16.hostedemail.com (Postfix) with ESMTP id 1A46C180010 for ; Sat, 8 Oct 2022 07:07:07 +0000 (UTC) Received: by mail-pj1-f67.google.com with SMTP id q10-20020a17090a304a00b0020b1d5f6975so5373275pjl.0 for ; Sat, 08 Oct 2022 00:07:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=egU+pR1mu0fbwNOlCXxgUzq7eQbUYmzREvrHjyDBDLQ=; b=GCLrh81Q8BMsf/xFgK/vL+6CrEoKNCw9UdiEuxw25yIGphTObD36x0dLFYVlVR7GOV mR32kmjJyEpYA2B4oaSUD9R2Iiy3c4MDfBsxxaGqEh/FawOGokUamFhme8kAN/eBb7WA isgPT/9okXWWL6sPCfZ8/Amnd1dgR0wAj1gyXZs2bBvCiLxRn4PixDSpqtfPJI2zuk0h lfvvAbMexpk+H6PLk75/+tSMKmcMzMxzFi6wbWA/adtSNCPYFFtgJIe7m00u49YMjVD9 QNrbLWYxIab29YLMFZ8QU8U04+0JTr+Z85icPXZ3OQKQHMlys82pWCbEZwixUjtwrMnf YMIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=egU+pR1mu0fbwNOlCXxgUzq7eQbUYmzREvrHjyDBDLQ=; b=YVnhNM9Zcmn8WP1EivlX+nVe99thJjsh3w7AFlkwIbLXQUZ9n/32+6mDj5mVuH4Tto 6xxwSVhC1XkiJh0Ryqgh0JzG+W/ofbZtdPptF6MvI/sGsSnZRfhEHdZDFuUGkRS5ehZr C0beSObuji8Oo2/QVjSWrZ7AhLhFBp8d3ySfLIZPVQBy+E0+Bk0jdPh6qe8IhqhbtsKI W7HqZK9WTrykG7B+Uxp/zoWnGFn2E/rXtWRin6qC/FakyOHk/VnsnBc0vLvka/OkryTB iZwlgXXwadI00iP0XI6R5n/tguU5IWLDbDtsSOGnCJ271AinW0ZgkMTJfYJYh6C2Iux8 +vJg== X-Gm-Message-State: ACrzQf1fj4b6rZstK5SkS7/tLzy+oAAYDK2jsnVeIKNI7rq5/rcSZ35J 5ks8bDxDg2ENc0ZJYHwOG0Q= X-Google-Smtp-Source: AMsMyM580/WE7K2J+FERsSfMBW9Alh5kTunQhUorLRTAyVYTY4npo87/H8JMIX2EgmHypeztFjSsig== X-Received: by 2002:a17:902:dad2:b0:17f:8290:fce0 with SMTP id q18-20020a170902dad200b0017f8290fce0mr8449772plx.168.1665212827179; Sat, 08 Oct 2022 00:07:07 -0700 (PDT) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id x20-20020aa79ad4000000b00562a5f29ac1sm2772054pfp.100.2022.10.08.00.07.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 08 Oct 2022 00:07:06 -0700 (PDT) From: xu.xin.sc@gmail.com X-Google-Original-From: xu.xin16@zte.com.cn To: akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, xu xin , Xiaokai Ran , Yang Yang , Jiang Xuexin , Claudio Imbrenda , David Hildenbrand Subject: [PATCH 5/5] ksm: add zero_pages_sharing documentation Date: Sat, 8 Oct 2022 07:07:01 +0000 Message-Id: <20221008070701.308747-1-xu.xin16@zte.com.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221008070156.308465-1-xu.xin16@zte.com.cn> References: <20221008070156.308465-1-xu.xin16@zte.com.cn> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1665212828; a=rsa-sha256; cv=none; b=0/0Or0BUYf9ouy0arQCy5K+94XW/Ivd6YHQInGhkkUDeK//EXT58U+PHJdC5JRP47oqkhU jJux4Xge1e81vTmkg89IXoFYcnoGRCXJznMmUPB6Wigfoo2Ym+4m/x0EC0H1AQvY3i3Rok FU6ZwENXhNXGQtrQ81qmurp5zMjCpas= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=GCLrh81Q; spf=pass (imf16.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.216.67 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665212828; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=egU+pR1mu0fbwNOlCXxgUzq7eQbUYmzREvrHjyDBDLQ=; b=3plbcXEzARF9sBTbsOVgrbHmX8dhPg4EC0ANf0wg+6npSNzzEfL8dGptcdbXANRKuM8m/0 HNz5R7LUTA0Jja6Vo110+dxvP1qwhZ30IHQVVFPKNxrO8NDTf6cshSws07B9UIpIbi8Xm/ OxwEmiicKcPNmUae/CH1jJB1Bg85R2M= Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=GCLrh81Q; spf=pass (imf16.hostedemail.com: domain of xu.xin.sc@gmail.com designates 209.85.216.67 as permitted sender) smtp.mailfrom=xu.xin.sc@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: p8hj9mshdrhipb9s3egpydhkgsffotsd X-Rspamd-Queue-Id: 1A46C180010 X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1665212827-518537 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000006, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin When enabling use_zero_pages, pages_sharing cannot represent how much memory saved indeed. zero_pages_sharing + pages_sharing does. add the description of zero_pages_sharing. Cc: Xiaokai Ran Cc: Yang Yang Cc: Jiang Xuexin Cc: Claudio Imbrenda Cc: David Hildenbrand Signed-off-by: xu xin --- Documentation/admin-guide/mm/ksm.rst | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/mm/ksm.rst b/Documentation/admin-guide/mm/ksm.rst index fb6ba2002a4b..484665aa7418 100644 --- a/Documentation/admin-guide/mm/ksm.rst +++ b/Documentation/admin-guide/mm/ksm.rst @@ -162,7 +162,7 @@ The effectiveness of KSM and MADV_MERGEABLE is shown in ``/sys/kernel/mm/ksm/``: pages_shared how many shared pages are being used pages_sharing - how many more sites are sharing them i.e. how much saved + how many more sites are sharing them pages_unshared how many pages unique but repeatedly checked for merging pages_volatile @@ -173,6 +173,14 @@ stable_node_chains the number of KSM pages that hit the ``max_page_sharing`` limit stable_node_dups number of duplicated KSM pages +zero_pages_sharing + how many empty pages are sharing kernel zero page(s) instead of + with each other as it would happen normally. only effective when + enabling ``use_zero_pages`` knob. + +If ``use_zero_pages`` is 0, only ``pages_sharing`` can represents how +much saved. Otherwise, ``pages_sharing`` + ``zero_pages_sharing`` +represents how much saved actually. A high ratio of ``pages_sharing`` to ``pages_shared`` indicates good sharing, but a high ratio of ``pages_unshared`` to ``pages_sharing``