From patchwork Fri Jul 7 04:47:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhongkun He X-Patchwork-Id: 13304388 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7DDCCEB64D9 for ; Fri, 7 Jul 2023 04:47:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 091E68D0005; Fri, 7 Jul 2023 00:47:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 041778D0001; Fri, 7 Jul 2023 00:47:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E4C998D0005; Fri, 7 Jul 2023 00:47:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D68D88D0001 for ; Fri, 7 Jul 2023 00:47:18 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A8ADF1402FA for ; Fri, 7 Jul 2023 04:47:18 +0000 (UTC) X-FDA: 80983581756.06.46C7F69 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by imf20.hostedemail.com (Postfix) with ESMTP id DCC971C000B for ; Fri, 7 Jul 2023 04:47:16 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=i9R4JehL; spf=pass (imf20.hostedemail.com: domain of hezhongkun.hzk@bytedance.com designates 209.85.214.182 as permitted sender) smtp.mailfrom=hezhongkun.hzk@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688705236; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=OEV7S2qQa9wjcpqCPZF1/IRLU/XLUimMDUCsPYYW7kI=; b=2OFos/Zfz1VA6gBZ819dHcYdakoImZzjY4Iks/O5vHuEyiZQ1tR14MwwSOsoRC1dK0F4+T Dh4PYKyGcrNe8wa2NsWgcgAnVeBAGnORewiaz6PXud3Wa+I8TkK0W0yUAgbvD8NVXgvO3h Fy2cKrhNEOTyPjJEr04iFIPcXAOsGpQ= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=i9R4JehL; spf=pass (imf20.hostedemail.com: domain of hezhongkun.hzk@bytedance.com designates 209.85.214.182 as permitted sender) smtp.mailfrom=hezhongkun.hzk@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688705236; a=rsa-sha256; cv=none; b=M+qNxFbAh9EsGSJsjE+fc6WCPZcWDM/2qlTcJLnDEVrJAG2sQdomCBlBMoqCZuCDWiGvkX dvECNApSpAiqyIcgefSCSpiOxYEyWig5C+7ZazDhRjr8Mds3K71hhKchyOmCyZJQzJIsk8 77Uj0iRpI2snDcJCOQaAHBCIzyrZ448= Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-1b8b2cb7384so7967115ad.2 for ; Thu, 06 Jul 2023 21:47:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1688705236; x=1691297236; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=OEV7S2qQa9wjcpqCPZF1/IRLU/XLUimMDUCsPYYW7kI=; b=i9R4JehL62yeUI0J/4ssJQxvz0puzqqchTK/dmDx0jOYqynb15e1FVB9HaIbdAnBap I+9q1hHV+sLd0FkRPQrxa357li+iU7oRQ3ZKljZNWV3srIwdrp0BycDD7H+epgpwzfRQ YVWBEAsXX8PF0UhPux74/+tZUH3QnrxpBO4WdqdABk7EeoVbQxDpUwN7GkVD8PfbmKbi jrqdFSt13I8trb9p3hzK1fwPq4uIr0nK3BpF2FMwkbqRd5wXqzHhSxMUm3PxZ/vdQo/j 5S8xAvzry2rY9pLS05xsu74C9bbszUTiarWjDUT6mckjYUbOG3yUlgSfPTRsN+uftCxC r1qQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688705236; x=1691297236; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=OEV7S2qQa9wjcpqCPZF1/IRLU/XLUimMDUCsPYYW7kI=; b=AF6NwSNIhw8jCZ1P3CwaRR7KVloJBSSVcZfklub44NWXKJZ/L10BZZtRrwryMwz95t zK6qm0WoWOWAGnh9N8uzD4UsV7qFd4Ei95xGzotgax4Wl+ara+nuIKSRQfG9eTBQTIMP /vbz5dPKvgrsZIoSRPyBv8gIHdQ4a7QZOqV8WlU2f8v+oQL+EJQ9tM4p3/v60AiZl8jn JUvvnpxVfAjzMFCw+Uvw8pTH0UDKronZkFIor9BjaRT01pUpKvIXDdu3gnmtFx3wzsG/ f/ELbQVbO5oTBM1RG/d6ES1PuYmLZFSoOHYSARXBFcPOB0dcavrclzYb2NWNyYoSHKFL grWA== X-Gm-Message-State: ABy/qLYO6C7HH9cCO71AXFJ8tiybgJDd3xF5VpVniIcnSy9yo5qSbOOO FHr3N6F4oHbU7W7XEK/UGnET7w== X-Google-Smtp-Source: APBJJlEtaEiw8usJCj2cjDIpbdnmQoBl+moxhFrnsASFMEpmDxsNajjhnvllwpDNFcF/3sV84z1rXQ== X-Received: by 2002:a17:902:7405:b0:1b8:b827:aa8e with SMTP id g5-20020a170902740500b001b8b827aa8emr3111867pll.11.1688705235754; Thu, 06 Jul 2023 21:47:15 -0700 (PDT) Received: from Tower.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id i11-20020a170902eb4b00b001b86deadfddsm2181858pli.264.2023.07.06.21.47.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 06 Jul 2023 21:47:15 -0700 (PDT) From: Zhongkun He To: minchan@kernel.org, senozhatsky@chromium.org, mhocko@suse.com Cc: david@redhat.com, yosryahmed@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Zhongkun He Subject: [RFC PATCH 2/2] zram: charge the compressed RAM to the page's memcgroup Date: Fri, 7 Jul 2023 12:47:07 +0800 Message-Id: <20230707044707.1169285-1-hezhongkun.hzk@bytedance.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Rspamd-Queue-Id: DCC971C000B X-Rspam-User: X-Stat-Signature: m3ft9ap67a6pzuuj883ertyg8mhzxmj7 X-Rspamd-Server: rspam01 X-HE-Tag: 1688705236-506105 X-HE-Meta: U2FsdGVkX19y91una1VWCKBfrCB6zxYwfdJSlAkWpU5enujrzx4L92KWYOpDNUyc6K2D+cQmLotGFQV581143DfWxMcwKtINaX7DPKXXVm50wUc45U4y73clopjbkRYZvx3dsRuA93fcidNYKwadG2PxYeYeqFGSmjEVGLC00c7RLCCQwy0qpVbFIPDlC0qzSZifSW9yAM02K1+kGb42vgwmG0ZgsLp8znx1/0mXbXD2wcEyYtNN/M8L12Z5qsTpZzGiG23ey5cYOEtEU8T1E9DUI53kSLvQqpQvGuHJugb7lffESMLjEOyMn76jPQkpAOh4vNjpl2oRUYYTpcbG752KJCn/iF4AcOZyIizjg8AckdHoSvv2SorsEi2SoDSELQFNN4eibo8SBls27+BFbw3+ZSWr1qPwJwl892VyBted0ppEopI8jyUNwWICNOQzntTGtfczeUD/6uRFXiai1R8/p3Lb1ovWQ/Jnew7v0XMcSfX2eXY+XQPuJBDd7zLFWFdeQMr5rauGBtVx9d270Lk6X5ldixbM1e/m17AuHfs8c9IDf02BW/cqjs8TOfhJ/gkv0IRVSGbw4V3u61mt0qHIXAsDVf528+QHUZ0x06TDWxXVr3SwZMbvtK0pBxPn/VNccoFuIuJgqkdPzDyW7YTxO5y2yih22gG3Q/v0cB2IZLWXy1Md+JFlzPLRj9GJ4zBcPWx7WNTeKVQBgs8hT5L1AzqWwUsjYRe8eKT2ozfTVX84ux+qpkTqhA1NzxdtLZzMEcpnAizz/eDkt4a5aDKebmxfFO7I5MWhRF5Z2+pEXF0Ccr3RAi0yeMPfXKsSLSdk20I/irFfEZE+xS3smGckUkppdhKMTYfFxj+5CLfDubxockEDd5SJ3XyKlUTcJIBL7vIsGWcD/PBk+3tp0JBCKsCyPURLwtBY89Ow8hIecCV6w0EOlxsqTOoTFvDDqNOHUEFvh2j+tQjd2ck IAMTfy2C Czm/4EpGz/bwZFTK5jtRqSo1BEQvtj2/w3mvbewYxkLe1ietk4yWF1WAXnrgQQ2xQydIwiwXIzPqTl1a6FH33rNGrU9WqrcH44n2U7j0YBhkqDd2mXJsXE+IHLvgHUIkEcXKIIxCxIc06fzookZ3gC0PwSdV3k9PoIC7hHmGD0+dSn4XgVhZC9StVLKLRC4XmMD9+TZ45ctAuohM6Wk0/W/ZckC6hf4Fo2SmMh+TuP9ripswtnoTljBFOakYRMOAqEQ54nOtVnbQYaubgT7gtzjwOqKiES/Z+Q3oBAi02kow0amUhyDq8jeseD768s8b3nJBBx1/L4gXmP9S7+Bf83m2sV0eiVVhMJ5M3eA/Jf50tyWgMn8ha/lfM1Vn82wXNijHcAajLpIj8JdaBEyuQqCDjKwp2IE7J03BH X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The compressed RAM is currently charged to kernel, not to any memory cgroup. This patch can charge the pages regardless of direct or indirect zram usage. Direct zram usage by process within a cgroup will fail to charge if there is no memory. Indirect zram usage by process within a cgroup via swap in PF_MEMALLOC context, will charge successfully. Signed-off-by: Zhongkun He --- drivers/block/zram/zram_drv.c | 43 +++++++++++++++++++++++++++++++++++ drivers/block/zram/zram_drv.h | 1 + 2 files changed, 44 insertions(+) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 5676e6dd5b16..770c7d58afec 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -33,6 +33,7 @@ #include #include #include +#include #include "zram_drv.h" @@ -135,6 +136,18 @@ static void zram_set_obj_size(struct zram *zram, zram->table[index].flags = (flags << ZRAM_FLAG_SHIFT) | size; } +static inline void zram_set_obj_cgroup(struct zram *zram, u32 index, + struct obj_cgroup *objcg) +{ + zram->table[index].objcg = objcg; +} + +static inline struct obj_cgroup *zram_get_obj_cgroup(struct zram *zram, + u32 index) +{ + return zram->table[index].objcg; +} + static inline bool zram_allocated(struct zram *zram, u32 index) { return zram_get_obj_size(zram, index) || @@ -1256,6 +1269,7 @@ static bool zram_meta_alloc(struct zram *zram, u64 disksize) static void zram_free_page(struct zram *zram, size_t index) { unsigned long handle; + struct obj_cgroup *objcg; #ifdef CONFIG_ZRAM_MEMORY_TRACKING zram->table[index].ac_time = 0; @@ -1289,6 +1303,13 @@ static void zram_free_page(struct zram *zram, size_t index) goto out; } + objcg = zram_get_obj_cgroup(zram, index); + if (objcg) { + obj_cgroup_uncharge_zram(objcg, zram_get_obj_size(zram, index)); + obj_cgroup_put(objcg); + zram_set_obj_cgroup(zram, index, NULL); + } + handle = zram_get_handle(zram, index); if (!handle) return; @@ -1419,6 +1440,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) struct zcomp_strm *zstrm; unsigned long element = 0; enum zram_pageflags flags = 0; + struct obj_cgroup *objcg = NULL; mem = kmap_atomic(page); if (page_same_filled(mem, &element)) { @@ -1494,6 +1516,14 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return -ENOMEM; } + objcg = get_obj_cgroup_from_page(page); + if (objcg && obj_cgroup_charge_zram(objcg, comp_len)) { + zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP]); + zs_free(zram->mem_pool, handle); + obj_cgroup_put(objcg); + return -ENOMEM; + } + dst = zs_map_object(zram->mem_pool, handle, ZS_MM_WO); src = zstrm->buffer; @@ -1526,6 +1556,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) } else { zram_set_handle(zram, index, handle); zram_set_obj_size(zram, index, comp_len); + zram_set_obj_cgroup(zram, index, objcg); } zram_slot_unlock(zram, index); @@ -1581,6 +1612,8 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, unsigned int comp_len_new; unsigned int class_index_old; unsigned int class_index_new; + struct obj_cgroup *objcg = NULL; + unsigned int noreclaim_flag; u32 num_recomps = 0; void *src, *dst; int ret; @@ -1692,11 +1725,21 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page, zs_unmap_object(zram->mem_pool, handle_new); + /* + * Recompress will reclaim some memory, so we set the reclaim + * flag in order to charge comp_len_new successfully. + */ + noreclaim_flag = memalloc_noreclaim_save(); + objcg = zram_get_obj_cgroup(zram, index); + obj_cgroup_get(objcg); zram_free_page(zram, index); + obj_cgroup_charge_zram(objcg, GFP_KERNEL, comp_len_new); + zram_set_obj_cgroup(zram, index, objcg); zram_set_handle(zram, index, handle_new); zram_set_obj_size(zram, index, comp_len_new); zram_set_priority(zram, index, prio); + memalloc_noreclaim_restore(noreclaim_flag); atomic64_add(comp_len_new, &zram->stats.compr_data_size); atomic64_inc(&zram->stats.pages_stored); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index ca7a15bd4845..959d721d5474 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -72,6 +72,7 @@ struct zram_table_entry { #ifdef CONFIG_ZRAM_MEMORY_TRACKING ktime_t ac_time; #endif + struct obj_cgroup *objcg; }; struct zram_stats {