From patchwork Wed Dec 25 20:37:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vitaly Wool X-Patchwork-Id: 11310307 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3D8EF921 for ; Wed, 25 Dec 2019 20:38:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EF5AB2075E for ; Wed, 25 Dec 2019 20:38:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=konsulko.com header.i=@konsulko.com header.b="fiFMOy3P" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EF5AB2075E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=konsulko.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F1A458E0005; Wed, 25 Dec 2019 15:38:02 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id ECBA58E0001; Wed, 25 Dec 2019 15:38:02 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D93878E0005; Wed, 25 Dec 2019 15:38:02 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0211.hostedemail.com [216.40.44.211]) by kanga.kvack.org (Postfix) with ESMTP id C0F0C8E0001 for ; Wed, 25 Dec 2019 15:38:02 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 79981181AEF10 for ; Wed, 25 Dec 2019 20:38:02 +0000 (UTC) X-FDA: 76304825604.17.apple20_2906e216b0e11 X-Spam-Summary: 50,0,0,9dea766b967008e2,d41d8cd98f00b204,vitaly.wool@konsulko.com,::ddstreet@ieee.org:akpm@linux-foundation.org:vitaly.wool@konsulko.com,RULES_HIT:2:41:69:355:379:541:800:960:967:968:973:988:989:1260:1311:1314:1345:1437:1515:1535:1605:1730:1747:1777:1792:2198:2199:2393:2525:2560:2563:2682:2685:2693:2859:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4049:4119:4321:4605:5007:6119:6261:6653:7576:7903:8603:9010:9025:9592:10004:11026:11232:11473:11658:11914:12043:12291:12296:12297:12438:12517:12519:12555:12679:12683:12895:13894:13972:14394:21080:21444:21451:21627:21740:21788:21987:21990:30054:30070,0,RBL:209.85.167.67:@konsulko.com:.lbl8.mailshell.net-62.14.0.100 66.201.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:32,LUA_SUMMARY:none X-HE-Tag: apple20_2906e216b0e11 X-Filterd-Recvd-Size: 8722 Received: from mail-lf1-f67.google.com (mail-lf1-f67.google.com [209.85.167.67]) by imf44.hostedemail.com (Postfix) with ESMTP for ; Wed, 25 Dec 2019 20:38:01 +0000 (UTC) Received: by mail-lf1-f67.google.com with SMTP id i23so17277732lfo.7 for ; Wed, 25 Dec 2019 12:38:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=konsulko.com; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=Mo2ak1kIA7Rw/E2vkibwgkDlZa6RVJ+/+x9in4v2A+o=; b=fiFMOy3PPQnWleLeZf2E/HIsfquNr0fOCm0mwFId1RDXV/NiPkyIzx388kRuJWIqB4 HcZLZV2yW+2mdYfUwK8bHObWEn9LlIzVFrsZ8lpHaWdeOP3koOUshKcayO0MUMMBLMgZ 5eu9DXJB0fr62eKxcxa7TQaBnAPgjRQIQ1390= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=Mo2ak1kIA7Rw/E2vkibwgkDlZa6RVJ+/+x9in4v2A+o=; b=B+zM7fNK1P9mNXtAL02ns3a+FsdPMmzZ5LQJaS5014MA4L2R+OEoJIFH4Xdw9O64Sr cZlsOezTBR88znQhBbCOn3jFfe+l8C3HY+oEIkkO2bDHWgX6L3pPu/rDV5e/w6le/ETe Yd9rj2MLvWdiLnkxjFG+SCi16f8vD7aACfiqw+x2U8Pf8TgK53ZRqO/fZHm0+/h5xzHq HU6z6b6muctnodPsKcGcZdu/nsUXrNHdncdmcsqPxhoooXBNwkBpyEawScvD2BtH6ipm /GBBalF7oVk1pf8UZU/HAheEhc1+8qAXXVj+u4JbxIrDDn4aUVMzfDFb2MeRy1Wfusx/ t2rQ== X-Gm-Message-State: APjAAAWXYxtYQLk01g8ioTcK0JQ9h4gYFM9PA/Zeph8rAHYAIX6ntTIC IK14vDfeANpUxT2UixTVjvLN6eplKcG1rQ== X-Google-Smtp-Source: APXvYqxxBU+CpDZzzvTzY9BEUmCuHYdoWRa4wJQmRHwSgAG/xg3a0Oi9d04VxoM2CkQV5O8Y4KEueA== X-Received: by 2002:a19:5201:: with SMTP id m1mr25162098lfb.114.1577306279981; Wed, 25 Dec 2019 12:37:59 -0800 (PST) Received: from taos.konsulko.bg (lan.nucleusys.com. [92.247.61.126]) by smtp.gmail.com with ESMTPSA id j204sm11984575lfj.38.2019.12.25.12.37.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Dec 2019 12:37:59 -0800 (PST) From: vitaly.wool@konsulko.com To: linux-mm@kvack.org Cc: ddstreet@ieee.org, akpm@linux-foundation.org, Vitaly Wool Subject: [PATCH] zswap: add allocation hysteresis if pool limit is hit Date: Wed, 25 Dec 2019 22:37:30 +0200 Message-Id: <20191225203730.4632-1-vitaly.wool@konsulko.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vitaly Wool zswap will always try to shrink pool when zswap is full. If there is a high pressure on zswap it will result in flipping pages in and out zswap pool without any real benefit, and the overall system performance will drop. The previous discussion on this subject [1] ended up with a suggestion to implement a sort of hysteresis to refuse taking pages into zswap pool until it has sufficient space if the limit has been hit. This is my take on this. Hysteresis is controlled with a sysfs-configurable parameter 'accept_threhsold_percent'. It specifies the threshold at which zswap would start accepting pages again after it became full. [1] https://lkml.org/lkml/2019/11/8/949 Signed-off-by: Vitaly Wool --- mm/zswap.c | 85 ++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 54 insertions(+), 31 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 46a322316e52..7ec8bd912d13 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -32,6 +32,7 @@ #include #include #include +#include /********************************* * statistics @@ -65,6 +66,11 @@ static u64 zswap_reject_kmemcache_fail; /* Duplicate store was encountered (rare) */ static u64 zswap_duplicate_entry; +/* Shrinker work queue */ +static struct workqueue_struct *shrink_wq; +/* Pool limit was hit, we need to calm down */ +static bool zswap_pool_reached_full; + /********************************* * tunables **********************************/ @@ -109,6 +115,11 @@ module_param_cb(zpool, &zswap_zpool_param_ops, &zswap_zpool_type, 0644); static unsigned int zswap_max_pool_percent = 20; module_param_named(max_pool_percent, zswap_max_pool_percent, uint, 0644); +/* The threshold for accepting new pages after the max_pool_percent was hit */ +static unsigned int zswap_accept_thr_percent = 90; /* of max pool size */ +module_param_named(accept_threshold_percent, zswap_accept_thr_percent, + uint, 0644); + /* Enable/disable handling same-value filled pages (enabled by default) */ static bool zswap_same_filled_pages_enabled = true; module_param_named(same_filled_pages_enabled, zswap_same_filled_pages_enabled, @@ -123,7 +134,8 @@ struct zswap_pool { struct crypto_comp * __percpu *tfm; struct kref kref; struct list_head list; - struct work_struct work; + struct work_struct release_work; + struct work_struct shrink_work; struct hlist_node node; char tfm_name[CRYPTO_MAX_ALG_NAME]; }; @@ -214,6 +226,13 @@ static bool zswap_is_full(void) DIV_ROUND_UP(zswap_pool_total_size, PAGE_SIZE); } +static bool zswap_can_accept(void) +{ + return totalram_pages() * zswap_accept_thr_percent / 100 * + zswap_max_pool_percent / 100 > + DIV_ROUND_UP(zswap_pool_total_size, PAGE_SIZE); +} + static void zswap_update_total_size(void) { struct zswap_pool *pool; @@ -501,6 +520,16 @@ static struct zswap_pool *zswap_pool_find_get(char *type, char *compressor) return NULL; } +static void shrink_worker(struct work_struct *w) +{ + struct zswap_pool *pool = container_of(w, typeof(*pool), + shrink_work); + + if (zpool_shrink(pool->zpool, 1, NULL)) + zswap_reject_reclaim_fail++; + zswap_pool_put(pool); +} + static struct zswap_pool *zswap_pool_create(char *type, char *compressor) { struct zswap_pool *pool; @@ -551,6 +580,7 @@ static struct zswap_pool *zswap_pool_create(char *type, char *compressor) */ kref_init(&pool->kref); INIT_LIST_HEAD(&pool->list); + INIT_WORK(&pool->shrink_work, shrink_worker); zswap_pool_debug("created", pool); @@ -624,7 +654,8 @@ static int __must_check zswap_pool_get(struct zswap_pool *pool) static void __zswap_pool_release(struct work_struct *work) { - struct zswap_pool *pool = container_of(work, typeof(*pool), work); + struct zswap_pool *pool = container_of(work, typeof(*pool), + release_work); synchronize_rcu(); @@ -647,8 +678,8 @@ static void __zswap_pool_empty(struct kref *kref) list_del_rcu(&pool->list); - INIT_WORK(&pool->work, __zswap_pool_release); - schedule_work(&pool->work); + INIT_WORK(&pool->release_work, __zswap_pool_release); + schedule_work(&pool->release_work); spin_unlock(&zswap_pools_lock); } @@ -942,22 +973,6 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) return ret; } -static int zswap_shrink(void) -{ - struct zswap_pool *pool; - int ret; - - pool = zswap_pool_last_get(); - if (!pool) - return -ENOENT; - - ret = zpool_shrink(pool->zpool, 1, NULL); - - zswap_pool_put(pool); - - return ret; -} - static int zswap_is_page_same_filled(void *ptr, unsigned long *value) { unsigned int pos; @@ -1011,21 +1026,23 @@ static int zswap_frontswap_store(unsigned type, pgoff_t offset, /* reclaim space if needed */ if (zswap_is_full()) { + struct zswap_pool *pool; + zswap_pool_limit_hit++; - if (zswap_shrink()) { - zswap_reject_reclaim_fail++; - ret = -ENOMEM; - goto reject; - } + zswap_pool_reached_full = true; + pool = zswap_pool_last_get(); + if (pool) + queue_work(shrink_wq, &pool->shrink_work); + ret = -ENOMEM; + goto reject; + } - /* A second zswap_is_full() check after - * zswap_shrink() to make sure it's now - * under the max_pool_percent - */ - if (zswap_is_full()) { + if (zswap_pool_reached_full) { + if (!zswap_can_accept()) { ret = -ENOMEM; goto reject; - } + } else + zswap_pool_reached_full = false; } /* allocate entry */ @@ -1332,11 +1349,17 @@ static int __init init_zswap(void) zswap_enabled = false; } + shrink_wq = create_workqueue("zswap-shrink"); + if (!shrink_wq) + goto fallback_fail; + frontswap_register_ops(&zswap_frontswap_ops); if (zswap_debugfs_init()) pr_warn("debugfs initialization failed\n"); return 0; +fallback_fail: + zswap_pool_destroy(pool); hp_fail: cpuhp_remove_state(CPUHP_MM_ZSWP_MEM_PREPARE); dstmem_fail: