From patchwork Tue Jan 30 01:36:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536684 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F61DC47DA9 for ; Tue, 30 Jan 2024 01:43:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7FE626B00BD; Mon, 29 Jan 2024 20:42:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7AF916B00BE; Mon, 29 Jan 2024 20:42:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5B0056B00BF; Mon, 29 Jan 2024 20:42:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 46B976B00BD for ; Mon, 29 Jan 2024 20:42:38 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 2E54DA0164 for ; Tue, 30 Jan 2024 01:42:38 +0000 (UTC) X-FDA: 81734277996.18.5D93E5A Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) by imf10.hostedemail.com (Postfix) with ESMTP id 8F29AC0016 for ; Tue, 30 Jan 2024 01:42:36 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=xuALguqK; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf10.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.176 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578956; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pWfrSpQCJ73gDMIVUCgMZppF172UOPzzTZlrxOiNavU=; b=N+dRJyC897HMgyfphYBIR1DbHibo9vNVGOAc2bXIM9gPZnxv/aK2zm5llkKM+JoMGg6Dgf 6W7563gxpKAE+wvZg3+YJVkevpcxKArYk1rVxWwsSbzBI8HLxT59SmuAqWapnZ3XRiC2TP oCjlmLfXWIlT/4soQx2MK1+eOwSrKek= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=xuALguqK; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf10.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.176 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578956; a=rsa-sha256; cv=none; b=Xqfc+pchxsvcLonH7Sfd7uJ5V99/8c7ti0cfxW1xYLTksRoMDMyTXbDGKskRrNXoDb/eF2 wuzmH/6juX6prjJRUARK/uHWMjZ/M+5hp61BwzTsKePXHZmRhzSauhQbQuxklpS1rBcrI7 IEz4YALbEWT2LQ3Kwl0c15NlQR9OXjs= Received: by mail-qk1-f176.google.com with SMTP id af79cd13be357-7838af983c1so281285685a.3 for ; Mon, 29 Jan 2024 17:42:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578956; x=1707183756; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pWfrSpQCJ73gDMIVUCgMZppF172UOPzzTZlrxOiNavU=; b=xuALguqKNfX8MS40M/WOs4T83ccp+zuOwqekWoa7oln13bDtDRDOG7sJdt3MDkUzTX QRQaVikAjqgJVa76RP+2EHpLcCHtZAJd+5Xj/U1WJUFesKkkGnCpbWLg0RHHcY2cy/DM PTT1PpZ7xCccldWECINkFH4EX2X5ly+LSk5uD7/KY+RpOAtN5SBBVBzGaoNK8IIt/CGM wRvhUkwn6gCikokVjd9mtVL63jZiu5BnJtpQIcKR086DzLSWNM90rUcF5AhAI9fk5vV7 2uYx9VgXX2B0BjGix7GX/7Lrxs6T9G1ibXox621zB0xXFfiMrqSWS7sEOojUc8xYqmyI aPyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578956; x=1707183756; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pWfrSpQCJ73gDMIVUCgMZppF172UOPzzTZlrxOiNavU=; b=LfMWnbDuLQIFQSjXb7vL8PeZvewZARoxu5YGj9ytkwIgbyvVHsU6IKbzqbUSNpg3WV ITtcqOWq/cGqzmFgWTlzQ/CoVUiUe72UuVMOYsHFnXvywuZIo8swd8tQpbjETY4pXnWv XhbyS1vGD//JyB1lfcuDYy6pxBiqHkHwfxmMEM+tgEhnix4PqA7KC7eJ2YiI+yqhwjjL mkG9YgtQORjssufXK+6F9zmFaWg69veeelwOjKbbbz+4z5CKXHwmPyzhrdHDAehiwg8k o/aUn3OgwQlY9rXEdn//pQx+OFYoUlg17p4Bon/ixkkmCOkBrRzif6+tATDCbOeJgxHp HJGw== X-Gm-Message-State: AOJu0Yzf1LkLE3KhmF+Jz92frhfekWpznRALmD57WovKv8xvJxWZuKpW LZ1WXo0uPY8UDKQ4EJVjyWKtrNTa5ZmpYl15EivVNoOcPgLpAKAbyyv022/8HpE= X-Google-Smtp-Source: AGHT+IEXs8SJCuuIZtfIF+y9ddQXhUqdNoy7qZLBVXx9Tn8aMjnAw4Zw+Z4ifp80mQOXs7/58wge/w== X-Received: by 2002:a05:6214:400c:b0:68c:5ae6:de8d with SMTP id kd12-20020a056214400c00b0068c5ae6de8dmr1717725qvb.39.1706578955770; Mon, 29 Jan 2024 17:42:35 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id df12-20020a056214080c00b0068c4542eb0fsm2419516qvb.137.2024.01.29.17.42.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:35 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 18/20] mm: zswap: function ordering: per-cpu compression infra Date: Mon, 29 Jan 2024 20:36:54 -0500 Message-ID: <20240130014208.565554-19-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 8F29AC0016 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: c1bt6sp48te3uarea5cubgboq7xhs7qq X-HE-Tag: 1706578956-660152 X-HE-Meta: U2FsdGVkX1874qxW8utB2hOVwMO35tqkArAdivZ3YRLKhtnExAxnrt62TMOZKD2pe7m2/qbgxKvHssXbR7muc5PdfNKP3mXW8IjfGgFWU1eMDrdqjx390doYTX7f1zzCh92hKfRLhmccSAJFTj+7uDe69Zvs5RWDIodUvvdGnSssDaJYYSwgFJalr2rJoKpbjmr8rNHQo0VBP50/jftdxIKF29CvqyoITEvu1LCGbEwSIJ6vr/blMVQalus8QlMNbg2VsT/TXAaP4ku8t5PPfoelRvi24EL6VVu8OujN5KFYALWtrleQEXa0bSuGb9bxeYPGIsx/HvCEYmzTas0+qmj1RRPYWQjWLVGVLfI6zRpLcFuHyuCT53GAttU4PFNMwQ/o24RAhQos/MDW/yhnScSeX3yI4WnwTMbSC0acVQb1Qk6I7ReFGsd1CvOTqbNRwwqzsmG364SGRY5VUmSnkKdDstoFkNB57cTqgsx5OarX860iCxUMsvc1uESUI/QsZsR2JHyI82QBHLLzhWgs7fw85f6MzasdX0jivzhkYhw5aDMfr821kqpn4LHzaIT6tUKVwgBKgCMmEOqA1Osy78PF+9Aa7dWPNrawrpZOmIcDN6vMZfFnQpudQGF7bEKW0WYo6ljSRl3pwM26nNQZ0217EWERBg9gX4E38CHlqPcd7EpsrlSDWw+V6nuFNnosJZ9xod4flfwj9TNAJJgk9QCxqk3j0TyTbsQIC6fo4Lqfx3rkES2uLr/gWYQxlv8DhRAFs0U4I918A7UdE2fzemq43fxq7aXcsY5tjDIRKZuMEJ9ioITQZFwXyhc9mC9Q0irQCBklDsm3+LTtmBEsTALgG65cwL9hGmKHIz9ppfBhvop/mnv6s/Y+1AoRCjjeMfJV+PG8mHeMfgysV69KOq4DwgzmdmSm3shdVKw+QudnU9OC5SEeQSb8DxE4a35FyDS6Yh7h2nk9hcWxbSk aN/UoU9O uDPj3bIbR5ZbgLgyBgdGKxG5kiNQd9jgtFtpsFmQthfldQ/iN07tlMEQ1ewL6+WoqgRktT5rUFH2RC8M/AN0JZJoT6vtQ7Gcfjii8wvzJAvAA/AZ59tiZJmhHKXajaPm1xvWrlEeiV4LPsWmmb7cpmwWlqTzK2q5+WhUwIOZw6Tip5oDkq6sHkzUX3g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The per-cpu compression init/exit callbacks are awkwardly in the middle of the shrinker code. Move them up to the compression section. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 135 ++++++++++++++++++++++++++--------------------------- 1 file changed, 66 insertions(+), 69 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 17356b2e35c2..acd7dcd1e0f2 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -995,6 +995,72 @@ static void zswap_invalidate_entry(struct zswap_tree *tree, /********************************* * compressed storage functions **********************************/ +static int zswap_cpu_comp_prepare(unsigned int cpu, struct hlist_node *node) +{ + struct zswap_pool *pool = hlist_entry(node, struct zswap_pool, node); + struct crypto_acomp_ctx *acomp_ctx = per_cpu_ptr(pool->acomp_ctx, cpu); + struct crypto_acomp *acomp; + struct acomp_req *req; + int ret; + + mutex_init(&acomp_ctx->mutex); + + acomp_ctx->buffer = kmalloc_node(PAGE_SIZE * 2, GFP_KERNEL, cpu_to_node(cpu)); + if (!acomp_ctx->buffer) + return -ENOMEM; + + acomp = crypto_alloc_acomp_node(pool->tfm_name, 0, 0, cpu_to_node(cpu)); + if (IS_ERR(acomp)) { + pr_err("could not alloc crypto acomp %s : %ld\n", + pool->tfm_name, PTR_ERR(acomp)); + ret = PTR_ERR(acomp); + goto acomp_fail; + } + acomp_ctx->acomp = acomp; + + req = acomp_request_alloc(acomp_ctx->acomp); + if (!req) { + pr_err("could not alloc crypto acomp_request %s\n", + pool->tfm_name); + ret = -ENOMEM; + goto req_fail; + } + acomp_ctx->req = req; + + crypto_init_wait(&acomp_ctx->wait); + /* + * if the backend of acomp is async zip, crypto_req_done() will wakeup + * crypto_wait_req(); if the backend of acomp is scomp, the callback + * won't be called, crypto_wait_req() will return without blocking. + */ + acomp_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG, + crypto_req_done, &acomp_ctx->wait); + + return 0; + +req_fail: + crypto_free_acomp(acomp_ctx->acomp); +acomp_fail: + kfree(acomp_ctx->buffer); + return ret; +} + +static int zswap_cpu_comp_dead(unsigned int cpu, struct hlist_node *node) +{ + struct zswap_pool *pool = hlist_entry(node, struct zswap_pool, node); + struct crypto_acomp_ctx *acomp_ctx = per_cpu_ptr(pool->acomp_ctx, cpu); + + if (!IS_ERR_OR_NULL(acomp_ctx)) { + if (!IS_ERR_OR_NULL(acomp_ctx->req)) + acomp_request_free(acomp_ctx->req); + if (!IS_ERR_OR_NULL(acomp_ctx->acomp)) + crypto_free_acomp(acomp_ctx->acomp); + kfree(acomp_ctx->buffer); + } + + return 0; +} + static bool zswap_compress(struct folio *folio, struct zswap_entry *entry) { struct crypto_acomp_ctx *acomp_ctx; @@ -1201,75 +1267,6 @@ static void zswap_alloc_shrinker(struct zswap_pool *pool) pool->shrinker->seeks = DEFAULT_SEEKS; } -/********************************* -* per-cpu code -**********************************/ -static int zswap_cpu_comp_prepare(unsigned int cpu, struct hlist_node *node) -{ - struct zswap_pool *pool = hlist_entry(node, struct zswap_pool, node); - struct crypto_acomp_ctx *acomp_ctx = per_cpu_ptr(pool->acomp_ctx, cpu); - struct crypto_acomp *acomp; - struct acomp_req *req; - int ret; - - mutex_init(&acomp_ctx->mutex); - - acomp_ctx->buffer = kmalloc_node(PAGE_SIZE * 2, GFP_KERNEL, cpu_to_node(cpu)); - if (!acomp_ctx->buffer) - return -ENOMEM; - - acomp = crypto_alloc_acomp_node(pool->tfm_name, 0, 0, cpu_to_node(cpu)); - if (IS_ERR(acomp)) { - pr_err("could not alloc crypto acomp %s : %ld\n", - pool->tfm_name, PTR_ERR(acomp)); - ret = PTR_ERR(acomp); - goto acomp_fail; - } - acomp_ctx->acomp = acomp; - - req = acomp_request_alloc(acomp_ctx->acomp); - if (!req) { - pr_err("could not alloc crypto acomp_request %s\n", - pool->tfm_name); - ret = -ENOMEM; - goto req_fail; - } - acomp_ctx->req = req; - - crypto_init_wait(&acomp_ctx->wait); - /* - * if the backend of acomp is async zip, crypto_req_done() will wakeup - * crypto_wait_req(); if the backend of acomp is scomp, the callback - * won't be called, crypto_wait_req() will return without blocking. - */ - acomp_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG, - crypto_req_done, &acomp_ctx->wait); - - return 0; - -req_fail: - crypto_free_acomp(acomp_ctx->acomp); -acomp_fail: - kfree(acomp_ctx->buffer); - return ret; -} - -static int zswap_cpu_comp_dead(unsigned int cpu, struct hlist_node *node) -{ - struct zswap_pool *pool = hlist_entry(node, struct zswap_pool, node); - struct crypto_acomp_ctx *acomp_ctx = per_cpu_ptr(pool->acomp_ctx, cpu); - - if (!IS_ERR_OR_NULL(acomp_ctx)) { - if (!IS_ERR_OR_NULL(acomp_ctx->req)) - acomp_request_free(acomp_ctx->req); - if (!IS_ERR_OR_NULL(acomp_ctx->acomp)) - crypto_free_acomp(acomp_ctx->acomp); - kfree(acomp_ctx->buffer); - } - - return 0; -} - static enum lru_status shrink_memcg_cb(struct list_head *item, struct list_lru_one *l, spinlock_t *lock, void *arg) {