From patchwork Tue Nov 8 19:32:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13036773 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37082C4167B for ; Tue, 8 Nov 2022 19:32:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 921706B0072; Tue, 8 Nov 2022 14:32:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8AE706B0075; Tue, 8 Nov 2022 14:32:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 65E1D8E0003; Tue, 8 Nov 2022 14:32:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 52DB16B0072 for ; Tue, 8 Nov 2022 14:32:11 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 0B385160574 for ; Tue, 8 Nov 2022 19:32:11 +0000 (UTC) X-FDA: 80111270862.27.CA00442 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) by imf13.hostedemail.com (Postfix) with ESMTP id 7D9D720006 for ; Tue, 8 Nov 2022 19:32:10 +0000 (UTC) Received: by mail-pf1-f182.google.com with SMTP id i3so14663411pfc.11 for ; Tue, 08 Nov 2022 11:32:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=HSThM6hFhhVtQ9XEuNUbTWYT7N7QyCqsOWJmCZ+Yc1HgTC5dVqVT5bIrBAjLJjYN4H DtZ4Od1NEvWEM7Q8mFdZTbfTJE3GcYDBXEa/wKhZksZQdEXvopPYjROUxytN+JRYnCAm PRXAIiGoz9m1Ca7Sa/Y36uQ24ak+Yw9WNj0uVQtUJNufX6MAtOUZYBw9F8TlnQraD6t7 okXhYKtevjEUpjKI3L3YStKt6jPXNUGnPKtTSWoAEANPeEcaIj34eCx8kbh+b8JlXl30 /fwr4LcBJksre4uH1ZMCiTg/wkn14O5eKPDZeoJmyIHSZ6IxnevkEL92goAY6m45eBPZ Y4KQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=JkxSr+ZxP6l3bE/9GKX3fzWdnK8AiJBEkGJ5vrvIZZ1bYkCUlo8AqMs6ASdkUy3ekz ukeGxsqJda/UGblbJxQFnOrD93i7TiHLM5104EW9a4aytGrHb+6waZxolKkm3wXQ/RsY 02e+RE9J+u6h11eYkB+yt2Ik7YevrzhfRQAVEZ3/VrpJ59L2RLJho900G7GcThzY+V5f zBNXRtE7xsHcQ2KwT91IE5OhTwsEsnLrRZDrshRjdtTf95a0Wifm8vMsYXQXAdksSzbS uc0pJRagS3hDVyQFNKFsHVmXjE6dzl7LEyzONRWdZtw8HZ4NyffI9H0WeFdcK7QN7U1Q Ni8A== X-Gm-Message-State: ACrzQf0OLtwNPjdhueowN1o2M6CaLmr6+2PPYC8w9Hd7M7eqaeebpr2H HeqWzxUOF0ddHqJ1yvwuYxg= X-Google-Smtp-Source: AMsMyM4hQi95Y9aEwO3gbncH+WrDysFLwztEEQpUWcNqQm7Mtby+jZc9482bPKMIM+RYTC65gLBLzQ== X-Received: by 2002:a63:5a05:0:b0:434:23a5:a5ca with SMTP id o5-20020a635a05000000b0043423a5a5camr49494580pgb.515.1667935929482; Tue, 08 Nov 2022 11:32:09 -0800 (PST) Received: from localhost (fwdproxy-prn-007.fbsv.net. [2a03:2880:ff:7::face:b00c]) by smtp.gmail.com with ESMTPSA id a13-20020a63e84d000000b0046ae5cfc3d5sm6054230pgk.61.2022.11.08.11.32.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Nov 2022 11:32:09 -0800 (PST) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, minchan@kernel.org, ngupta@vflare.org, senozhatsky@chromium.org, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com Subject: [PATCH v3 1/5] zswap: fix writeback lock ordering for zsmalloc Date: Tue, 8 Nov 2022 11:32:03 -0800 Message-Id: <20221108193207.3297327-2-nphamcs@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221108193207.3297327-1-nphamcs@gmail.com> References: <20221108193207.3297327-1-nphamcs@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1667935930; a=rsa-sha256; cv=none; b=i9VEMXjWcGwsHYq96qVZakrhuSFRz54DrWC2q0KYTD4BmpuhSOJpwq/cRFNYBagovBMcgG dsSwJWrgqbQPntsMKJs8Bt+rnfe1f+v2kWVMA5jd/uK7r27TygJQiCSQF92rN1Wl1kflg4 YfaUWbt+yM2lDa4zIWBK62C9Vgqjshs= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=HSThM6hF; spf=pass (imf13.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.210.182 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1667935930; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=7iJy6Ffk1arj6o33lTheEN9jIpK8k6ZrXv4RciWM/47qL5oZGvxyi1oj2VxQ9gYt33GO20 PbvFaYJK3V/Hyp0IwSv/3lvAQLjE3vH0VoIHibuX0wxZ5JTNGBXAdvj0FRksLHjuvh/46Q 8hz7cYoqIH7qPFYMM/ATfzHvj8q4eZg= X-Stat-Signature: 7u1wz44yaqxar4mwo1xqyitutqcqejfu X-Rspam-User: Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=HSThM6hF; spf=pass (imf13.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.210.182 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Queue-Id: 7D9D720006 X-Rspamd-Server: rspam09 X-HE-Tag: 1667935930-740021 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner zswap's customary lock order is tree->lock before pool->lock, because the tree->lock protects the entries' refcount, and the free callbacks in the backends acquire their respective pool locks to dispatch the backing object. zsmalloc's map callback takes the pool lock, so zswap must not grab the tree->lock while a handle is mapped. This currently only happens during writeback, which isn't implemented for zsmalloc. In preparation for it, move the tree->lock section out of the mapped entry section Signed-off-by: Johannes Weiner Signed-off-by: Nhat Pham --- mm/zswap.c | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-) -- 2.30.2 diff --git a/mm/zswap.c b/mm/zswap.c index 2d48fd59cc7a..2d69c1d678fe 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -958,7 +958,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) }; if (!zpool_can_sleep_mapped(pool)) { - tmp = kmalloc(PAGE_SIZE, GFP_ATOMIC); + tmp = kmalloc(PAGE_SIZE, GFP_KERNEL); if (!tmp) return -ENOMEM; } @@ -968,6 +968,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) swpentry = zhdr->swpentry; /* here */ tree = zswap_trees[swp_type(swpentry)]; offset = swp_offset(swpentry); + zpool_unmap_handle(pool, handle); /* find and ref zswap entry */ spin_lock(&tree->lock); @@ -975,20 +976,12 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) if (!entry) { /* entry was invalidated */ spin_unlock(&tree->lock); - zpool_unmap_handle(pool, handle); kfree(tmp); return 0; } spin_unlock(&tree->lock); BUG_ON(offset != entry->offset); - src = (u8 *)zhdr + sizeof(struct zswap_header); - if (!zpool_can_sleep_mapped(pool)) { - memcpy(tmp, src, entry->length); - src = tmp; - zpool_unmap_handle(pool, handle); - } - /* try to allocate swap cache page */ switch (zswap_get_swap_cache_page(swpentry, &page)) { case ZSWAP_SWAPCACHE_FAIL: /* no memory or invalidate happened */ @@ -1006,6 +999,14 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); dlen = PAGE_SIZE; + zhdr = zpool_map_handle(pool, handle, ZPOOL_MM_RO); + src = (u8 *)zhdr + sizeof(struct zswap_header); + if (!zpool_can_sleep_mapped(pool)) { + memcpy(tmp, src, entry->length); + src = tmp; + zpool_unmap_handle(pool, handle); + } + mutex_lock(acomp_ctx->mutex); sg_init_one(&input, src, entry->length); sg_init_table(&output, 1); @@ -1015,6 +1016,11 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) dlen = acomp_ctx->req->dlen; mutex_unlock(acomp_ctx->mutex); + if (!zpool_can_sleep_mapped(pool)) + kfree(tmp); + else + zpool_unmap_handle(pool, handle); + BUG_ON(ret); BUG_ON(dlen != PAGE_SIZE); @@ -1045,7 +1051,11 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) zswap_entry_put(tree, entry); spin_unlock(&tree->lock); - goto end; + return ret; + +fail: + if (!zpool_can_sleep_mapped(pool)) + kfree(tmp); /* * if we get here due to ZSWAP_SWAPCACHE_EXIST @@ -1054,17 +1064,10 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) * if we free the entry in the following put * it is also okay to return !0 */ -fail: spin_lock(&tree->lock); zswap_entry_put(tree, entry); spin_unlock(&tree->lock); -end: - if (zpool_can_sleep_mapped(pool)) - zpool_unmap_handle(pool, handle); - else - kfree(tmp); - return ret; }