From patchwork Sat Nov 19 00:15:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13049322 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E432C4332F for ; Sat, 19 Nov 2022 00:15:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CA1CD6B0072; Fri, 18 Nov 2022 19:15:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B8E218E0001; Fri, 18 Nov 2022 19:15:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9E28E6B007E; Fri, 18 Nov 2022 19:15:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 807706B0072 for ; Fri, 18 Nov 2022 19:15:40 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 59BBA80377 for ; Sat, 19 Nov 2022 00:15:40 +0000 (UTC) X-FDA: 80148273240.15.4EE16B3 Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) by imf10.hostedemail.com (Postfix) with ESMTP id 03472C0004 for ; Sat, 19 Nov 2022 00:15:39 +0000 (UTC) Received: by mail-pj1-f42.google.com with SMTP id y14-20020a17090a2b4e00b002189a1b84d4so70916pjc.2 for ; Fri, 18 Nov 2022 16:15:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=NHkIkh8RaR25rj7+Wlw0rbe1vzjooV8Fsu/f07L3zCG5lUFzcdTGGBtND54rMrvFn2 1nztmuMnmKeDkLPZsShRcJCn8KfZDSS2xoMxkJ+8eWbfouubZv8PETaQAHxCEEQyuRcP 7dtpmnLSJFtricXu29dRABoMD0uTm0Xv+9F3R8Z0zOZfzFHEh/jZfd89vph1ur+jnvoi qe4O0YDtzONH4Dze1Yr/u8DHfwdnbozJEOewCLH0RyGcNrdx/Q/rLMIXB5yDuzL+iHp+ RKiSsuMYn6PCIoOSsyr2vmZRFTqbXCde8fgnZhLJQEkH9ydslMP7E4Ww+Oh0mh53K7e2 lK5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=1ffGgIPgBLNKyjaG8y1Jy/uArP79mZEUBSs5lhG26z7dSKK8RJg85NkT3uKUBUafdC gnlYqxd4oOHiR7WCDs1G8ulZIExb8hVcCj56rOQ1BEPmeQ4XpGApTtxiFkuBxTVtb1Gm rypzYNWvC98NKcQ0iOhOMe+mUlWjWPJlhjryIGX3Hdff4YHURPaIq9Uz9lPxCcSb5DYR lzh9KFw1/GE3RKyLJVPWoW58l7d9tKM9fOI10TzyIoHB+2VheEYeuR1gFc/codfZEHoy U3IQmGlQ3vzsKfyY21OhenKcdJdwzjwCMdEod2vd8LVO7lOi1RcPz6fIBKV8hDs/ebjq PH2w== X-Gm-Message-State: ANoB5pnFhmZcQ+Cw7ZHIo+ht+Uj7r2Lwj7sDH3uvMT71QpL3tJIxeVbP NI1RpB6hD7hdiJkyYF6NMEk= X-Google-Smtp-Source: AA0mqf7QZyc8yIJTE5WAZ0IZUUaf0nqx+pCwp3EewXFYXciM0TOgmftRCHrHwYQYKAVKePw7j1Pq+w== X-Received: by 2002:a17:902:9a03:b0:186:9f20:e7e2 with SMTP id v3-20020a1709029a0300b001869f20e7e2mr1578912plp.174.1668816939044; Fri, 18 Nov 2022 16:15:39 -0800 (PST) Received: from localhost (fwdproxy-prn-010.fbsv.net. [2a03:2880:ff:a::face:b00c]) by smtp.gmail.com with ESMTPSA id x22-20020a170902821600b0017f48a9e2d6sm4202332pln.292.2022.11.18.16.15.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 16:15:38 -0800 (PST) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, minchan@kernel.org, ngupta@vflare.org, senozhatsky@chromium.org, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com Subject: [PATCH v6 1/6] zswap: fix writeback lock ordering for zsmalloc Date: Fri, 18 Nov 2022 16:15:31 -0800 Message-Id: <20221119001536.2086599-2-nphamcs@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221119001536.2086599-1-nphamcs@gmail.com> References: <20221119001536.2086599-1-nphamcs@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1668816940; a=rsa-sha256; cv=none; b=fJdxSipM/WkDGICA7fjN7fM0HL/BAdn7uvBiomLVKpMuaLZAFx2v+hBUsBR34sJXRnE4dK yayux/6KeSGyrSH0VAnJ1eguf9B6OMl1vfCYQcBpG87TCEcq5TsDLUzrj6aVqG6a8eEBiz KqsJM9Ewc5e0EtCWhqRBPGDWP0e+gNQ= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=NHkIkh8R; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.216.42 as permitted sender) smtp.mailfrom=nphamcs@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1668816940; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=gmWBy44+ZZYiWFupBsIhXVA1+ipk5ZeF8iIpJnkbHqF73G2t2eVS0zPmyXlJY1sUznH1+n JVPgWoiaNlbWWbWdd+VtXGl2ioIeQ0JZAur0vhgnitGZWjyXzgcwdc3IapPDRVEKnraiyB jlClrnc8z64lj+4U7Up7+E0JOiiimTo= Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=NHkIkh8R; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.216.42 as permitted sender) smtp.mailfrom=nphamcs@gmail.com X-Rspam-User: X-Stat-Signature: 5ng7mzrtw1taxuzkzqis9cz8cugg6tey X-Rspamd-Queue-Id: 03472C0004 X-Rspamd-Server: rspam11 X-HE-Tag: 1668816939-62573 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner zswap's customary lock order is tree->lock before pool->lock, because the tree->lock protects the entries' refcount, and the free callbacks in the backends acquire their respective pool locks to dispatch the backing object. zsmalloc's map callback takes the pool lock, so zswap must not grab the tree->lock while a handle is mapped. This currently only happens during writeback, which isn't implemented for zsmalloc. In preparation for it, move the tree->lock section out of the mapped entry section Signed-off-by: Johannes Weiner Signed-off-by: Nhat Pham --- mm/zswap.c | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-) -- 2.30.2 diff --git a/mm/zswap.c b/mm/zswap.c index 2d48fd59cc7a..2d69c1d678fe 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -958,7 +958,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) }; if (!zpool_can_sleep_mapped(pool)) { - tmp = kmalloc(PAGE_SIZE, GFP_ATOMIC); + tmp = kmalloc(PAGE_SIZE, GFP_KERNEL); if (!tmp) return -ENOMEM; } @@ -968,6 +968,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) swpentry = zhdr->swpentry; /* here */ tree = zswap_trees[swp_type(swpentry)]; offset = swp_offset(swpentry); + zpool_unmap_handle(pool, handle); /* find and ref zswap entry */ spin_lock(&tree->lock); @@ -975,20 +976,12 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) if (!entry) { /* entry was invalidated */ spin_unlock(&tree->lock); - zpool_unmap_handle(pool, handle); kfree(tmp); return 0; } spin_unlock(&tree->lock); BUG_ON(offset != entry->offset); - src = (u8 *)zhdr + sizeof(struct zswap_header); - if (!zpool_can_sleep_mapped(pool)) { - memcpy(tmp, src, entry->length); - src = tmp; - zpool_unmap_handle(pool, handle); - } - /* try to allocate swap cache page */ switch (zswap_get_swap_cache_page(swpentry, &page)) { case ZSWAP_SWAPCACHE_FAIL: /* no memory or invalidate happened */ @@ -1006,6 +999,14 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); dlen = PAGE_SIZE; + zhdr = zpool_map_handle(pool, handle, ZPOOL_MM_RO); + src = (u8 *)zhdr + sizeof(struct zswap_header); + if (!zpool_can_sleep_mapped(pool)) { + memcpy(tmp, src, entry->length); + src = tmp; + zpool_unmap_handle(pool, handle); + } + mutex_lock(acomp_ctx->mutex); sg_init_one(&input, src, entry->length); sg_init_table(&output, 1); @@ -1015,6 +1016,11 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) dlen = acomp_ctx->req->dlen; mutex_unlock(acomp_ctx->mutex); + if (!zpool_can_sleep_mapped(pool)) + kfree(tmp); + else + zpool_unmap_handle(pool, handle); + BUG_ON(ret); BUG_ON(dlen != PAGE_SIZE); @@ -1045,7 +1051,11 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) zswap_entry_put(tree, entry); spin_unlock(&tree->lock); - goto end; + return ret; + +fail: + if (!zpool_can_sleep_mapped(pool)) + kfree(tmp); /* * if we get here due to ZSWAP_SWAPCACHE_EXIST @@ -1054,17 +1064,10 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) * if we free the entry in the following put * it is also okay to return !0 */ -fail: spin_lock(&tree->lock); zswap_entry_put(tree, entry); spin_unlock(&tree->lock); -end: - if (zpool_can_sleep_mapped(pool)) - zpool_unmap_handle(pool, handle); - else - kfree(tmp); - return ret; }