From patchwork Fri Nov 18 18:24:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13048566 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E4B8C43217 for ; Fri, 18 Nov 2022 18:24:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9B1546B0072; Fri, 18 Nov 2022 13:24:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 939B06B0074; Fri, 18 Nov 2022 13:24:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7B3038E0001; Fri, 18 Nov 2022 13:24:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 5D8B46B0072 for ; Fri, 18 Nov 2022 13:24:12 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 28E49C015C for ; Fri, 18 Nov 2022 18:24:12 +0000 (UTC) X-FDA: 80147387544.14.EC2D83C Received: from mail-pj1-f43.google.com (mail-pj1-f43.google.com [209.85.216.43]) by imf28.hostedemail.com (Postfix) with ESMTP id BAB45C0009 for ; Fri, 18 Nov 2022 18:24:11 +0000 (UTC) Received: by mail-pj1-f43.google.com with SMTP id u6-20020a17090a5e4600b0021881a8d264so3475774pji.4 for ; Fri, 18 Nov 2022 10:24:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=nkZTR8Kb+B0gk7H1xIfb6LfPuKa1/cZSjTASVzpInLhhWrDd52VMjYvyLSxibmHHod XS8PyxlBFQKCwIY7VnVGxHp2GBZlNfH/mDQ4f5E1mblACckXhlpqRDGQD3VKGXrVaLNy bo1/KFKE5Sqo1Aa3JN4cuKvISQluxEyKaDUihk3A+AHs589MG38F3CTjzVYiv3aBbekT x8m49ihtYKr1AJtwmsvWsD+DhRasBVPL6r5zFur0+0F9Yfp9UH+kmP1I9x8jXDu4nyYc GLJJzZhXs5fchmAsncztYOnoY5qanzae0S0voCitkq4P4TSqCe1NvZpSE82KV0vudWgM mn+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=USRIJxT2cijl43Q2t7sZoxYaDo125qtI3W5cz1vm4Tdd7QQth7KdRZvJO9pAVRvVgI rB81WFLGDr1OgrGoaRuV/F187QBHKThdiBq+TfatYfEWUJyj9z7fi6agHASTPKmVQMBr Ix7Hh186gPdOoZN0OjYmYrNBsul6VabJEE5jfy0+jRwhg4AkuBkUPnznsmYAWeqnJ10R Un6y1riAgjNF1aRBR5i1pkzmGWhscz7XY/xZ3dVUWVtR7Dba6vGB/9jbxwkJR30QNdhB BFi0WBt8gxx2GK2TImrcYA3rYzYhAVDQ1GXjORPP6oIo6CB48wdkoZqr9Mh1FS4zxeuf 8i3g== X-Gm-Message-State: ANoB5plnhzHxoviKxeUVlGkgD+Od2sek3MUIm1SwTJkRR2ffu402nAKZ HmVeWIx0Ezz7B3tiEquDCaw= X-Google-Smtp-Source: AA0mqf4on4UGDZHhIP6GW7D0Rfy6F/6F3IQcotM1yQTtdjyOb/lap7oA9Q9zfmpCA8bD6qY6+PtuEA== X-Received: by 2002:a17:902:cacd:b0:17f:7d9a:4952 with SMTP id y13-20020a170902cacd00b0017f7d9a4952mr612738pld.117.1668795850563; Fri, 18 Nov 2022 10:24:10 -0800 (PST) Received: from localhost (fwdproxy-prn-023.fbsv.net. [2a03:2880:ff:17::face:b00c]) by smtp.gmail.com with ESMTPSA id c8-20020a170902b68800b00188b5d25438sm4058927pls.35.2022.11.18.10.24.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Nov 2022 10:24:10 -0800 (PST) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, minchan@kernel.org, ngupta@vflare.org, senozhatsky@chromium.org, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com Subject: [PATCH v5 1/6] zswap: fix writeback lock ordering for zsmalloc Date: Fri, 18 Nov 2022 10:24:02 -0800 Message-Id: <20221118182407.82548-2-nphamcs@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221118182407.82548-1-nphamcs@gmail.com> References: <20221118182407.82548-1-nphamcs@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1668795851; a=rsa-sha256; cv=none; b=SzyviNU6mBtP94Nt+/PM+7xxu3FdvRNjhSxdF3XqDDjPjMlx+Suy9LUZc53esrk5UVezln 1iNocKiAKTZzy0OQJJtc/LpS0F9Cx7v8rhMLFTmL/bmeYQTDZJ+i88BAPIVzlGb5MiyfyC bwU0pj7KDcoJcCNyvIU5I4iwWnRTSU0= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=nkZTR8Kb; spf=pass (imf28.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.216.43 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1668795851; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=XxYaBJ+YCAkiJZfvLznQZvtx/JjWzFuJ26jquETaD3hKArRULa/+oRkDLb2bFiLjcd+vGp ug7TNsy7CWZ6L9G4OKBwJPrM+x3pDnRkkqqrcKLmyo9ff46Hyn03KjnLKCTqDDSDU7gSNe nBGTAu0rGuVpxZmzVluFasDmn8TCRzY= X-Stat-Signature: k58jzr1z7ynidj96d1k51dezzdigxoqr X-Rspamd-Queue-Id: BAB45C0009 Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=nkZTR8Kb; spf=pass (imf28.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.216.43 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam04 X-Rspam-User: X-HE-Tag: 1668795851-267206 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner zswap's customary lock order is tree->lock before pool->lock, because the tree->lock protects the entries' refcount, and the free callbacks in the backends acquire their respective pool locks to dispatch the backing object. zsmalloc's map callback takes the pool lock, so zswap must not grab the tree->lock while a handle is mapped. This currently only happens during writeback, which isn't implemented for zsmalloc. In preparation for it, move the tree->lock section out of the mapped entry section Signed-off-by: Johannes Weiner Signed-off-by: Nhat Pham --- mm/zswap.c | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-) -- 2.30.2 diff --git a/mm/zswap.c b/mm/zswap.c index 2d48fd59cc7a..2d69c1d678fe 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -958,7 +958,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) }; if (!zpool_can_sleep_mapped(pool)) { - tmp = kmalloc(PAGE_SIZE, GFP_ATOMIC); + tmp = kmalloc(PAGE_SIZE, GFP_KERNEL); if (!tmp) return -ENOMEM; } @@ -968,6 +968,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) swpentry = zhdr->swpentry; /* here */ tree = zswap_trees[swp_type(swpentry)]; offset = swp_offset(swpentry); + zpool_unmap_handle(pool, handle); /* find and ref zswap entry */ spin_lock(&tree->lock); @@ -975,20 +976,12 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) if (!entry) { /* entry was invalidated */ spin_unlock(&tree->lock); - zpool_unmap_handle(pool, handle); kfree(tmp); return 0; } spin_unlock(&tree->lock); BUG_ON(offset != entry->offset); - src = (u8 *)zhdr + sizeof(struct zswap_header); - if (!zpool_can_sleep_mapped(pool)) { - memcpy(tmp, src, entry->length); - src = tmp; - zpool_unmap_handle(pool, handle); - } - /* try to allocate swap cache page */ switch (zswap_get_swap_cache_page(swpentry, &page)) { case ZSWAP_SWAPCACHE_FAIL: /* no memory or invalidate happened */ @@ -1006,6 +999,14 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); dlen = PAGE_SIZE; + zhdr = zpool_map_handle(pool, handle, ZPOOL_MM_RO); + src = (u8 *)zhdr + sizeof(struct zswap_header); + if (!zpool_can_sleep_mapped(pool)) { + memcpy(tmp, src, entry->length); + src = tmp; + zpool_unmap_handle(pool, handle); + } + mutex_lock(acomp_ctx->mutex); sg_init_one(&input, src, entry->length); sg_init_table(&output, 1); @@ -1015,6 +1016,11 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) dlen = acomp_ctx->req->dlen; mutex_unlock(acomp_ctx->mutex); + if (!zpool_can_sleep_mapped(pool)) + kfree(tmp); + else + zpool_unmap_handle(pool, handle); + BUG_ON(ret); BUG_ON(dlen != PAGE_SIZE); @@ -1045,7 +1051,11 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) zswap_entry_put(tree, entry); spin_unlock(&tree->lock); - goto end; + return ret; + +fail: + if (!zpool_can_sleep_mapped(pool)) + kfree(tmp); /* * if we get here due to ZSWAP_SWAPCACHE_EXIST @@ -1054,17 +1064,10 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) * if we free the entry in the following put * it is also okay to return !0 */ -fail: spin_lock(&tree->lock); zswap_entry_put(tree, entry); spin_unlock(&tree->lock); -end: - if (zpool_can_sleep_mapped(pool)) - zpool_unmap_handle(pool, handle); - else - kfree(tmp); - return ret; }