From patchwork Wed Oct 2 17:33:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Sridhar, Kanchana P" X-Patchwork-Id: 13820150 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B7B4CEACF3 for ; Wed, 2 Oct 2024 17:33:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 613856B0420; Wed, 2 Oct 2024 13:33:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 59C116B0421; Wed, 2 Oct 2024 13:33:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 464836B0422; Wed, 2 Oct 2024 13:33:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 1979A6B0420 for ; Wed, 2 Oct 2024 13:33:34 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 7E4D7140C18 for ; Wed, 2 Oct 2024 17:33:33 +0000 (UTC) X-FDA: 82629359106.21.3867F0B Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) by imf08.hostedemail.com (Postfix) with ESMTP id 5452716001C for ; Wed, 2 Oct 2024 17:33:31 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=goLrPKz8; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf08.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.16 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727890371; a=rsa-sha256; cv=none; b=5z6QCfzSvS18ux0HtFrsg5RrTDtyCR7ay+dcPwdzJ3uKYgCS8IplZz0Q+jLCRy7K0DbMYE bKcV4YIBKw+RDdKpPlS6GQW4OTy9lKvhmywAqXtlaUIzgQ/7QxWQTvfG0FOiSUramdnUFs 2B+uNoZaGiLVO2rsBtI5ohsbugxogic= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=goLrPKz8; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf08.hostedemail.com: domain of kanchana.p.sridhar@intel.com designates 198.175.65.16 as permitted sender) smtp.mailfrom=kanchana.p.sridhar@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727890371; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=RQPyxMWkljGkRsjpPrIGdpIPpxAfpWQdlGgEYRATctk=; b=MpkYStzfvbQxeN9LP2AY+Pq0actwPFiD/vJ1leWKxvKsJYS4Oa8JnwzOAgx5Q838u3dz1l OaDvNi+pvJJUVcs10wledls58nw9686WBFnZqbbN3u1aW1FbKcpbit4iWxkOFYQua59PnJ 6wV0cy0vD6XJa1OTQG9biuYk76brxEo= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1727890411; x=1759426411; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=1csFwrhxEdRzmEJzvTdg5rK5UGxUh7oEkFpr7q5uGNM=; b=goLrPKz8g2aUfLvvJEFxQhUFkrFZfCkxNz5ajWFAjcu3n4BsDoij6PEN vD9Pwsl9myxUkzlFW4RVG/6rNn0ZK6klXiFOiLSQfGgZviSuCYckKMdQL 5kjlMSt+p4nmHTLzCQEht4G/dO0Sh2wkmrr0svXCkVtP45ERqeA6g1Br5 g2xJQVYo+hROhaju6wdZjS0QBML1DQjfNyYG7OG6Vfji0c3eGhLfaP15a KOnm2JXXr+/Sfp+mPsr/N/ncSOcj9OhBUhc0GfcDHnirXPk00LhmGbnkD CxvJyImsE7tOUH1/W7Qyfrd2eXaPA64GVp+3QKhcVsyccr30uOIZmEm1H g==; X-CSE-ConnectionGUID: C34Xjr2oRVWGy1n9XQve7w== X-CSE-MsgGUID: NsoulFISTH6NTAaUy9efsg== X-IronPort-AV: E=McAfee;i="6700,10204,11213"; a="27158933" X-IronPort-AV: E=Sophos;i="6.11,172,1725346800"; d="scan'208";a="27158933" Received: from orviesa010.jf.intel.com ([10.64.159.150]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Oct 2024 10:33:29 -0700 X-CSE-ConnectionGUID: a7d7LA8UQjWECZOh3nGyoQ== X-CSE-MsgGUID: WP87RBtpRi6GjkE+WlYMyw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,172,1725346800"; d="scan'208";a="73934168" Received: from jf5300-b11a338t.jf.intel.com ([10.242.51.6]) by orviesa010.jf.intel.com with ESMTP; 02 Oct 2024 10:33:30 -0700 From: Kanchana P Sridhar To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, hannes@cmpxchg.org, yosryahmed@google.com, nphamcs@gmail.com, chengming.zhou@linux.dev, ryan.roberts@arm.com, ying.huang@intel.com, 21cnbao@gmail.com, akpm@linux-foundation.org Cc: wajdi.k.feghali@intel.com, vinodh.gopal@intel.com, kanchana.p.sridhar@intel.com Subject: [PATCH v1] mm: zswap: zswap_store_page() will initialize entry after adding to xarray. Date: Wed, 2 Oct 2024 10:33:29 -0700 Message-Id: <20241002173329.213722-1-kanchana.p.sridhar@intel.com> X-Mailer: git-send-email 2.27.0 MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 5452716001C X-Rspamd-Server: rspam01 X-Stat-Signature: 7iutcjgzi1kjuha6bayhxypi1i3dyn4g X-HE-Tag: 1727890411-375579 X-HE-Meta: U2FsdGVkX19P+NrqPi1+CkMOIe3NTRlmTKn+4vLLNsWUyvLcP+kd07hHbwgTUIrMNJJQUkqoby84yi4HV4eHNHswIITlurQ0D6i/kMTfEvgsIDst0Tj8yOQGrmVSmEXHgVhzMwSi0u6w7w6zbcQXH9lefAPMgxGkZnLEjaYI9zq6aKUtfyy2gDm6OGss0lqCkTPM/48YSm5atFcPjmi2FJTPp+9z4m43KZvqcdRtfgOr17T4jZJnKLl6XLRDl1HypdtALqsWGLjo3f5JFMf1aMB8Wt1JGaqe5fUyy0DIjFREXNKMYmuHr6VC+LHY4Q9+C+ASJDLtFJPrNXstxZgiAcXxJuqVBI1aKlOYfW53FMPP2LmZM3QSUaaPXkLZ7tzdpjU8HYtikpRxOs1UT4epgPmASdYFMBAWGOm5LkQw14zeLp4JrLeUUrf7ELYDRbYNKgVS8FVo7F7VigUtalOCwHBJ9ZdHM97/+q+78+QCbXVO7IvxyJ2qsqzFul2R0Zbbmsj3FD22emSynFhTZESg7vYHRq21UUZKTZmsCXejBZCq2Z4Ol1h0D2czQQSeMLcIhBSRvpzaJ1MSOFIVewdRMFbwMo2eTByb4kkzga9qAyQWUngnrkU+fvY4ETE3+AXHeEatAtMIYVbF36fp0M2WJeKRna6KeIuzj+N8gE1iegSO89nus9aMVVx39BgyHuWidjDT0UUC4XQin+zjw8Oep9tB4Nw3fL+/bDQjrs3fWDF8wSQO9pPiDpSJaRUHrq00A4D9YTx+Wcakq6e55GusaE7/eCV3ClE0P0diaBmL4gft3fc+HNYzc3UdmBgf5qcse5Y0x2sKQj3FqpLfx4zaxyTRyj1s9yRrnfIXAFeJZcRpPiY9XXhvmJRAyRziONNCZGczFllaYwSLkIYmO0ShNLnQuMQf1A9XLAuaa1dBOpbITQnG/cNMfEaUGmgrlr0T/Luy2kqmnTzZ8vPIGsz dk1Ti8l+ RgTqPzi4UR18b+wi3gZoea6rG+4KtjCRaDkFd3sgqJ8uTv8J7ECmAxYCKI+V4f0VyJDo004Z8eACyngAmWuR4z7NWYLnlucwF8zv7VxQybiK65dbJdwdo8mcj4kHf684P/l+jHhM5kQsV55bQIgDwQR4MY345EtVbXLR1ldgTTTWQw2lJIcxqXjbufL/nEMWbKl2vCkasg55XS7Cq48JksTLbKRIrDdeQk/aUV9k0Q9BZlGothSlDbIdk9KHoLX6ZCSVXBmiF9Q/NNk4pUcj+wd6deQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This incorporates Yosry's suggestions in [1] for further simplifying zswap_store_page(). If the page is successfully compressed and added to the xarray, we get the pool/objcg refs, and initialize all the entry's members. Only after this, we add it to the zswap LRU. In the time between the entry's addition to the xarray and it's member initialization, we are protected against concurrent stores/loads/swapoff through the folio lock, and are protected against writeback because the entry is not on the LRU yet. This way, we don't have to drop the pool/objcg refs, now that the entry initialization is centralized to the successful page store code path. zswap_compress() is modified to take a zswap_pool parameter in keeping with this simplification (as against obtaining this from entry->pool). [1]: https://lore.kernel.org/all/CAJD7tkZh6ufHQef5HjXf_F5b5LC1EATexgseD=4WvrO+a6Ni6w@mail.gmail.com/ Signed-off-by: Kanchana P Sridhar --- mm/zswap.c | 56 +++++++++++++++++++++++++----------------------------- 1 file changed, 26 insertions(+), 30 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 09aaf70f95c6..ddbcf34676c5 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -881,7 +881,8 @@ static int zswap_cpu_comp_dead(unsigned int cpu, struct hlist_node *node) return 0; } -static bool zswap_compress(struct page *page, struct zswap_entry *entry) +static bool zswap_compress(struct page *page, struct zswap_entry *entry, + struct zswap_pool *pool) { struct crypto_acomp_ctx *acomp_ctx; struct scatterlist input, output; @@ -893,7 +894,7 @@ static bool zswap_compress(struct page *page, struct zswap_entry *entry) gfp_t gfp; u8 *dst; - acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); + acomp_ctx = raw_cpu_ptr(pool->acomp_ctx); mutex_lock(&acomp_ctx->mutex); @@ -926,7 +927,7 @@ static bool zswap_compress(struct page *page, struct zswap_entry *entry) if (comp_ret) goto unlock; - zpool = entry->pool->zpool; + zpool = pool->zpool; gfp = __GFP_NORETRY | __GFP_NOWARN | __GFP_KSWAPD_RECLAIM; if (zpool_malloc_support_movable(zpool)) gfp |= __GFP_HIGHMEM | __GFP_MOVABLE; @@ -1413,32 +1414,21 @@ static ssize_t zswap_store_page(struct page *page, struct obj_cgroup *objcg, struct zswap_pool *pool) { + swp_entry_t page_swpentry = page_swap_entry(page); struct zswap_entry *entry, *old; /* allocate entry */ entry = zswap_entry_cache_alloc(GFP_KERNEL, page_to_nid(page)); if (!entry) { zswap_reject_kmemcache_fail++; - goto reject; + return -EINVAL; } - /* zswap_store() already holds a ref on 'objcg' and 'pool' */ - if (objcg) - obj_cgroup_get(objcg); - zswap_pool_get(pool); - - /* if entry is successfully added, it keeps the reference */ - entry->pool = pool; + if (!zswap_compress(page, entry, pool)) + goto compress_failed; - if (!zswap_compress(page, entry)) - goto put_pool_objcg; - - entry->swpentry = page_swap_entry(page); - entry->objcg = objcg; - entry->referenced = true; - - old = xa_store(swap_zswap_tree(entry->swpentry), - swp_offset(entry->swpentry), + old = xa_store(swap_zswap_tree(page_swpentry), + swp_offset(page_swpentry), entry, GFP_KERNEL); if (xa_is_err(old)) { int err = xa_err(old); @@ -1456,6 +1446,16 @@ static ssize_t zswap_store_page(struct page *page, if (old) zswap_entry_free(old); + /* + * The entry is successfully compressed and stored in the tree, there is + * no further possibility of failure. Grab refs to the pool and objcg. + * These refs will be dropped by zswap_entry_free() when the entry is + * removed from the tree. + */ + zswap_pool_get(pool); + if (objcg) + obj_cgroup_get(objcg); + /* * We finish initializing the entry while it's already in xarray. * This is safe because: @@ -1466,25 +1466,21 @@ static ssize_t zswap_store_page(struct page *page, * The publishing order matters to prevent writeback from seeing * an incoherent entry. */ + entry->pool = pool; + entry->swpentry = page_swpentry; + entry->objcg = objcg; + entry->referenced = true; if (entry->length) { INIT_LIST_HEAD(&entry->lru); zswap_lru_add(&zswap_list_lru, entry); } - /* - * We shouldn't have any possibility of failure after the entry is - * added in the xarray. The pool/objcg refs obtained here will only - * be dropped if/when zswap_entry_free() gets called. - */ return entry->length; store_failed: - zpool_free(entry->pool->zpool, entry->handle); -put_pool_objcg: - zswap_pool_put(pool); - obj_cgroup_put(objcg); + zpool_free(pool->zpool, entry->handle); +compress_failed: zswap_entry_cache_free(entry); -reject: return -EINVAL; }