From patchwork Thu Jan 30 11:10:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13954493 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B7AEC02190 for ; Thu, 30 Jan 2025 11:13:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CF00A28028A; Thu, 30 Jan 2025 06:13:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C51462800D0; Thu, 30 Jan 2025 06:13:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ACA6F28028A; Thu, 30 Jan 2025 06:13:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 883722800D0 for ; Thu, 30 Jan 2025 06:13:07 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 972EFC0EB0 for ; Thu, 30 Jan 2025 11:11:40 +0000 (UTC) X-FDA: 83063852760.03.BAEC7F7 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) by imf22.hostedemail.com (Postfix) with ESMTP id B5F0CC000A for ; Thu, 30 Jan 2025 11:11:38 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=nAAKhyq5; spf=pass (imf22.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.173 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738235498; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ujanKWetHL+pQynH0HIAEO2xKM/XaqicaLx2SNCooQM=; b=nYrHMR0Fgeaa6TGtn8BZcl0HDZ+HvQrYJnzSNBaHnJ+WU0mzCDA9p1GL+idNn2aUS6txI0 4FDOHF2iPyJB7eV+KMLOD6keS84K9pC9apmvE9Vi8l58hXCiTcnFF7EcM0XJPtr419VA4A OXEmnI9tWj0uOLctTsE0lQQNhYoB8vk= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=nAAKhyq5; spf=pass (imf22.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.173 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738235498; a=rsa-sha256; cv=none; b=mjmTDevbO7BpEzr004LOXUGILxRDnr93QtBVbCtlgsrKFEFjxtLrKiskk133GgJ+VtxuBf Ll0zhvvpl3/R8vmRk5Lr9/+v+nyigpqtqz+6Hln3MGoZhP2mHuP7gE1Gi7hgdOKwCHuwCJ ZR+5XeJVUglbbcBZ8x8xfkJPC8cjbwE= Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-2167141dfa1so9626125ad.1 for ; Thu, 30 Jan 2025 03:11:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738235497; x=1738840297; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ujanKWetHL+pQynH0HIAEO2xKM/XaqicaLx2SNCooQM=; b=nAAKhyq5sxN3jfp7Fp2+0s0wuGEQDqXGeQ+y5g94geg8ag6GL7Q8ja7iuC2zjMEcOe ArKGa03ZngyrEpwaAa61dtMyRqB8hrYx8Dk7NRDg0ASfsLKEzisTfMuiEuPHrW0gqW9k EPgsHpP6rdRbYUzi099wxWdtmbdeHZugKPCmI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738235497; x=1738840297; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ujanKWetHL+pQynH0HIAEO2xKM/XaqicaLx2SNCooQM=; b=r/JQJvp7ItAGZNKz5wPJCUzHCnNB8SGiLy+sxt9r5wYWudSJEihcwi54YSyoUxDH96 13hu8H/2LbHkwQXpfX6I1tY9MJqQxuozcNM6ejaPRw2+cd0WI0T51BsUqARpblNi6dND 4URJnfNRkfGELaSwYsfgeA4WycxwknoAgk+ihFp+isPLOYJwj/kOxQCfpUTyfORLxRyQ o5r5GcOcKpnXFRj0QDGAs/l0UAx2dgOyqDrB0gum+G78KoCsDiwVbT6Y2eDxZo2mM0cp OhIzwjA7lVz3tpv6hIPcMD3hJg2SVgQgNCUZy0rtVGsAag/X4pAgrwwXBsA4a+hHTkmW 2RBg== X-Forwarded-Encrypted: i=1; AJvYcCVnPAhjC/rH4BBk/w4BDj06o7ODEDDCnmsISwNVv44dFvPQ04KlLmN+29bdGCkh3EvJXvmtFk5Gwg==@kvack.org X-Gm-Message-State: AOJu0YyoAP2En7kbSoCKdMQvt9w1U+ic+j1BkETrnYva8mZOYk4YN9yI L8MYHApGwvXXYSOUJju0+X1JkOvVNysmHg0g6QZqhHUs/IsxsF/pSC95Eb4sNw== X-Gm-Gg: ASbGnctrhmPiYoIJde0aPTy79mNet+68oahFIheT7j0kPq8bNKcguyTOUQBBM1fMo2v h+96o3YlKkVcU0dIDa1Cwit3bSpdX+jErUkTdJs/xn/ve6WjEpOA1cg6RcV1lHuw5/x/uOBdFbs 1LMAdoPWxoeEej9xf5AYJ3gZMxSiZLr80Hu3E3cGEmGNxyymiVW0K5a+OAy4jLEttv/JIpB0ekx dqN+g/t8hejOlGAeOXdXkSW6RjC5CHBVhBu7aaheOc1O2yzbd7uuRyGHrPEBF0uxhyUsAe/zORd GuG/k/dVqTv+yRaI X-Google-Smtp-Source: AGHT+IGoJPBvbRFKgcHes3aZpqhjq0bpB3vf3UMoxOpT1s+vDvy+5JFl5BGgTN400Fd1/JslRst1fw== X-Received: by 2002:a17:902:c946:b0:216:14fb:d277 with SMTP id d9443c01a7336-21de196fb0amr42131665ad.22.1738235497536; Thu, 30 Jan 2025 03:11:37 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:d4ce:e744:f46b:4fb]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-21de31f836csm11410515ad.93.2025.01.30.03.11.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 30 Jan 2025 03:11:37 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv3 05/11] zram: remove two-staged handle allocation Date: Thu, 30 Jan 2025 20:10:50 +0900 Message-ID: <20250130111105.2861324-6-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250130111105.2861324-1-senozhatsky@chromium.org> References: <20250130111105.2861324-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: B5F0CC000A X-Stat-Signature: 9i4x8mb76hz11c5rwhcmzccec9fbp66p X-Rspam-User: X-HE-Tag: 1738235498-895611 X-HE-Meta: U2FsdGVkX19tg10KlYHCdzVfyPv04vJWI4bpTVSFc1I6ti9cgAJG7rgAL700CSPpDT0qrlyFhf5ohMM0qtuBiS1/BXn2B2/z0uePXfEkrIlu40ejI/K1mPXjll/9JgaRQZxTgJJCTu6PxUAhV8xsYUHuNs0cGxKfEgx45jXMnj4wW94sviuH0LzxB7FwVo4Xagw25Gq04KmNXBHzPBjnTP5d1oRB+rHzHVbhj1Sgga1onYemkgQ+YmWVTL9MA1AtL8H0T2CtgMgwndTfGGfB3DXnLlrd1UW7twJ1AC0Amx60IIEO/qYlFALY77XYqnaX/rawT+aJQOdhlDOCnD+pNaCus8dTUvos/4Ajj/obrrYDWSL+wnQY8HVjhsI7+q9ZKU9D/f/98d1pZubY8Zc3li6nf2jYp5q6hjQZw8DJ9k1jxBmeDoBX0bW2G7FxRUVwj+CIgoY577cfcvcj/PRWMvbxAyKu61MNpQ9MStUWh5m1V7qBDeo81f6c3Uyz594VEmzcWu+nHwoBesch86diVYoUkwom/uj4V1lnTjwEqP8rR8ghjf68/YHNvbcOu9930vtVylrxYYv6GnTJegCYJeH3x4WNCvasJ3Sjkq2muhmwjA9Px7CzFvEhaSCVHpzzSUbQML8FmcrchaEl76CuiB8DmECpk99diCvO/5CrmumBdmBf8MEDQg4RkkYci2AZHlds5yH0tteR+v0aG9R7vHl9ExoKbLNKyc9bDzOZMEmzi3GGEPSJ4g/ikYgf7BVJS+EW33K1CiVyl3bgr6z8af90XTtL1zhK2m8F2ywl/3nJ8HCvCH9JLCs89og/4MCvVqd6j9GYQyyQsOH8w+s5Q/AmOZqey/VpvKjSSQXSj3uPj17U0ijJyJsYv5df4EN4b4RN0EZ+8hHahVqzAWkWzBsBEIIjwkJsOAobxk5r6VLVp7p8ScCn6Ui6MPvUNB1j6fJZLrCA5WqpojrDM36 sp3R45Gc H69SOIIXDPJmGJw3AdtsHT3zij+MwtDNOstrpaJsZH6CkHyWB1jk/UP+u5uY2Z8YVuY4IJprepfR/wxP0KzNHYYNB9mr1PtxiqO1PtS1x+gsyG2ke0hV+sSui6mV9X2Uy7AKWWQjfkEQO1UGfSZkq+d4VvuYq6GWGSnyDRBICGUbxq4Y7xwddS7YpKezr8ak+qnr5DZgFXIzTBM+niiRWRDH4Bxbx7ObdgoVNh/nPfuiicreayt7IMSTF6E9Udl2ll7kc2dkKULi70IJIaUA5TGP0fmcws18OxYu94qNPoVx4FUdu3bXhywyemQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Previously zram write() was atomic which required us to pass __GFP_KSWAPD_RECLAIM to zsmalloc handle allocation on a fast path and attempt a slow path allocation (with recompression) when the fast path failed. Since it's not atomic anymore we can permit direct reclaim during allocation, and remove fast allocation path and, also, drop the recompression path (which should reduce CPU/battery usage). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 41 +++++------------------------------ 1 file changed, 6 insertions(+), 35 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 0c9b68fa8783..28e5b02dc3a1 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1718,11 +1718,11 @@ static int write_incompressible_page(struct zram *zram, struct page *page, static int zram_write_page(struct zram *zram, struct page *page, u32 index) { int ret = 0; - unsigned long handle = -ENOMEM; - unsigned int comp_len = 0; + unsigned long handle; + unsigned int comp_len; void *dst, *mem; struct zcomp_strm *zstrm; - unsigned long element = 0; + unsigned long element; bool same_filled; /* First, free memory allocated to this slot (if any) */ @@ -1736,7 +1736,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (same_filled) return write_same_filled_page(zram, element, index); -compress_again: zstrm = zcomp_stream_get(zram->comps[ZRAM_PRIMARY_COMP]); mem = kmap_local_page(page); ret = zcomp_compress(zram->comps[ZRAM_PRIMARY_COMP], zstrm, @@ -1745,8 +1744,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (unlikely(ret)) { zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); - pr_err("Compression failed! err=%d\n", ret); - zs_free(zram->mem_pool, handle); return ret; } @@ -1755,36 +1752,10 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return write_incompressible_page(zram, page, index); } - /* - * handle allocation has 2 paths: - * a) fast path is executed with preemption disabled (for - * per-cpu streams) and has __GFP_DIRECT_RECLAIM bit clear, - * since we can't sleep; - * b) slow path enables preemption and attempts to allocate - * the page with __GFP_DIRECT_RECLAIM bit set. we have to - * put per-cpu compression stream and, thus, to re-do - * the compression once handle is allocated. - * - * if we have a 'non-null' handle here then we are coming - * from the slow path and handle has already been allocated. - */ + handle = zs_malloc(zram->mem_pool, comp_len, + GFP_NOIO | __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) - handle = zs_malloc(zram->mem_pool, comp_len, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); - if (IS_ERR_VALUE(handle)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); - atomic64_inc(&zram->stats.writestall); - handle = zs_malloc(zram->mem_pool, comp_len, - GFP_NOIO | __GFP_HIGHMEM | - __GFP_MOVABLE); - if (IS_ERR_VALUE(handle)) - return PTR_ERR((void *)handle); - - goto compress_again; - } + return PTR_ERR((void *)handle); if (!zram_can_store_page(zram)) { zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm);