From patchwork Fri Jan 31 09:06:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13955131 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66D92C0218D for ; Fri, 31 Jan 2025 09:07:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E5A9D2800F1; Fri, 31 Jan 2025 04:07:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E13FB2800EF; Fri, 31 Jan 2025 04:07:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA97A2800F1; Fri, 31 Jan 2025 04:07:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A8D4F2800EF for ; Fri, 31 Jan 2025 04:07:28 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 1A8B712097C for ; Fri, 31 Jan 2025 09:07:28 +0000 (UTC) X-FDA: 83067168576.02.E507DE1 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf26.hostedemail.com (Postfix) with ESMTP id 296E0140004 for ; Fri, 31 Jan 2025 09:07:25 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=aRvxUvGl; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf26.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.180 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738314446; a=rsa-sha256; cv=none; b=k6m+pVC7ar8d8HoUFiU/pH9mmFZQHba6iAQxXE7RrFxi4H3MjdHe+0VRL+150d0+8TCS5G B7AgqiSxcll3TZ47WDRS7UJtqJjYM/lhNfUUlDHcwmnbN6CwacJffTtnrlRTRzpag931QN ZJREkI4to2UUYp1vyR0QnUn20Fn/C4s= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=aRvxUvGl; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf26.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.180 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738314446; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MtsIOf/TBIswacsBKgt0lP8Ca9NRj+2uNLPxrfHTdS0=; b=Bd3VgJKB19YNJWmEuU5kO7yPezWMZ1eroqQa0TzQjn4dH01oD7ZWZWPXEovV00uXhgN8AV 1ffzqcAkPBhClDwu80T3e90qVJceMvo5EqQ+ZDKKJQk/EpptI8sMnSy3RbuARZcw4boNPC XIf95yp1Naau0qVq8hqLMEChukCyEbo= Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-2163dc5155fso30730735ad.0 for ; Fri, 31 Jan 2025 01:07:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738314445; x=1738919245; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MtsIOf/TBIswacsBKgt0lP8Ca9NRj+2uNLPxrfHTdS0=; b=aRvxUvGlPYbVEJ2toYDMmZ6hIZrX7cu5yBd2XetOHfyV/cOasyPWbrPCrP9S872GbM sl5NwCZMPPgjWJqqcQwYlwQffgh+0UH1GLTteLNSbl5BS0HmbLqYHkz5NIN5gmoUX5zI IytcYew1t2d6VZ32/jy7aViko+/kYwg+7p1So= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738314445; x=1738919245; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MtsIOf/TBIswacsBKgt0lP8Ca9NRj+2uNLPxrfHTdS0=; b=Gp7yYth9QsVaCos7p3wa0NPBR8XX5bk4OTnUHYSwZopbhaJyWvYc+m0saw8xauZ3+j bzn1/cm1OvVpuQi4ptQfR3pxTZyylVSQEYHtVTwmdhYkhJaFg1GKmpEKmvvRIH19jFAY WAVn+NbUmhLK9LNW/RI9vpjR21jFKMvP4Sitlr9fXGlB6azahAwbmxZuRvJMTCm8Z1k6 3kaIyxE2JBLSWXXhNSF6iVpuSPNqNwjBo/TwY7l2z/tDJ+mrUN9nYzNlFR48i8m5OT5n bWwH+0VsNCQ20nwoX/GZrEIQvytwAtsokuWEtSASrvU3jwyaY73juNiVtjIdPfIXTbz9 kYMA== X-Forwarded-Encrypted: i=1; AJvYcCVP1dkT//mB7+oYGWvPXCg3/9Q9fwnFCRtbwD7EnuDYY0bb4oyvNxeIyQWqai8ozP76beTjnh601g==@kvack.org X-Gm-Message-State: AOJu0YxtDK4fqICrvGGnAL7cDaxyobjoxh05JaP1rtCjbTINIgVyoC1X a9G64/Qfv1waZvl2GOMMo5NQ5HtHiP0iBp8YeVcFZihSE8Hwd+m+Ls1m/tNJgg== X-Gm-Gg: ASbGncuUmcb2KK7zb6PUs9kTxtT3MRzm2ESz/gBBFETHUTCkNOkoMU+8A9W8vbDkfHT kEk7KzyM0G6xJnCLllLaY+jI3FMxdj9W8tNhtVTvZ3+nMwYFerVEEQ/JzGS7MvlMO8Z0JkqUVfS P5R9jlI6+vfuDuhmkuMxTsV6k/NyhxZsjwJ3alaFUXidi597tDNruMfwTftRX3i+T5ZBchPlr1j gxB4ECCWaLuICR718Gmf2wCCOcazlGhWKpBw6XdOQD/Sbn3ICkN/vjQfbHSodZcjb9fqD+Xub3w e7ZzUWC79P82Oamjqg== X-Google-Smtp-Source: AGHT+IG1fS5h4JbjO4Mc37VY88UPXs9TdEujSTlHdi2szESpBE3nfr8tk5rye/+mNhmuHHPPmpGn9w== X-Received: by 2002:a17:902:c40a:b0:212:63c0:d9e7 with SMTP id d9443c01a7336-21dd7b61a82mr166145155ad.0.1738314444996; Fri, 31 Jan 2025 01:07:24 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:c752:be9d:3368:16fa]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-21de32eb9f1sm25752285ad.115.2025.01.31.01.07.23 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 31 Jan 2025 01:07:24 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv4 05/17] zram: remove two-staged handle allocation Date: Fri, 31 Jan 2025 18:06:04 +0900 Message-ID: <20250131090658.3386285-6-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.362.g079036d154-goog In-Reply-To: <20250131090658.3386285-1-senozhatsky@chromium.org> References: <20250131090658.3386285-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 296E0140004 X-Rspamd-Server: rspam10 X-Stat-Signature: bws9y5fehz4swfkmazc7jdadqno7hjcd X-HE-Tag: 1738314445-576200 X-HE-Meta: U2FsdGVkX183f6hysPOSoYmbL/udtjFuWYobJsnezcmWwNekkyWsx9Lwe2QGNBXhVShiJTf/ydutylivxbKhqXJUcN4Xt6k8rVCMt13li943uwplzmDc2xW/qohVtmnJJ869rotcH7hcDIBKFYJwIzhJsSd0krooNtZSlzldmzZmIhGHS0tIsDvfHc9box1YqZsls2LPDiwStorqqi/Rta3fa2uwA4mRteuVWQqzgTQcRjI7UulEUapQTfEh3GFqJpBZKDGhNsXH3EeCTxRnev87jeAlCmucVO6PomDtQpQ7qgYv8xS2/+8DLMBgKlJxy2VUDpcXt3v+qpIGCEKmNVNJNScmBtWJ8BLchzSO+vV4GT97RH4xYT7rLrcRiQRFj7+UQzwOBTjNIfEtCG8XlFJ7jtnRx6SZBG4rXZ1F9F+QO0++i7IhHA5RNnVdbwa4fyEUCcHCQfNMfCgSLroOTUmaQ8dCtRmgZbL4lIoiJosUoq6p9gFNZ9oBaB7gscFSn6qcRW8tz0NAk32K7b1IQEUyxkZzZa2l6EoSfA8EB0FpkD9uRxFFwhh6j2I3WkOHg3y0zOduJ6M9doLQJNnmaAZY+SJ2owQUNjK9rkNyQAAA4LuCIuQsNV9igUzR2zY7DsdtvScyDQ/yMNxvEDlnl4zYIpSuBhBZhQn/WrEr+XfkyCq5LY6k6D1c4TY2LcrhGhaifPd8RBieGvsj5zJJ3FDKuREqkOzyYZB09SbNx1o8tsVwfOuydmx8UZDVMK0PhabeOvQu1llPMIBN8Y0IpbE2LSy11xw2eJS6kZAiIUtbkpRepT7D241u28MnoP5pUyOQFLge23j6Vh4B6Y0kE2lMnh0b0qtJWuLAW7McuOqnXJRoPsyhWftUu4JBY9G8JPnLL2wb0qzKQ9yWlTcqv6lfJDW6Q66mhFVJKGsmkzeivYRUWE1QN+5F1W1FE3Q+AykBR4tVlYgpL2TwQXk zcaym3qw cK7AoiL2Jsde3dp4ASvsMCMRuK54YwmAdvTUjkrz53/DxpejIWVppUlfPlMnW9m7jne4IaYvGzjN+dkAPOIehp6LLSIhxI3hqC0Jadi6eqAjDPpfXoXlG1U2ykxm2Bevcpj1cCi7UAyq8WuWEeHHExBdF7dKlptLI6A6IAoY350sNQtmPGvCTM0/px9NE9pMxUzu6J25i1xPagEXY/r0wHulJvI9HYfnkmtsVCTI6Q2sKGl5W9QLEXug6lX8qTCy8sYSSyEu7TZcxWl09O18iS7YYKh3rUqnbBy2dILmiKqgy9jJsV1l4KgibOw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Previously zram write() was atomic which required us to pass __GFP_KSWAPD_RECLAIM to zsmalloc handle allocation on a fast path and attempt a slow path allocation (with recompression) when the fast path failed. Since it's not atomic anymore we can permit direct reclaim during allocation, and remove fast allocation path and, also, drop the recompression path (which should reduce CPU/battery usage). Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 41 +++++------------------------------ 1 file changed, 6 insertions(+), 35 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index dd987e3942c7..0404f5e35cb4 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -1716,11 +1716,11 @@ static int write_incompressible_page(struct zram *zram, struct page *page, static int zram_write_page(struct zram *zram, struct page *page, u32 index) { int ret = 0; - unsigned long handle = -ENOMEM; - unsigned int comp_len = 0; + unsigned long handle; + unsigned int comp_len; void *dst, *mem; struct zcomp_strm *zstrm; - unsigned long element = 0; + unsigned long element; bool same_filled; /* First, free memory allocated to this slot (if any) */ @@ -1734,7 +1734,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (same_filled) return write_same_filled_page(zram, element, index); -compress_again: zstrm = zcomp_stream_get(zram->comps[ZRAM_PRIMARY_COMP]); mem = kmap_local_page(page); ret = zcomp_compress(zram->comps[ZRAM_PRIMARY_COMP], zstrm, @@ -1743,8 +1742,6 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) if (unlikely(ret)) { zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); - pr_err("Compression failed! err=%d\n", ret); - zs_free(zram->mem_pool, handle); return ret; } @@ -1753,36 +1750,10 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) return write_incompressible_page(zram, page, index); } - /* - * handle allocation has 2 paths: - * a) fast path is executed with preemption disabled (for - * per-cpu streams) and has __GFP_DIRECT_RECLAIM bit clear, - * since we can't sleep; - * b) slow path enables preemption and attempts to allocate - * the page with __GFP_DIRECT_RECLAIM bit set. we have to - * put per-cpu compression stream and, thus, to re-do - * the compression once handle is allocated. - * - * if we have a 'non-null' handle here then we are coming - * from the slow path and handle has already been allocated. - */ + handle = zs_malloc(zram->mem_pool, comp_len, + GFP_NOIO | __GFP_HIGHMEM | __GFP_MOVABLE); if (IS_ERR_VALUE(handle)) - handle = zs_malloc(zram->mem_pool, comp_len, - __GFP_KSWAPD_RECLAIM | - __GFP_NOWARN | - __GFP_HIGHMEM | - __GFP_MOVABLE); - if (IS_ERR_VALUE(handle)) { - zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm); - atomic64_inc(&zram->stats.writestall); - handle = zs_malloc(zram->mem_pool, comp_len, - GFP_NOIO | __GFP_HIGHMEM | - __GFP_MOVABLE); - if (IS_ERR_VALUE(handle)) - return PTR_ERR((void *)handle); - - goto compress_again; - } + return PTR_ERR((void *)handle); if (!zram_can_store_page(zram)) { zcomp_stream_put(zram->comps[ZRAM_PRIMARY_COMP], zstrm);