From patchwork Tue Jan 30 01:36:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536667 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 223FAC47DA9 for ; Tue, 30 Jan 2024 01:42:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 870976B00A9; Mon, 29 Jan 2024 20:42:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 821566B00AA; Mon, 29 Jan 2024 20:42:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 674206B00AB; Mon, 29 Jan 2024 20:42:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 546616B00A9 for ; Mon, 29 Jan 2024 20:42:18 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 2924B140AD6 for ; Tue, 30 Jan 2024 01:42:18 +0000 (UTC) X-FDA: 81734277156.07.C16433A Received: from mail-qk1-f179.google.com (mail-qk1-f179.google.com [209.85.222.179]) by imf14.hostedemail.com (Postfix) with ESMTP id 32A37100006 for ; Tue, 30 Jan 2024 01:42:16 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=qwx1yi1U; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf14.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.179 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578936; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=shLTg8CbwWEN2YIGT7dz17BRQIDu5G+tg/uQhre6QiQ=; b=Z+zDiFLbzWSjvaiJDg8A3pq+EK6jC98r8W+Fs6vt2iU9CUJQ3Axfj/Ekv2lPBXoXhfn/Hq p9P3rYBk3kGWq+VFezcPoy9iIp824DqywIDy1Hb4QY2jj0RyIjYh1jeJW1qh/8pKlDBQZn Vuo99fQXYgS7bJKxLh0uE7lg8rfn/CA= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=qwx1yi1U; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf14.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.179 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578936; a=rsa-sha256; cv=none; b=MQRYlD8c1rsVIWEIUVCuvDpjSAQHuyr+m/wS6oIqwQcz6zQrkyfnKmlXyiHzHFgDPxcPaE u6dViu04+xUoiOI+zF/P9bNiwhYvyVuEC1UykNKZmOA6S0acF1k0LR/7DOZs9I6fStNE8B G5aSokHvgK7mMirrKkMgSVRXUPOoO0k= Received: by mail-qk1-f179.google.com with SMTP id af79cd13be357-783f49812aeso147112185a.2 for ; Mon, 29 Jan 2024 17:42:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578935; x=1707183735; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=shLTg8CbwWEN2YIGT7dz17BRQIDu5G+tg/uQhre6QiQ=; b=qwx1yi1UBTz77b3b7rgmi/8HfVOjJDGh4y6qFmGOydL8fW/gGwtMh4JK1b9Ku+oXA5 mrbtbUhllPxGMHPBzq3tRP1WEBO5vOtxLd6nlf87zTxA1n4LODlxKYmT/xumLUfeJA43 O9hjdsbWw6003R/+TNtUNOx0MCl02cf/I0VcxiLwiaXPTSvp9R4lIvUaSNrCO/jxBl+e FUDBt/o7m5CHJChpg5/LHxfJ+368aDYsQtC6/Z56OAmP7FcFtxdqYPRGVslfku0ZyM+o kBrfFBUTcJSaYFcUUGgvKXY5Tb7XpeYMcM15NNm7iDgHVpLlKCdnx/jkDD6tuapVR5AH h6eg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578935; x=1707183735; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=shLTg8CbwWEN2YIGT7dz17BRQIDu5G+tg/uQhre6QiQ=; b=VvYgjIKKxhy/hgtUQvuenax1h7M1QbPN51yugOEdGjgeED5g3QTIm25NtqqRss7N3T MuAx292GXCq9EV1O702qr5aCwuDwXaU/wpLAgKJKNGK4mEgpxbf5k4K398Y17/0gUq64 mK71PDvcRLicWTP3jTx69liGfsnCQuNlXKBivgaS79b5KZMLpRJ1eY9GD6gNhQcVturR ZXsTwFZ3/jfYiNYlhrK7unjE1/s/pE5CL4uqLS6pap7pHAvqP3GpnwNgnJMHMSshIi/P 38RN038ApRuw8UtsqDLm/uFmR1rAqwn8L9TDL32/AOAXD9gM1Iw00xmNoV9p0OgrccZ5 DekQ== X-Gm-Message-State: AOJu0YxqZfg4EM674PWc0KIdBfxggvZyzWlfnZSiCGFEdwPXU1SNo/E9 U2fiDClpl4avVjSQodLmg8+6UvSvw97KUwlTbZCe4NIWDtvmqPEi1rBBZ0m64Rk= X-Google-Smtp-Source: AGHT+IE9qj83y7G4z9fihmAKWWcQVG1MGvLQrnNFWv+a/2imYaxhEYRxeU9+t7c0pWlonGDYF9KF1A== X-Received: by 2002:a05:620a:15ae:b0:783:8a70:f64 with SMTP id f14-20020a05620a15ae00b007838a700f64mr6881421qkk.65.1706578935257; Mon, 29 Jan 2024 17:42:15 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id o26-20020a05620a111a00b00784046b7868sm794053qkk.90.2024.01.29.17.42.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:15 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 01/20] mm: zswap: rename zswap_free_entry to zswap_entry_free Date: Mon, 29 Jan 2024 20:36:37 -0500 Message-ID: <20240130014208.565554-2-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 32A37100006 X-Stat-Signature: gt575emkgttemep343jbm9gcwbsrdwdz X-HE-Tag: 1706578936-170591 X-HE-Meta: U2FsdGVkX19zwlJ4qXDN92Y6/VBC4Jfi9t/Zu9tQIJ8ZJ7aPQjP2qfDY2YV7KYFoz06rBkq7oDchUN2Ewr0OasRpdUJ70n1s38zyKgdwTMAC1Z3Y3XfD4iBfi+TdLU6i0tT1/isUIn2vQnsBI8bWKMzZnLAEGww6xnR1fBBb1gevoihux6Ye1DTXYt1JpY1rRf5z7AcxwxDPLL/3+UCv2ATAY+mx1Wc305I+fLReHwjuLRC0bgNzYfv61hDMszSQJdDy/VXumnEyeJzS3rLpZNtqiHmw/+FIS9V+UOL58JL4DgmaCsOCADRs8QZLtnBmEM4KfMmk96s9iGVqzZZg8p1hEy9FibUrvWSnF4gSqRooZx8K+iTDHa+NsVGSb9C6cujLWkjye7asfUzJC542QAaQ5YFnKDFuuTnVMd7InT/rMAUjHssTXxbEfocav/Wro/1wJQdXIwTuVQF+EVcUaLn8XklB7jNwvFIszrkcfjLKj+SgYafxPBfEaq/JUxoQHVRt88fJe2OQ4qGozCwq6e0mgeX+eddwYcDZ/WxNH++yU/ZTl4h53l3Rz3bxEStDY+21gueYkFCbnFY6fOLRygMfPoZZPnQFJzXeYR6b55F6ujsFhGm9ItxanvA07XtST2EdJXPn9FCCuxiBcLSU7E8pg9ufxd251KlcckV4o9HGHJXbXpLZvWdmAPMrWVOFauc9bzmlk7ulOIga0tNHxCWN+tDuctwdWzfa2fhnoH2Z6GwmK++DMv33EfwYZ9xdITOcy4BfJ07B4qdqz0KlB6r0HCkhuJ8T4Eaf+a8xCH6hnTrY5n0QoEcS3lyXm0v+6Djyi2c/syGh5iH3G9tzZl+51x1eQVRjOWgRkri1cyQ8aSqKpwTDYjljWhzBUSnnvm3TP/Vpj7/BJTkb5N/4TCiWkQrb2ctsgTQsqMDbvx2p+2azWaQxKxw2oy9gtuG8Rc+POUCLPiH4TjVMxeN QjC3Dz/A fQzBYev11bMOOX7q1GU2+DunGXQHdeRCT+uS1dtAhk2Ehyp7HfAz2sNy4g9d+muc+p2XY3g+9A+AHr0b05LjNNh+zEF1WN01jpTaJoPXXZjD25oitae86THOHBcQjN+XqRm4j267XSv18+Ssfi25yNIjyPbQy67zbi7/lOpXQx+DY8hGi/sLZ5ssy4MM7yOO2+d8zTuh6fNFCt/gVIfyEK2fpwPV19qGSg9ZlKZn6qaBHXSC2/n4wT+J8mo5TeDKZxkayrD4kaoOD2jmbcq3hrGV1IYOxM/dnS/m54zbECJB9ql4rbM74bypOiWf7UrpmtbWyjCVzNrd9D7Id+xn5QkqV42MBknMhrifp8eX5YJC+SHYxIo8hU2Lq/+LT6kYuKOuZS1rVazGyuSiU7V3EKTIMrqYA5EwjmdPneVFhmimekpxPmRbEczngmQ9xf9j9jg8Jg5zPvzEKoAtCWgIPYEwMi5O4ha8M2CJA8YRFFKjoOpQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There is a zswap_entry_ namespace with multiple functions already. Signed-off-by: Johannes Weiner Reviewed-by: Chengming Zhou Acked-by: Yosry Ahmed Acked-by: Nhat Pham --- mm/zswap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 7f88b3a77e4a..173f2e6657de 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -520,7 +520,7 @@ static struct zpool *zswap_find_zpool(struct zswap_entry *entry) * Carries out the common pattern of freeing and entry's zpool allocation, * freeing the entry itself, and decrementing the number of stored pages. */ -static void zswap_free_entry(struct zswap_entry *entry) +static void zswap_entry_free(struct zswap_entry *entry) { if (!entry->length) atomic_dec(&zswap_same_filled_pages); @@ -555,7 +555,7 @@ static void zswap_entry_put(struct zswap_entry *entry) WARN_ON_ONCE(refcount < 0); if (refcount == 0) { WARN_ON_ONCE(!RB_EMPTY_NODE(&entry->rbnode)); - zswap_free_entry(entry); + zswap_entry_free(entry); } } From patchwork Tue Jan 30 01:36:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536668 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55D6EC47DA9 for ; Tue, 30 Jan 2024 01:42:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 62AC56B00AA; Mon, 29 Jan 2024 20:42:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4ED4D6B00AB; Mon, 29 Jan 2024 20:42:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 340566B00AC; Mon, 29 Jan 2024 20:42:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 1D0AF6B00AA for ; Mon, 29 Jan 2024 20:42:19 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id F347F40293 for ; Tue, 30 Jan 2024 01:42:18 +0000 (UTC) X-FDA: 81734277156.01.D1ED6F5 Received: from mail-qv1-f41.google.com (mail-qv1-f41.google.com [209.85.219.41]) by imf01.hostedemail.com (Postfix) with ESMTP id 655AC4000B for ; Tue, 30 Jan 2024 01:42:17 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=QdYTEIOZ; spf=pass (imf01.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.41 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578937; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XjDB9WMZenyNFfJ6FneP+eMAzd2uIdtYL+Q7hbWifG0=; b=u9aFt8AwRq9k6QSBT5EcV037cmf+Y0rYzIjaWuyLIdDDmxZ/Q4K7PKYY4JdydmFY2z8zvY 3UhJyr+xfFv6uAA/I7sqI2HEb6o+O7AMk4cr+WllX+k7s+EHsQgEa3GJB3IvbIBgZblxlt ZMbs+yOY7CS5P/VgIoQ9Stp3kpgCG2Q= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=QdYTEIOZ; spf=pass (imf01.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.41 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578937; a=rsa-sha256; cv=none; b=lyHj48BQyJhbhLcKm7ZDjpKruWxS+GrPToUK47HGlcxqRyFtKKgEh6Ht/8TskXndZTskIH YmztASX1/0Rchkrc71MZ05eaWgN3Lt1sX9iWj3nlmf/LyFcj100E6xQax/U4EmT2eOznvK uzqF7ZZ+YTAebmgq1tcgQUYvohGImS8= Received: by mail-qv1-f41.google.com with SMTP id 6a1803df08f44-68009cb4669so26878136d6.1 for ; Mon, 29 Jan 2024 17:42:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578936; x=1707183736; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XjDB9WMZenyNFfJ6FneP+eMAzd2uIdtYL+Q7hbWifG0=; b=QdYTEIOZMAd/OeaWKwCz8uFVXs16m875ZLAL/ENdFRBQLNevcJQIwmP/XHLg6/N3Gb 5M19MGkS1pTYatn3LssKctSEVS2iQYQfJxfNDPHCE+4WKYq2rt5h+VYLGk/W6TZfH374 Yr/tz5MDMlH1MwOuX7n6ARAxp7pGKwAKpClGMq/DcCu/fk7/UXFmkZIgZMIJdq5S08qu CNycy0BsROYdaHrJr6wK0avoj9nGT/R4Vym6w7AyHOPhwa56DfdZbVOXHb5Ne5SKNaxx VeViZNRft0qNf8GXA9Lh48eTlhUVPd3q7o+N649P0Cay1H1aAPt0AtjMpZx/yfce7bfR lNzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578936; x=1707183736; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XjDB9WMZenyNFfJ6FneP+eMAzd2uIdtYL+Q7hbWifG0=; b=K0SSpH+IEN6iWPdAWrqpzKlHGYXFh2WnfjRBQWswCL6qQodH7bBE7ikN4k2E8rkaqR uP+2V6RPoBHFiCrs+Qdk767dvZ/xxZlUQ9G63anm9Z9D0JWKzY7HsjuS4rafjU/nzeJj XdQ2tDjk9fs5ITjqVt2clcqzfyLjg3KdQuSG4VrvbfFNqTpWAwZICzwI0SFuvBn1M58x RPTldqz82t4j5621BU/yam+raj3AVUthXyDjwsrVRYj5e543WuzsaqIJny2+MB4OgvJ3 +f9QkFv3QezU1fD/xte8ahEU6ujIXK5t4Sjia/s0oZjcRtSK4kAVIM/MhSYDhKKN+TuH o2HQ== X-Gm-Message-State: AOJu0Yw7OTwkM4ROtg/lvc1Tt7OZ6aJbtiRH2sPGfnpSpIx3GlGCW2w5 26tA8Yf2+NXUddhcIuLEDUfoTOPLTrgomFRrMa6qKde8HFNeT5adevzuyZxagNc= X-Google-Smtp-Source: AGHT+IHh4RV/gyq1SbWHKSfxOL3yFJXOcLXUJDddqpgCqc+YTjcqv3Unc9iTscwNHWga7GShhVQD0g== X-Received: by 2002:ad4:5c4d:0:b0:686:ac5b:fe7d with SMTP id a13-20020ad45c4d000000b00686ac5bfe7dmr9546558qva.125.1706578936382; Mon, 29 Jan 2024 17:42:16 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id u12-20020a0cea4c000000b0068c43ac393asm2518373qvp.146.2024.01.29.17.42.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:16 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 02/20] mm: zswap: inline and remove zswap_entry_find_get() Date: Mon, 29 Jan 2024 20:36:38 -0500 Message-ID: <20240130014208.565554-3-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 655AC4000B X-Rspam-User: X-Stat-Signature: qy5mn1dnjryyjdqn9hext9a7kdasrbsa X-Rspamd-Server: rspam01 X-HE-Tag: 1706578937-574401 X-HE-Meta: U2FsdGVkX19QqWXPMMB5WSRN4RdJAX+OHuIuP9nz6KvUFhFjW0euC6L6DIokTgxE64ni7S+HbGgUQN5UhuSdV0n4hxXtEvtJWEsHhjMdD3F198cfN3XJRBbdBKfW5NAePG4qk46PDMxDfJKP2fI9PkJ2TQ0P7diBk/OZjyb3DNlazpsisOnfN+jg/0flAaOiQUNTsi4AB671G6wPt6U4POpo6VUBvvWdqJzYRFaPvq7RrTZBRSQZePHRL3lIvj37gh3CduiOBDxXaE01BRyfHGPpW5Y8pQENZHPeee24pxOaHo1rc/D4Amv0JTIyOb14dz0gSeu7XQ8OeTGnp44QEriBmN2Wk0H/4nx5S/IcTRR2SRXAPVMZ+Nnf71Dk1uoc9wfiIUsMGLGYl57lvLaJbImWhC2zn7u85FcfSn4p/FXnE1xsSFI2nYfQz+daZSVUyd+VXW1fXIeC4UOYtLjFsWBz1y4OfLy5O2oMIh750psfGs40/xjw/DTn+Fdbf/nxl8jEgC6R19JIbe/znowLJ+nkswkMn27aeuL00KiCPKqh1d8yrkajvRFH8vRXhcNldXXYS3YaqdAs3QhUQrHEEOu9mIKEW7WNpD2mrmsXTVQFx+7xxEaOmq6TRcpNggbciFGXMw3zThsPPpfxD3e9V5LWcjN7SZvi2hb0XK9DRm+yitKh0LvsiChhsqBVBU1hRc3gYriEYwaltI84PVzfN7Q7TOCyj+cQ+kQvqsqwtRxNaXszV6vIJF5MrgoXVBFfaLAMBfMSZrjY3o8+hYQaoKoToqt7xZEH+c2ELhWvwdHOopC8eXvs8zmmiVHCq2HsS9Q5eZWZngrskxC8spImncPhctCqV+YAyV3UWhbTvoFP5lk/AHD73fNwlRjD0d3s3d0WjmYibu/8EM4prrOoL1Qkqkh86zXIVOOK8JIglCUxO9FZMXg+Eb3tYDPpxwmUUHs/6EUc9+1UjD2R2IC 5lzNDScR 9KlNFQQfCITfpeHksAqrzHedLYRkjvr6789OrrynVZy06WBAjJwqU3rbrAiujvyRWz3wnELlBsG2rPpoicdqNIk536ufU/fafuyp35KXtcJlfn7T4qC0YQYNkqe6h43xO3kzVIABGoczxcVYgU2lQVaL4II36wRYzJ7I0zInEl/z0OmPm7065bHr4Q/g1a936IB5p5By1GfigEnDFDoh9Dmh21BBk7LaJqpsP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There is only one caller and the function is trivial. Inline it. Signed-off-by: Johannes Weiner Reviewed-by: Chengming Zhou Acked-by: Yosry Ahmed Reviewed-by: Nhat Pham --- mm/zswap.c | 17 ++--------------- 1 file changed, 2 insertions(+), 15 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 173f2e6657de..cf864aaa214d 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -559,19 +559,6 @@ static void zswap_entry_put(struct zswap_entry *entry) } } -/* caller must hold the tree lock */ -static struct zswap_entry *zswap_entry_find_get(struct rb_root *root, - pgoff_t offset) -{ - struct zswap_entry *entry; - - entry = zswap_rb_search(root, offset); - if (entry) - zswap_entry_get(entry); - - return entry; -} - /********************************* * shrinker functions **********************************/ @@ -1708,13 +1695,13 @@ bool zswap_load(struct folio *folio) VM_WARN_ON_ONCE(!folio_test_locked(folio)); - /* find */ spin_lock(&tree->lock); - entry = zswap_entry_find_get(&tree->rbroot, offset); + entry = zswap_rb_search(&tree->rbroot, offset); if (!entry) { spin_unlock(&tree->lock); return false; } + zswap_entry_get(entry); spin_unlock(&tree->lock); if (entry->length) From patchwork Tue Jan 30 01:36:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536669 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 936FDC48285 for ; Tue, 30 Jan 2024 01:42:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 452936B00AB; Mon, 29 Jan 2024 20:42:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 401256B00AD; Mon, 29 Jan 2024 20:42:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 27BFA6B00AE; Mon, 29 Jan 2024 20:42:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1006E6B00AB for ; Mon, 29 Jan 2024 20:42:20 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id D45521C1494 for ; Tue, 30 Jan 2024 01:42:19 +0000 (UTC) X-FDA: 81734277198.24.4B7745B Received: from mail-qk1-f179.google.com (mail-qk1-f179.google.com [209.85.222.179]) by imf08.hostedemail.com (Postfix) with ESMTP id 4EE84160002 for ; Tue, 30 Jan 2024 01:42:18 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=NwQ7LO9I; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf08.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.179 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578938; a=rsa-sha256; cv=none; b=fmcl+Vx5jzS3zkQjj8w+p80/hKLmnjjvB7xu7FLpsiIMu7bcNAwUvZ12Q61IPjy0GBKR7R txA5OaIVcxxLnHJTrflEnDv3ZCD0Tk2bZdoTgSoPdAY6qO8cTG3QRPlrP7HlXA6ErR8KTW trUKOJTJQh0JvGVtNAkBtiopVzcrngw= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=NwQ7LO9I; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf08.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.179 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578938; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2V/6lwbRMGjS57LQdA40055rq8F7L2NGNcAA5/p/KKk=; b=fHr73TKYxNppupOpG++ewKxaMvCToGaYWBfEuKJVyWiWRZRDU/YjuMdRLeKllK+7qed9B2 pT3njRJ5xJCcSjAnVw8z82iW2ODyXiu8Io2TNFxXu3dh9npVK6IeI66qvtzLDe38XJf/L5 2I20sA/x3vQZajQk15Js9r1jropCSYw= Received: by mail-qk1-f179.google.com with SMTP id af79cd13be357-783ced12f9bso236056785a.3 for ; Mon, 29 Jan 2024 17:42:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578937; x=1707183737; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2V/6lwbRMGjS57LQdA40055rq8F7L2NGNcAA5/p/KKk=; b=NwQ7LO9IyOIWVR3BVTyHlKnu79At65RNwqk8BIWjx6naFHVaSxb15AB/h6ieoMdFL0 b4NyFjChbXITgQIOOLqszgOIH2vUc1kqOJIyZxDGUZFuDZo6gf5uLR2dpXIrfbAMH9I2 9Pj/osZItMfIM9a1d2sSU4RmIQn1yBuanhc84NLVsjA6nw9Iv6ALkBksQHhQV5FvJMeN 1uzcC7hyHvFQAeDe11Ak7E+AzOj9e8gqKBEJVY+SK9SgRkiSGuU54kjmLLX+zbmxdea5 ai4JgYz6bshXDCYI/8ZApFSnAIXBJzryN5TwMa/UCnQKsCy3WHwjsC7PSqcOB/UZxlZh a34A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578937; x=1707183737; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2V/6lwbRMGjS57LQdA40055rq8F7L2NGNcAA5/p/KKk=; b=fp0pP1daSvLjul1BOSTDJgMEPArc+q9BMjLphPcLX/tehwhLTDekonigWT+hMIKazG pN6vHaIyZIbHljhQVHf+YhRqrn6wi40kLWNl9IbKgqhP8d6X20XOoGdDToZQ9bLfV3U2 QstJuuPod1tafRMXE+UmYtJE6JKNMSOMkgZHJRRSqOLvy1e5xSGcxzbIzsVYaSyvdffw pFgP9xTlktyXD2DW+IdoIZa1+WD1Ee+P/nZi4yXOr2YP++xfUrvL0lZKr0f81bsjkYJ+ XNcw8ABE1FPvIkaXMj7VodBBLviR90A/amlFL8oPcIoHPKEjaU0qBX6eN1cLBSXvlsc9 olXQ== X-Gm-Message-State: AOJu0YwpV3OF7UfWnz5/QJ6afa+49GHIdsfGGh6bzPunbC515OGcNsKR NI3noLvjL5lFuSAb5lKZsvusE7g4Nt8OqblVkygnV4aiewdSC/fYlc0hOlBC0UU= X-Google-Smtp-Source: AGHT+IFtbmXej4OpvwtpOTmn5+j0+Crv33a0WDVXCIw0rfc+AKtf6Z9PD/0SvdYRbq+4bF63apG45Q== X-Received: by 2002:ad4:4ea7:0:b0:680:815a:44d8 with SMTP id ed7-20020ad44ea7000000b00680815a44d8mr7696476qvb.130.1706578937523; Mon, 29 Jan 2024 17:42:17 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id 11-20020a0562140dcb00b0068c5116af11sm1223091qvt.142.2024.01.29.17.42.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:17 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 03/20] mm: zswap: move zswap_invalidate_entry() to related functions Date: Mon, 29 Jan 2024 20:36:39 -0500 Message-ID: <20240130014208.565554-4-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 4EE84160002 X-Stat-Signature: mk358apdg7ki47uqmrqxj19c3hjapczi X-HE-Tag: 1706578938-248771 X-HE-Meta: U2FsdGVkX19pDq7qF3I+W/xOlBifrVBgumfM+vYMQXyjog14cQ4E3jB9Xg3Kqby7tuvBeRXQTcjb7ygfdfIbMpqgDhOVZaN52mgAJ4yW9igO6DwX+Rm8YxWYYb+sSaVade6NfRcFiRwxCKljuCOoT8iHkQEWXK//4uIaZ/8oyuTlDW8uksC9DIoBZ/AEw4y60DxKc+WAoFVU7+mGdxSTknjQsPBbuKqfPUDQ8b8O8f0w5n6caD8PvA/FoLLDCrxfwQkWOLFLl47mTn5g4Yb19jTLRPex+MaVxOiLUESBcCZt7adsWLb7VDjpU63auDf/ariHSy9gojL39Mbky/WdOaRApUS4uTAhCIOK3u0hDbYeJvyogmkwI5wrVYl3i9RQsTgtOtBQFhdBMMaq6aGwRDfhh98Iq+bwfw7HkBaWBYZTWXXtU5Rf3p5bnOJs+ExRXGh0i50v+rgZV0AYs20NSgUpOwysvltqE1FxRpUYaoMTruWogr7z7VP6e4XvQNeIw7ObAcUseiD3fZNzeFaorH/c4DzIoSWtaous45Y97nx1ZlXixKtdufW1SJTvyWqwzvT0XNjUi5AenyrxYnZaJJYkacSAwQ3Bc2MB5BblC8R4ATD8AxV/VLl7jHzIepU4yP7Z8wAMvPGTgEv+AX8iTTWJrJpPvKh82xkxt2Hw/qqs5J/cwIA0aCV0OSQYEeSbm5bXbNleG2CURDq5Z9WpdVl8Ip/3a1iNU5fsKpdn9ZH/1etgvZeoeSTUedhr6sbk/Vxn4OPBUSuSbWu6LhjiGwzcQygtzmR+EeIWZTGiXJDsVbbi0rFA1nLIrjVEibvpYMImI6e/P9uh8/0K1pI58fi37dJG1qcpFUZSAlLQpPVH3roQxL2tNabo90stGWivK6ny8kTK7UG+XYxDep1Zqx8w+rrrhEPuvEE0iHHA8nZG8hJEc347S2i1CyZcJaPXSxV1H24nE6X1wvDIk/E 2fo6qkBp z4Vs5WgjoHW0wpPg6UL/tPLRDtqoDIfMVswGQ9Q2CLBuTFTov5QR9UiDKt/twnjBK/BVt4nmRrRCPQsOBNEaA7bS19A3suFQnmmlDmZYsIdjjgtSCUT9Otd+jNuakwnWsXCBKHnKsxbUw7Pcq/sC2yZzjFEOPxg8Z5tVarZ86Mb9Ip94WdoMLQnLnuU84bLcLa3VpF3IZKEWL/JdLywGQptSuSYKu9XevLcAF X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Move it up to the other tree and refcounting functions. Signed-off-by: Johannes Weiner Reviewed-by: Chengming Zhou Acked-by: Yosry Ahmed Reviewed-by: Nhat Pham --- mm/zswap.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index cf864aaa214d..9f05282efe3c 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -559,6 +559,18 @@ static void zswap_entry_put(struct zswap_entry *entry) } } +/* + * If the entry is still valid in the tree, drop the initial ref and remove it + * from the tree. This function must be called with an additional ref held, + * otherwise it may race with another invalidation freeing the entry. + */ +static void zswap_invalidate_entry(struct zswap_tree *tree, + struct zswap_entry *entry) +{ + if (zswap_rb_erase(&tree->rbroot, entry)) + zswap_entry_put(entry); +} + /********************************* * shrinker functions **********************************/ @@ -809,18 +821,6 @@ static struct zswap_pool *zswap_pool_find_get(char *type, char *compressor) return NULL; } -/* - * If the entry is still valid in the tree, drop the initial ref and remove it - * from the tree. This function must be called with an additional ref held, - * otherwise it may race with another invalidation freeing the entry. - */ -static void zswap_invalidate_entry(struct zswap_tree *tree, - struct zswap_entry *entry) -{ - if (zswap_rb_erase(&tree->rbroot, entry)) - zswap_entry_put(entry); -} - static enum lru_status shrink_memcg_cb(struct list_head *item, struct list_lru_one *l, spinlock_t *lock, void *arg) { From patchwork Tue Jan 30 01:36:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536670 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97DF8C47DB3 for ; Tue, 30 Jan 2024 01:42:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 84CCD6B0080; Mon, 29 Jan 2024 20:42:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7D34D6B0081; Mon, 29 Jan 2024 20:42:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5FE1B6B0082; Mon, 29 Jan 2024 20:42:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 49CC36B0080 for ; Mon, 29 Jan 2024 20:42:21 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 276B11C1491 for ; Tue, 30 Jan 2024 01:42:21 +0000 (UTC) X-FDA: 81734277282.17.49101C4 Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) by imf16.hostedemail.com (Postfix) with ESMTP id 7649A180006 for ; Tue, 30 Jan 2024 01:42:19 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=NKVuP3Pp; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf16.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.172 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578939; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sMNhWdXKrsGXbbTndUjwna9Z9KC3QLUt3pRgP5ry3e4=; b=IC0cAQonIXSYwUke6XHAozT6RY+FIBzjmXDOHjQ0ClQiKSX8uex4ttHe1VW9F0uf8hP5uN bOTFRFdqqJxK+cYIZTT2l6HFuemb8hXyl6ik66dgzWTjGsNFpPwK80Mc+Bz+9AN8eFci9u fGF0AGVZ31I9tPnTgSDHWpUkKSnA2xQ= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=NKVuP3Pp; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf16.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.172 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578939; a=rsa-sha256; cv=none; b=2rBzHNSDQfU+zWkYjjkNSXxgZyKc0PIIhMMVXpGxr3X2HA+z3Hl84bKkHQlzU6O/Up3kN/ QJS71NuuJgedL/2LoYJ++efhb33088U9sLdyznxqj9JadKLDdez4UkQpW1RTZfcQxarm6M YixvlMCltvCh1SRztA9wZ79b4JfTEJ8= Received: by mail-qk1-f172.google.com with SMTP id af79cd13be357-783d8e09a1cso241104285a.3 for ; Mon, 29 Jan 2024 17:42:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578938; x=1707183738; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sMNhWdXKrsGXbbTndUjwna9Z9KC3QLUt3pRgP5ry3e4=; b=NKVuP3PpG8uJasRVuGBCAw7LxHFZKHEx2B5lIgfbszFZWk30RVsN/mceu4eKC4a2vI zOLS9amTOWo2eGWViHrdPD320lUq0siK/jJ/d3Cl37WHH98npG5Ht+CNxasNLJC+I656 38ztnAL8vk6BEWT6HML1G6VyDd181kA7MEzzT/6KrQxlmBWuLAXRUMX1BQ9e70qEHGx3 j+zn5H+L1GbHE4Q0CREGlZl9eKejGbmtgODv7ZFmuR57LTPSyPIoA9ZD+NgHkJExA90h 9maTr1Lqi1VYeBQxNGhlviDILTKOX9AnnnY0QfAzXOHdMcZ4v2g5jRGr26gon6E4MQEe OQBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578938; x=1707183738; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sMNhWdXKrsGXbbTndUjwna9Z9KC3QLUt3pRgP5ry3e4=; b=D513K0grTLt6LJ2GgObMfig3pYvzH/ZTDNlaWIEo5yuTYfezA4gOhehbpPzK+L/+rl SJyerRDjUNbakLFmFDqPQ7ddmz0ielTNihz+PoeD8f7xllh/KhGynrnO16Z/wxaT+i7+ WqYKwQt7GW8wBYZ2kDLnDh0r6nLYUZoAiGhsHXnJQpYJkOWovqqx+tzbqtvheQkvfflq zIexIIvXuDyFh7SkkURv8qdOyVGdTV8GpVl2OTQRJKcVy+xsYDSAyya20kui7b2jBZoH pQCwkXqF1uwpl+k47EYkar5qAHUrTyJ79N+Zrnl7N6u1nb0J2R7p+AGydT3yekcFpcv0 SDew== X-Gm-Message-State: AOJu0YwtUss9vl7Qx0uHFFRgaxk/VQHGrnqwcssjYchtTqM27JKCbrXW kn27/6ksPfo48rePV1Z4viC2UfZDY05xTXJf9DJ5Grgz6WwnnyMNPkxRl+lTEgs= X-Google-Smtp-Source: AGHT+IF5MO6AEMmCAsk5IQGTGkKhQ4HvnX3tReOL/xtjiCFFl+s4iNpGpwrLVsQ3sbMAj+NPQ5HOgA== X-Received: by 2002:ac8:7f0f:0:b0:42a:aaba:73cb with SMTP id f15-20020ac87f0f000000b0042aaaba73cbmr2427380qtk.74.1706578938767; Mon, 29 Jan 2024 17:42:18 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id y19-20020a05622a005300b004181e5a724csm3357547qtw.88.2024.01.29.17.42.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:18 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 04/20] mm: zswap: warn when referencing a dead entry Date: Mon, 29 Jan 2024 20:36:40 -0500 Message-ID: <20240130014208.565554-5-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 4aw6tbop78xrzzet8fptoc8aonsx1yta X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 7649A180006 X-HE-Tag: 1706578939-864391 X-HE-Meta: U2FsdGVkX1+K0fWH1UUQUkH/SYUAvX1hq/3Q6NZyNhz6eJ9U1Zz7IG3CyqjQAa6zoCN0m0WxvOVPZFDGqR50aZU5Skck6Jq1W9paQTEC3IwbPSJMvOydc7HxUp0booCCHYk4LLRLZwhJDTiZpbBdfNn2SalCaP/jBjA3MwzdBzWOcP+01Y+rwKjncbErvDpLq7vSqPAdYLLGT5Qi458kLQmSc9wQ89q6CEzrC89W4JEVq2tzSaUuCToAxMkmiYGlsSKTjVQRZmEIfYwSk7ft7QZwwSODHlfT/F4CsTKvVuJ9cwVRC89tRc7DugsaLjn79P9P7wY5p580iKi8H8UHQsqgv2K+vzyFtokDBLX5QYB12i4OT3UxIdphAKo3wHUDr99p0n4o1oEzeQ3GkR36ugJ6j4divEUDPQgoRZXH+OypRVJ+cgWseZcvrnBZgo2jICc4FvOmSzO8CUmXTS/wk9zkTRicWZ8+9k2pxfxhPAqpNHioe/+WudZ1zhvmMxFZ3rIh0INF9q93uKgZUQ/KgYZI+JEuGYoP6Z/Ey076cbnens3hr/uvbRwCJGrp/wCGYbenUCX6BNkLC6l5Q3UonuqqzglJRj/8XUi/OSGdQ/DvyFFYfJI+Oa8MGsW4CNu5OYrpvkpuWntQWhqb+TlsmURGoS0iCM6quuYsDRA4L4URDJ+EwfkqxvoDHWsZ0CoPMs4uaGmInYPZBD23GiBOgqxnf7mR5qeOVhfffcK66OiAe7nYr7aoV65rEHMMpBKOD36Dl2+xfnhtIVlviBAhOzKnfzXy4yRhDX9lvtFl+mQFnHGBMLPPuuuh99ngWOhA0zlHgui3YI0/QWQuTlJfLHiID8vARSP7RpFHfYBzLoyH+y7uBHGJtGCSwAi+w4amG9KQX3XySZykSSM83EwK8eRX+UC6seQYxZiILnAc7zq2otxTzJV81LX1N4T66GO2iUVr5wLrdxrtWvXGQT0 k1bCIIzT IwyXgzuocf6bYJK8336FmPyMNbRf7fJBp9mFjtd3R9owRPINkTpFfbp3hTDgrlOecJP8mflFmvsrXaLv8auPqm2xh7TDtPb9dbdfjeQez2WdkO+sSeQZW+ybcvzd5Tu/+n7y/OTSDzGbqGVoDnppw8uGw2T+sUZ+lNvg9U46g2Z0PsCi1wRVC6eZDVZhedsfxU4od2vCHaXiE6YbOdoeGRXVVQ3mUofilJaSYczihBx7hrtrTMpg1/4vzsOMWhYFBz5qXDc6z52dFpssD7kJaUXqUwdk6ao3Ul4awSz65sqs49DMGg4VF5aXfj776SBNRfeVvYepISKYW3LCdJW/28ajRBAGSoCYwNp3X02kVAOuxZyV7n0CmaoqPot5z2QIKvDvZn/7HCysDRV4m2/6/iIHfzPxb4D1ySRfu7av4ZL+ScKPpn943RGneNci4oCkw6P8A0C8h+HImgurbpXRNE53xmA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Put a standard sanity check on zswap_entry_get() for UAF scenario. Signed-off-by: Johannes Weiner Reviewed-by: Chengming Zhou Acked-by: Yosry Ahmed Reviewed-by: Nhat Pham --- mm/zswap.c | 1 + 1 file changed, 1 insertion(+) diff --git a/mm/zswap.c b/mm/zswap.c index 9f05282efe3c..0c6adaf2fdb6 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -542,6 +542,7 @@ static void zswap_entry_free(struct zswap_entry *entry) /* caller must hold the tree lock */ static void zswap_entry_get(struct zswap_entry *entry) { + WARN_ON_ONCE(!entry->refcount); entry->refcount++; } From patchwork Tue Jan 30 01:36:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536671 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06D69C47DB3 for ; Tue, 30 Jan 2024 01:42:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C6DB86B00A4; Mon, 29 Jan 2024 20:42:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BF87A6B00AD; Mon, 29 Jan 2024 20:42:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A92A06B00AE; Mon, 29 Jan 2024 20:42:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9740A6B00A4 for ; Mon, 29 Jan 2024 20:42:22 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 71589A0164 for ; Tue, 30 Jan 2024 01:42:22 +0000 (UTC) X-FDA: 81734277324.08.AABC95F Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) by imf19.hostedemail.com (Postfix) with ESMTP id B06661A0002 for ; Tue, 30 Jan 2024 01:42:20 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=cDIDHpW2; spf=pass (imf19.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.172 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578940; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3noBpEoNcYkgCi0cr1quPQy/gMKhnK1FE9Og99etBLE=; b=AZyeNh9HQ5HigUiyzla+t93KOQ0BSZgPKO3Vqj+TXvZ2yJtvmLgOafGYRBVxktfR59l4E4 WIijWq9qMedOEzCJn+gho/2ZoqtDFxekdRjAO+FluDWbMP5Zo7mfZTErfqa04Ch6V1o5fx zvaEPHdcXodQPJzVE9aJRGLx8bpWvQ0= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=cDIDHpW2; spf=pass (imf19.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.172 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578940; a=rsa-sha256; cv=none; b=Aek780SOsiJQd0R7khiMxASda8GkuW7mbO4izG4uw26TAjJzabcjm0Du8aAo8LAbBd9ffH zqrZt7ubStc6MhGW/ryEkywue1HKrgKIW44onL5WTK5w95ANw0pzCm/8OLvO8wqGlwatRB MDxTIQVvedmStgPwn7+/D4Zv4hjOKtM= Received: by mail-qk1-f172.google.com with SMTP id af79cd13be357-783d4b3a00bso313573985a.3 for ; Mon, 29 Jan 2024 17:42:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578940; x=1707183740; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3noBpEoNcYkgCi0cr1quPQy/gMKhnK1FE9Og99etBLE=; b=cDIDHpW2ipSoxF1XUKo+YVyP+dpRI0YBd5rufDcu7IjIFWJP799voLn3Q1FJItXKk1 WGpPD5dsaqsK3VrFZJW9ftQaeNn1D2z/MWb4CE0yzJZypJdvzGG57p9Zic7NKK1FnkxD oHAJD1ERkTkNns3K63KWcUVIw33GVII9XUINQ3Xlg3gG7piJhG1cQEypmDb83rwp6ifI 2t+ug1REe1wXHOv+fyBRrYhpjH+H9hrkWLwH84sQ37wWbC10cJTyy5v24F4M+v00Rf8z 8+gCRHNcLnviAIljuHAYELAFmhKtHz1e6xWFlh0kozbrUVh+wTM3uPWBl6QhqSAbdmQ+ IHmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578940; x=1707183740; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3noBpEoNcYkgCi0cr1quPQy/gMKhnK1FE9Og99etBLE=; b=dkMTKmd2PASJjWRjA15JfI2l63DDTYXWer7/dyoDsUfbHAw5mEnc1j98ep3CreJXY+ O8jFWZYyGvBqtDT0GYicUhVcm8JjPPW2IDRtPxTSYt7abJp8U8k+RWWXR7/akRfp8GdJ d7uax8wfbbvD9K3pqUGETLEvmNPAvBvm7yayTYhy7WE2DUDypDUdlyPy8aoyuSYbC2l8 2sqDLF7JakZ7cHxXttbAl4bNmwIcu0rk+kqem7EfZ+MVU526EHvyFuuoUxZIcsDsCyCB 6Ax2n5IyuuyDwt8sH/Ceb7S3HuTqH+W8ffJ1JYiDMeEssoKPtFpKCCgQTMNPInPslxa/ Gwbg== X-Gm-Message-State: AOJu0Yygt21I5AQpZg47WnOHarRWsCweNIv5NHiWvjpH1lqjpX4vO+eI 4AbMxhe/DkTuBAxaGfefpIXMbbyy7ZnzPea6kNR1kJUOBamVMtXbTiHxfzKpQfM= X-Google-Smtp-Source: AGHT+IFva24g3oCWu71j33LtFcxZP97ALn5FVyB0OWjRs+hYHzvHicsl+CzPH8nqY1WBmS0gQvju2A== X-Received: by 2002:a05:620a:28d4:b0:784:79a:ce59 with SMTP id l20-20020a05620a28d400b00784079ace59mr1212077qkp.26.1706578939943; Mon, 29 Jan 2024 17:42:19 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id d23-20020a05620a205700b00783f77b968fsm1879593qka.109.2024.01.29.17.42.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:19 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 05/20] mm: zswap: clean up zswap_entry_put() Date: Mon, 29 Jan 2024 20:36:41 -0500 Message-ID: <20240130014208.565554-6-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: B06661A0002 X-Rspam-User: X-Stat-Signature: mf5ozg1w7bhm9cpz981ab73knrbchzqa X-Rspamd-Server: rspam01 X-HE-Tag: 1706578940-625475 X-HE-Meta: U2FsdGVkX19MgXKdVMydATdXF2/T8629BTKitXyi+me30rfkNOxTJqyi+5RszyTjXabZeEm7A5n6Rfeww6EtkQ2Dt1Q2zgvlQpnYTHcgwWlpbUt7S0/YSJXgR23Z9lA2tlqlbDj2E86IfpQVsGu6y69bVxSy5v+aZzSw7067iWvDxPfmiBAil5urc5J5dXekDXvEi13QBJLyNN/cTd7Q98N1N0RjTDeWTkybTx4CmhQGf3iMJ+U7HpZEaX7KxUA+c4xCqVDPaQc0FIAsGbhYipR66PSLVHza1kGBLNssYVZxgJcLK/Gtvuhb2tVBPO2DjMAY7nFPUgbJVmVtCImN4fk2K0IOO0D2vBmbn7CBmW4oSn4FOa2SCeDIkawZUkgHWwd72nHDECF800+tBG9Rad9QQ98uNWkFPkCdpbjpCBAmdolvJhvUhUfMnGGejXY0AtkWTBwjFWPGp32PyHaC1RF8I1S0um5gxS52InUHMia7QSGDzXE2tDnlTjqhjnCTu3VkP5XxcrW0Lnslu0g/2ihfc4TmTDSOWUfDnfOxr8VU6ubCkQvTUGA3pTquAiW752V6xpeb0VTnkpulsEyd5h4YW1ElWBi4AnNsNjb18c4aR/XoW4PoA7n1y+1vtflLjtHW5mBTZhVp+rbYBX7NbGNM05kk1Jm+IKoZoBE0i5adUkR5Ir9GbWz57wrNDryf28gjf2RzU9lxUBstszNDhpUCECfdanTM+GTnBpxkj7L3lcYRFEfULPHOAIUc59zJOvby/SbI2OPUbiVCSLC75+7BPt7lPStCjuYTOfu5ACVmISt9gm5xiXXjElMkOh8gwSBy7dJ3fLy8mvRZCAgxEq5qgYUSbTkSHxypmck3SQ/Cl9Bhqfn6sPik9a02fITM0uZwbh2jQRTvycf7NA9DLaoTQWpFUZ7OtgOcQL1J8GuvYoTL5ZM7TCvegCuioUsmBBkm9CcetTcuP81EEUX /Iuh/lV3 AyuDtMKHny1dBdZ8FmMvULgNKoHYdB2tROv4r/9RJQwk0u8gzaJyMZSr7CdLJ2FlkUFHSTepBHu082ZqEKk+A3kOS2EZikoRJonH2mHmjs5vGZECdZwePVRbYxVaE5lJTH2VNqtOkUVUIpbX2e3xxcoudddDUebgyN1TmbGcbVYO8Bv0pUuXh47JuJk3T1/xsZngHdVwWQtl0FtUYlqWLlcpU91VMKW6A2Md3HEx7LDbFb1C7JZNXSQs/bxBkUOAJZx1QwXAh1fJhTwlpx+Rq+YAmuhgGsEHcsvHA19kHpIwHeZbacaMTm7ek19Wd7wT8NfFGH0/NTuqojhe2uFgZXHR3+fD91LWKFvnKxEmawtkAuch/9C3mekzh6jSKRXcR02i/crqXbQtT8jNd3W6xOR3fcQCrC+YA9lFB8W1mXG5aJUWaR2T73KXfln9tfJCK0qm0+nRBtB/qQNqessq+U/jpboyDo1ShgNT008cDXiuf+iB6CQSIjeLG+ghTr9jAs6ptid0dFOEGUE0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Remove stale comment and unnecessary local variable. Signed-off-by: Johannes Weiner Reviewed-by: Chengming Zhou Acked-by: Yosry Ahmed Reviewed-by: Nhat Pham --- mm/zswap.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 0c6adaf2fdb6..7a7e8da2b4f8 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -546,15 +546,11 @@ static void zswap_entry_get(struct zswap_entry *entry) entry->refcount++; } -/* caller must hold the tree lock -* remove from the tree and free it, if nobody reference the entry -*/ +/* caller must hold the tree lock */ static void zswap_entry_put(struct zswap_entry *entry) { - int refcount = --entry->refcount; - - WARN_ON_ONCE(refcount < 0); - if (refcount == 0) { + WARN_ON_ONCE(!entry->refcount); + if (--entry->refcount == 0) { WARN_ON_ONCE(!RB_EMPTY_NODE(&entry->rbnode)); zswap_entry_free(entry); } From patchwork Tue Jan 30 01:36:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536672 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82069C47DB3 for ; Tue, 30 Jan 2024 01:42:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F01986B00AD; Mon, 29 Jan 2024 20:42:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E8A816B00AE; Mon, 29 Jan 2024 20:42:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CDFFA6B00AF; Mon, 29 Jan 2024 20:42:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B91316B00AD for ; Mon, 29 Jan 2024 20:42:23 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 99A631203BB for ; Tue, 30 Jan 2024 01:42:23 +0000 (UTC) X-FDA: 81734277366.04.2BFA7AF Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) by imf21.hostedemail.com (Postfix) with ESMTP id CE1E21C000F for ; Tue, 30 Jan 2024 01:42:21 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=1d3s+uIn; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf21.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.176 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578941; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CQR6B9VdWNRnJssZsaOhbSU079VyNfF2R3mXiYKEIjI=; b=TirOgV7/q2zVhrItxtRJLthX46bgnnBX3GYy2IQ2m4loA8GbSZ74KJL21bLdvLsEf7Uj7w luhDqUP5y8HdAKhwess15ST9YsA1RMwEv8RtvzaiquN97H5vujWHOEJynloQQD0d27YkzO /ccUoq1a4MqRiAn8R3ACnjPz2Erx1H8= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=1d3s+uIn; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf21.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.176 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578941; a=rsa-sha256; cv=none; b=nlsMiM5mgS3TQoBOWhhCKnGipVCFg4GhS2/PZdwVtUrA7ccMQLypKbvfPxJK5li6pzrEAw BqlN06Xigl8DjjmnSmuNthihDHR4UgTdyM28+MqaFs/rB2Kw4vXK+Xtx9pKnSfmu/SlZOc 0rdk3O+/5nzuPziRGPiv3JuYG6gSFek= Received: by mail-qt1-f176.google.com with SMTP id d75a77b69052e-42ab4f89bdbso2145991cf.0 for ; Mon, 29 Jan 2024 17:42:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578941; x=1707183741; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CQR6B9VdWNRnJssZsaOhbSU079VyNfF2R3mXiYKEIjI=; b=1d3s+uIn+/YajlyLCOP41HfCRMCugJEo/Pcsk4eV+55v8nPTLN2H2/d8V6crDr0P+m 0abPMCLlqRq51Zw8AnK4diXH50d1KN1bE+JTT/iFWxxKnJEzmZ8agIEfAkbuVuYhsku6 w0a3RGZ3uiklIyezowS/kX3tdyKUE72nzwaM2+Yjnrem8PoOzNRs2c4FNcRF39CwW4xH gvpWXMVzu7db+qfGra54Q4TybAZC0bpILATiaoPffjiFOztxUrn8gzBa71aBcw8irioE 6oe+hKEzdhodvR1os0BpI4YwVKJAtswWwHeM+zTSi5Wg5gN/thnNrdx2Kpbey3fc14dL 4Stw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578941; x=1707183741; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CQR6B9VdWNRnJssZsaOhbSU079VyNfF2R3mXiYKEIjI=; b=Ts3L7GGDWEcq332AtpwHjA+iROCO2qbYa1xEClSqEmgf844e34+gGSZAj/DLXfk/x5 dgi/qkgXnZoiP0KCkf32TFx8EDK0HF0sG6/XHaaBvJZXDR9ugHl2YAaOJiymij4G1dRp uPMzYRf64agC0dAdTWh6O9BV/mULG9qt1z8LPIuYeLE4NNW5+l5q764GCnnrGjrDc090 L0IxZ+Wu6iYVn1iv8RiUVjSo5G0KlGEKBPqyc+XRKmvjXRyeIb0I0+QCQrww6fQK8x6g 7o06byUyJlHeZcnNP1TeZMdbZwcd0m+pWyUD+opNaNxzHhzR0i0SY57oCeIo/FJxoHWH sVzw== X-Gm-Message-State: AOJu0YzfY1RTo7cc9Mar8YxI6jyOQIIAjJo07jx9tej76+MZotJBChq3 FpYnnotFsnWjED3Cf8EEsOjLCAPK5z+1xx88N88iyJt3uT41D3wGAFNpkH9tkDE= X-Google-Smtp-Source: AGHT+IErWm9kcg1l0jaG/xKO+cFOv3Pm8kp13SmDYTiZeddMh6ZfVthab4Aji7BCAsCUkY1gLrj0kw== X-Received: by 2002:a05:622a:40d:b0:42a:b079:e665 with SMTP id n13-20020a05622a040d00b0042ab079e665mr1722772qtx.12.1706578941087; Mon, 29 Jan 2024 17:42:21 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id bw21-20020a05622a099500b00427f8c50f31sm4145530qtb.46.2024.01.29.17.42.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:20 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 06/20] mm: zswap: rename __zswap_load() to zswap_decompress() Date: Mon, 29 Jan 2024 20:36:42 -0500 Message-ID: <20240130014208.565554-7-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: CE1E21C000F X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: yyihrbjhriib9k5ic3ek68ipi1fo6gc3 X-HE-Tag: 1706578941-328171 X-HE-Meta: U2FsdGVkX19oKhsi7Z7Qmn7YJQtyEpBaPc3PqRkadDg/mAxkohkfTNfTMsesV11/e90z7xXmNUp8vMLHoTc3SYoAFbwYlFrIzljiBzrlY4hN0JHl8U0cM5nVzqMylPbMiZfrKifHU41dSSRjsdJsKdrfwIK7gZfVsNTqzVooaa8NnECRrufVmFB7Tu710YJkq1LtfwaHo7V4OjRhu6xjrKFa1BvnJ2w0n+Mw1eBXWNjHF7ykalX04Aq5IU8zvIFzgtZurhGHKoa5SQEVwvyMhjEpZv9ARSxEIQ8L5PsE1sMeA5dVlWVI+7cZfUg3QQ27hVZStsx3raEfB3S5nKRXwX4kdTCxQXD2ziFklVQLWNEIbpIpeYl+QiWRXGg5qwQIRK1T9aCZnpXFc+7S3/JSbpHIKNRq+6Wn9Y7r9IbI6OMGR2yiiG+52kwNPd6ajpTv8w0cLnI9fCrhWrnWhTCND53psd/xcDN960EcMm41HPYv4JZDxO/hRn9gQ5d5cMJdTUMJqptrPRE4EDjgpLYab9hWFdD+9NqBVRZJCnMWjONJUxSjgr238Gxy2Boc9Z810Vz5xKHi1LgQ9VDAD/dghcmdCqk01upKqfSh/4RdQxOoePstkMb5NCbxrVmq4mJbU9EKRExei0telXKjI+mnIDAc+udXwa2UFhord6UyKho7L6xUrNXM99qiMOsIhJKxqN407Cb5R0WjX1uLewT3zkHwD+A7kwSLojtQmBn1r0+BV9xte53kuk1A73mMQP7D3klE4dty8Ifjxgrt/FiprLjP118VFoO4r3hMD9wEXDxkC2nYiBf1WavYZv8pqjbVNk+OtmIFpOXsOu6j7R0SwrXH5m9L98oVum2ZNBxjAXiTKxbim/9dFMF3wPjRK1bYeKLIf7LCwOaWg6qT5iBW+2zILy4FK6JLuVQbtTaJwJaHqQSh78QBHPYLHSP9Y0Y4S4TAiasrxGKIdFb5m5E B7uJeoOd pphbpUesHG5rHn3vWOdHc7idQrr3SNzE6mjeOQC+pK8CrBdBT5gykXPUPbVTd7kJvbIKCOCVDpRTacdVQatTWxB/O94Vl8SWTK24uiDlDu+s/0kx/4IZlQpwaheTd8PfTy3tcwCQs3ZicdiA/MNZAJJAFHqaOkBT4jz0KukNE7sDbB7y/M1rp52eeBrqecHqilVXKjoxAsLd6bFP0tTMXuUzW6/bU/ebgjRUNlKt1DFY8pDtcZbpMAb/nthCDNagQXvS53yiBGb2/ugT+YwgngDG1RjXpHdgv6zU8mIbhgZhvvFZO92peq9R8Hv0P3c4ha0TrawxJt0uAhLp6oCpd29paw/HGNd0Ctv37ElxwXgWn/N+070CdqsnoAMapUXHmvsEiYHHsGC1e136g5qB8aahJOI+4h4Cry2d3pOM1Wxpq40mPKpP8SGuotpV2wi2O4NpqqRdmFolyGRoEPOqh4XNWzUe1/NWT68GGYnUysNp8At8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Signed-off-by: Johannes Weiner Reviewed-by: Chengming Zhou Acked-by: Yosry Ahmed Reviewed-by: Nhat Pham --- mm/zswap.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 7a7e8da2b4f8..bdc9f82fe4b9 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1316,7 +1316,7 @@ static int zswap_enabled_param_set(const char *val, return ret; } -static void __zswap_load(struct zswap_entry *entry, struct page *page) +static void zswap_decompress(struct zswap_entry *entry, struct page *page) { struct zpool *zpool = zswap_find_zpool(entry); struct scatterlist input, output; @@ -1411,7 +1411,7 @@ static int zswap_writeback_entry(struct zswap_entry *entry, zswap_entry_get(entry); spin_unlock(&tree->lock); - __zswap_load(entry, &folio->page); + zswap_decompress(entry, &folio->page); count_vm_event(ZSWPWB); if (entry->objcg) @@ -1702,7 +1702,7 @@ bool zswap_load(struct folio *folio) spin_unlock(&tree->lock); if (entry->length) - __zswap_load(entry, page); + zswap_decompress(entry, page); else { dst = kmap_local_page(page); zswap_fill_page(dst, entry->value); From patchwork Tue Jan 30 01:36:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37D29C47DB3 for ; Tue, 30 Jan 2024 01:42:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4CEBA6B00AE; Mon, 29 Jan 2024 20:42:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4515B6B00B0; Mon, 29 Jan 2024 20:42:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2A3DE6B00B1; Mon, 29 Jan 2024 20:42:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 167C56B00AE for ; Mon, 29 Jan 2024 20:42:25 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id EF62C802DE for ; Tue, 30 Jan 2024 01:42:24 +0000 (UTC) X-FDA: 81734277408.23.E1AFC6A Received: from mail-qk1-f180.google.com (mail-qk1-f180.google.com [209.85.222.180]) by imf08.hostedemail.com (Postfix) with ESMTP id 37E1E160002 for ; Tue, 30 Jan 2024 01:42:23 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=in4cnkkw; spf=pass (imf08.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.180 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578943; a=rsa-sha256; cv=none; b=cI+ZfYvQnnpTJWwsGiMg+3Gg9Exbv0qcXgUO2r8B4S38UiUQnVY3wK/tue9Ag7OHyMJKx6 c6GJaRGcHomTIoauGc3HIoI4j12tcND5OP56XUJOsFFjz1zhhaWooJvykR6Q9vI075njcS RaKwnswC30P70voges60Pixd6nvL/iI= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=in4cnkkw; spf=pass (imf08.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.180 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578943; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mpwOGEgV3dvwZoaILLJtRsR55d6E5hoMpUKMvxq3d70=; b=jFsAqW2xfRe5TiuXcm32fKDuqphCliUQ3oWGW203NGk0te3Q6mnUxpEOrjYXm2kEiAO3QM vD62lgG36ZF2W8zkTCtEmyrM/QJoaSqqpiQFQg8K7FUc4EASwUI4zLddc21hCUA4ORxprm IPOs/3Nm/Qt2Woy7eMQ/nqowTgls/6w= Received: by mail-qk1-f180.google.com with SMTP id af79cd13be357-783da26489aso280792685a.3 for ; Mon, 29 Jan 2024 17:42:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578942; x=1707183742; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mpwOGEgV3dvwZoaILLJtRsR55d6E5hoMpUKMvxq3d70=; b=in4cnkkwLNOR5rM549JsNbueXvdxSGPdERc4QAfaBzUqjM98P0A4GF/vCq06vLg6yA IXyk92SBRK8JQm+pHQRnBKMKF9fOGXXQGtGmTZjACm0t7lR73wgWLpsdy/UkmFQ4hzV0 dnIS5mDhZw6HK/FJUUx6tNKWZ+N10sK5lOnB5fDmYDpAGAhPZyMdup91gHztacS3JjUR jrOtV2/pzUpZsLh4cmHv1SvxL2P2PNt4yvUW/YtrGPvEH7QIYDus/zsCoSofCboGocwA pglNa9cxDNmYLMfU18YEzR575TUWjLfd8BiUfcCkKzYrsqTgtTT1b4di7vgWla32ch0K 3c5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578942; x=1707183742; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mpwOGEgV3dvwZoaILLJtRsR55d6E5hoMpUKMvxq3d70=; b=g2dXfg3eOlxo3QBZMSXjOI4uljaeE6+clL5ERWVVV717cA/uk7f7Pm0lXWnL1mSDA/ 6goIDcM0SBUaiujOZ+yLr35r+QRyG8eDvQcdyOIq50JHQyCg2TiUBgBrTEu2P8M/h8Lg ntdB/WjhkE5sGyT8q5ew4DgLsLG0PqGV47pt7EXzqDse4vJvm5u31jQlIwP3n3iIAxXv fUrvqztJYi8+wP85UE7EfhL4gyNCOi5X0xyGHDCbBDaGFtpTxIYPjzgpdN4QNnLJiIus hMLwZvK7jhgmqkTgW9bFY2mkPcmidnvwbHRxCAYbLwsGHpygsNOVyYQ7JQxZq6HmrVIl FMgQ== X-Gm-Message-State: AOJu0Yz6FMCobsGnkD4J6BtGmLSIQ/cKEwZ/EeV+NxmkUyX3hUCLUq4g WNgpWEUJh0Ea4UIZxbhkIpKp7NVp1unIZTCOPzBX66m15jfvrw6W8+jpZIO1HxU= X-Google-Smtp-Source: AGHT+IGAaMHNgHtWkFaPNS95tGZ/hwePQLFIY+O4EOkM3Bj1xV6augPRQXjZ/4cnW0lftD5scOiZPw== X-Received: by 2002:a05:620a:b11:b0:783:d096:d761 with SMTP id t17-20020a05620a0b1100b00783d096d761mr5637186qkg.31.1706578942391; Mon, 29 Jan 2024 17:42:22 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id po30-20020a05620a385e00b00783e3b030e7sm2669196qkn.33.2024.01.29.17.42.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:21 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 07/20] mm: zswap: break out zwap_compress() Date: Mon, 29 Jan 2024 20:36:43 -0500 Message-ID: <20240130014208.565554-8-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 37E1E160002 X-Stat-Signature: 36xmrphegmqtfxzded6dmjtej3iirei5 X-Rspam-User: X-HE-Tag: 1706578943-92456 X-HE-Meta: U2FsdGVkX1/8UHMogSHtlH1aEHfkFVc33yOljSCxI86ip0Zzpjvolt6o5hifMb84ybdVlPA5bz6wubyuKAtUMAx9UZKh+7tccMswdlPL9AOSmbcmn7sR51TooOJrPgWjWpokxbgUpzlHAvjwc/ypvBvqnCNEIg3bTC5+nq2pTxOjlR/jmnOmdjR56jx4mK2NDGcC4GU2uuuDy3R02aCtbZrafHAjqhTKg287iGaozhFgHN+OwDIHWv2bsycDiNwZ3kOXYhnoKneMdJnE7LCYxbzl0zrU09XboJhQjx/8m3SjfiPt+jLamgS4d3FBBnZAigWaoBN6nBY+hCXl3cuLkyqq6I/MSoOsIKHEEr17xer9d+Rm0mY8FxnJpiY1Vw+fDN2Gd2zttdEi62qIYH1s6XgppcJdR7IGu+Q2meCJcBfRP2pWV+7rzW4h+EVTzZ9xaVrUFbOpJBnm3S/zaD3hxinZsG7w20oMOIXZONMp9bnRcxZihIIT9v0xa3ymO8cHJl2b3sQNYImh1bL0NKGy3iQh+laB99qfv2j/p59+KVRELJfy1toLpI4pIEl64fQOXqxYIYdD6GAQl69OzkTvtyGRcFImxUCMpjDBH9faAza53aWJXnoKF6LUcfwok4dwwSlNf5I1LBsPw2EZOWDf86NL7oTx4s+80+CjyyZfWmo7XVDbXLl/goIa5cXE5Ws/KBJIUFqUNqeBQdi4udKsfMHrVtCbDCAkrMwI34WHX7vDgd7qYvK5ZPfO5zgfGfHL4gy2yDH0UUSMaO8lyGrH4FnNpwn7JtkWc/6Ho3XGWAFZ20GZIliwoghjqFFzksMQG2TwIU1JUGLLwMJNBK3kSZeZOI0g8OZdwSNv2wYf2L4nmM7cHVfRbHrea3/UfeD/uGpnIlrKBgL4roB5zlNVbIbMuEi5/XjYSjc1qzJ4rSlSdk8zyMn9VaoAaxZOhHTyOkWFAUvt8U2Q3zKhnrD /RvL3Pht l0XSIbFSuJBA7MYfcziKQy0bOP+C15+T5anPm5/e/4fML8iQQgG4AyjzgNVbKVmlDaA/XQ60YeTQr98dPuqMMoB4prEhO90S1O8Cc3tyOjdUuPtT4DKhYc5+DRyod+AEbMzdSEG+VAhksFQUA6IUaLZoBTZ6FDOPrukSWhJ/LjTvjgQMj/LhUHZ280SXEamCwJKeKWPvGNGOVtLTJv14WHwIsEokDxMQhayFeB5QgNlD/jTOYJKv1s3KG9QqXENNEkFraKSilXiqIVnD0oylap00o/ZHF9XNATuZnWIUUcFUwTfdFKVmNFrh3alqGG2UPsyJ5B1/y6ODZytDbbxqFH8Zq7htS3RcXFvFk3qXPt0TZWngOeZQ36zBiIF/qWAMVHK87E6+Z41r82Ia8ZooK65EHSmn2eopxCg6qDCwLnUE7oHaT9llKPYUOExR5Y3FT6ppU49mRnV8YbUpw6Loofu0t9Inp5VY84VMGUecsBZ2juPc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: zswap_store() is long and mixes work at the zswap layer with work at the backend and compression layer. Move compression & backend work to zswap_compress(), mirroring zswap_decompress(). Signed-off-by: Johannes Weiner Reviewed-by: Chengming Zhou Acked-by: Yosry Ahmed Reviewed-by: Nhat Pham --- mm/zswap.c | 145 ++++++++++++++++++++++++++++------------------------- 1 file changed, 77 insertions(+), 68 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index bdc9f82fe4b9..f9b9494156ba 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1316,6 +1316,79 @@ static int zswap_enabled_param_set(const char *val, return ret; } +static bool zswap_compress(struct folio *folio, struct zswap_entry *entry) +{ + struct crypto_acomp_ctx *acomp_ctx; + struct scatterlist input, output; + unsigned int dlen = PAGE_SIZE; + unsigned long handle; + struct zpool *zpool; + char *buf; + gfp_t gfp; + int ret; + u8 *dst; + + acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); + + mutex_lock(&acomp_ctx->mutex); + + dst = acomp_ctx->buffer; + sg_init_table(&input, 1); + sg_set_page(&input, &folio->page, PAGE_SIZE, 0); + + /* + * We need PAGE_SIZE * 2 here since there maybe over-compression case, + * and hardware-accelerators may won't check the dst buffer size, so + * giving the dst buffer with enough length to avoid buffer overflow. + */ + sg_init_one(&output, dst, PAGE_SIZE * 2); + acomp_request_set_params(acomp_ctx->req, &input, &output, PAGE_SIZE, dlen); + + /* + * it maybe looks a little bit silly that we send an asynchronous request, + * then wait for its completion synchronously. This makes the process look + * synchronous in fact. + * Theoretically, acomp supports users send multiple acomp requests in one + * acomp instance, then get those requests done simultaneously. but in this + * case, zswap actually does store and load page by page, there is no + * existing method to send the second page before the first page is done + * in one thread doing zwap. + * but in different threads running on different cpu, we have different + * acomp instance, so multiple threads can do (de)compression in parallel. + */ + ret = crypto_wait_req(crypto_acomp_compress(acomp_ctx->req), &acomp_ctx->wait); + dlen = acomp_ctx->req->dlen; + if (ret) { + zswap_reject_compress_fail++; + goto unlock; + } + + zpool = zswap_find_zpool(entry); + gfp = __GFP_NORETRY | __GFP_NOWARN | __GFP_KSWAPD_RECLAIM; + if (zpool_malloc_support_movable(zpool)) + gfp |= __GFP_HIGHMEM | __GFP_MOVABLE; + ret = zpool_malloc(zpool, dlen, gfp, &handle); + if (ret == -ENOSPC) { + zswap_reject_compress_poor++; + goto unlock; + } + if (ret) { + zswap_reject_alloc_fail++; + goto unlock; + } + + buf = zpool_map_handle(zpool, handle, ZPOOL_MM_WO); + memcpy(buf, dst, dlen); + zpool_unmap_handle(zpool, handle); + + entry->handle = handle; + entry->length = dlen; + +unlock: + mutex_unlock(&acomp_ctx->mutex); + return ret == 0; +} + static void zswap_decompress(struct zswap_entry *entry, struct page *page) { struct zpool *zpool = zswap_find_zpool(entry); @@ -1472,18 +1545,11 @@ bool zswap_store(struct folio *folio) struct page *page = &folio->page; struct zswap_tree *tree = swap_zswap_tree(swp); struct zswap_entry *entry, *dupentry; - struct scatterlist input, output; - struct crypto_acomp_ctx *acomp_ctx; struct obj_cgroup *objcg = NULL; struct mem_cgroup *memcg = NULL; struct zswap_pool *pool; - struct zpool *zpool; - unsigned int dlen = PAGE_SIZE; - unsigned long handle, value; - char *buf; - u8 *src, *dst; - gfp_t gfp; - int ret; + unsigned long value; + u8 *src; VM_WARN_ON_ONCE(!folio_test_locked(folio)); VM_WARN_ON_ONCE(!folio_test_swapcache(folio)); @@ -1568,65 +1634,10 @@ bool zswap_store(struct folio *folio) mem_cgroup_put(memcg); } - /* compress */ - acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); - - mutex_lock(&acomp_ctx->mutex); - - dst = acomp_ctx->buffer; - sg_init_table(&input, 1); - sg_set_page(&input, &folio->page, PAGE_SIZE, 0); + if (!zswap_compress(folio, entry)) + goto put_pool; - /* - * We need PAGE_SIZE * 2 here since there maybe over-compression case, - * and hardware-accelerators may won't check the dst buffer size, so - * giving the dst buffer with enough length to avoid buffer overflow. - */ - sg_init_one(&output, dst, PAGE_SIZE * 2); - acomp_request_set_params(acomp_ctx->req, &input, &output, PAGE_SIZE, dlen); - /* - * it maybe looks a little bit silly that we send an asynchronous request, - * then wait for its completion synchronously. This makes the process look - * synchronous in fact. - * Theoretically, acomp supports users send multiple acomp requests in one - * acomp instance, then get those requests done simultaneously. but in this - * case, zswap actually does store and load page by page, there is no - * existing method to send the second page before the first page is done - * in one thread doing zwap. - * but in different threads running on different cpu, we have different - * acomp instance, so multiple threads can do (de)compression in parallel. - */ - ret = crypto_wait_req(crypto_acomp_compress(acomp_ctx->req), &acomp_ctx->wait); - dlen = acomp_ctx->req->dlen; - - if (ret) { - zswap_reject_compress_fail++; - goto put_dstmem; - } - - /* store */ - zpool = zswap_find_zpool(entry); - gfp = __GFP_NORETRY | __GFP_NOWARN | __GFP_KSWAPD_RECLAIM; - if (zpool_malloc_support_movable(zpool)) - gfp |= __GFP_HIGHMEM | __GFP_MOVABLE; - ret = zpool_malloc(zpool, dlen, gfp, &handle); - if (ret == -ENOSPC) { - zswap_reject_compress_poor++; - goto put_dstmem; - } - if (ret) { - zswap_reject_alloc_fail++; - goto put_dstmem; - } - buf = zpool_map_handle(zpool, handle, ZPOOL_MM_WO); - memcpy(buf, dst, dlen); - zpool_unmap_handle(zpool, handle); - mutex_unlock(&acomp_ctx->mutex); - - /* populate entry */ entry->swpentry = swp; - entry->handle = handle; - entry->length = dlen; insert_entry: entry->objcg = objcg; @@ -1663,8 +1674,6 @@ bool zswap_store(struct folio *folio) return true; -put_dstmem: - mutex_unlock(&acomp_ctx->mutex); put_pool: zswap_pool_put(entry->pool); freepage: From patchwork Tue Jan 30 01:36:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536674 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AF79C47DA9 for ; Tue, 30 Jan 2024 01:42:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 605296B00B0; Mon, 29 Jan 2024 20:42:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5B5896B00B2; Mon, 29 Jan 2024 20:42:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 42C1C6B00B3; Mon, 29 Jan 2024 20:42:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 2FB926B00B0 for ; Mon, 29 Jan 2024 20:42:26 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 00CE514065B for ; Tue, 30 Jan 2024 01:42:26 +0000 (UTC) X-FDA: 81734277492.17.072DC42 Received: from mail-qv1-f48.google.com (mail-qv1-f48.google.com [209.85.219.48]) by imf28.hostedemail.com (Postfix) with ESMTP id 46597C0012 for ; Tue, 30 Jan 2024 01:42:24 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=VkirtGcu; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf28.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.48 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578944; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mALgLaCiaIzpkji8oSjATmvxJWj+cS9YMUqWO4u+ZHc=; b=pZ+FnK1AvdgYTqGONxzaajdwAEaDRTjPE389NfEue2lp6xCxZdfvfFM/sJqrhFmJh9dCqf D6pfZR84qawHANFYhSK6ibv/d4iNGd5vRXTAnCEq9KUG877K0lcU5jdXh1P+KN8CDnnDy8 D/FfHa/LUtpnnQ+FTuol25HR0twLEV8= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=VkirtGcu; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf28.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.48 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578944; a=rsa-sha256; cv=none; b=ttL5N5ues+yNcDmIxycA14htZVbmxJpmcSBth92cm3iBasVN11Rc/SEfJLWP7kTHzuPlzR ipCN0NzGwbFuoYAQsX4anF0BqpMVI/K3050jxoixcsRrBrOOY0/U/Is6BEtOLjn5xTasJb Lmkf4jT6rdUThStyhLx45kGyX0XPBdo= Received: by mail-qv1-f48.google.com with SMTP id 6a1803df08f44-6818aa08a33so25063496d6.0 for ; Mon, 29 Jan 2024 17:42:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578943; x=1707183743; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mALgLaCiaIzpkji8oSjATmvxJWj+cS9YMUqWO4u+ZHc=; b=VkirtGcuXNWJzb5ATQX8e/rufmJpvEaGfep0CwT6owOigDKgDrsxfMHLKcyOi/CMsK KgTLk0WS9NZ7FANxrAFDtr7qelLJEWklVSTVMozPuQpauS9RYpJlykcXPwB7SnamEgUE nz+hWf49ANkVzwd2a83rVSngfcDoSGqTyVPnKAZfB6NTgoGaUaQ0MV/2V1kpy2Q/1aYZ fr0wKIvZvwrphzZJfR7X3ZYV7qdkbKqi+PyMzjbkjo1gtH2RdOXnJbN3utZ3paZJdGQt NQigX8VzdIdaVch1uatoc6b5VCtcCjK4wIbhWvA9ZLOwb29XlRIseZSEflboYWm1xoaH tfPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578943; x=1707183743; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mALgLaCiaIzpkji8oSjATmvxJWj+cS9YMUqWO4u+ZHc=; b=aRV8PCIquwok5riWYR6f8nOHw2wc+mnJIZvbZzlmMfg3wtxHvLBySRkMMNmPuWKl6y Ojm/a3hEND8fBoJQ0R/IxP+ZZkUfu8z8krYzjzL6MGef+0bYVhaEGIfre29Jmb3+lz+M xLjuewNhYpj9QqqVdNay1Yv9D9mlkI9Ux6E2v2l1m7r7WnDO2n7LJffYlGeCPpcPswoy XeI+WWMPG55AjzpfS2wfJfi4awDnM3pv/LatOHiuZqCz76sv1ElsqVd14+TcS8+i7y97 MjanuB4o9dRf7BMS78lCKH4EuMv6M6aKBtJGJBHElhE7PlrXF3EyiJCDiqClDVxSkO6E w7qQ== X-Gm-Message-State: AOJu0YxC44QIY4KlFlMmekpiMxlm2EdzQsR4rr22N6nPxgp5PetVDqyw J02nCb7xlv0zL2Eeur4vepTkU63qmE4GMayQ+Nkr7UIUnoOFQgTeAYOZeI9R3NQ= X-Google-Smtp-Source: AGHT+IGoErOj2bpl9IZ9knv+7JV64kZmN0sJ9QZ4US8bA45j2Rdf3yjU12SQ8udCU2m6baBLuq016A== X-Received: by 2002:ad4:5aa5:0:b0:68c:5c8a:dc7b with SMTP id u5-20020ad45aa5000000b0068c5c8adc7bmr396936qvg.13.1706578943526; Mon, 29 Jan 2024 17:42:23 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id pc2-20020a056214488200b0068c4aefa96bsm1889828qvb.69.2024.01.29.17.42.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:23 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 08/20] mm: zswap: further cleanup zswap_store() Date: Mon, 29 Jan 2024 20:36:44 -0500 Message-ID: <20240130014208.565554-9-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 46597C0012 X-Stat-Signature: np15b1qgiyjmj9iascwue8t4xnyjessx X-Rspam-User: X-HE-Tag: 1706578944-315449 X-HE-Meta: U2FsdGVkX1/FxgzrCwTFTBigA7WuTmMmuJ93N0OuKR20LJvuaSelKfhwphqT63giEDILA39oDRt4Ly6/olud5JFpCFl9EwrpPSMYXqqVsvatqLwATn3xnqeEzCaeL9V6n0jZm3LUz+6SI7f78AgfVSDy7WvzHemhLce0Qm0vrDfhdnlE0RtJoqAGRsiGk/bzxlKvLrWXJciz46FdWk4TL6P62O62vUmfnxwTWc9uXDt5/Eu467hWgd7cDXdnNTbZUdwiouLzF/aDMyixc3BoMnokagL21CTCE8f+ZlWyigVFW+mfI3ApKwnE2YHXiZhaeKGkUQGxzeCdmR3IFFcw9HlQhphoKfofWf26A1Fb2WvJpZEQxcfk5vwOcd7lQoVWR8z3sT9xcNUG64jWVag4cyUiq42r7zOj+z28e0gZ/olhJl215Skq51vzbpwojWEnzcVZ1ZEXA5HfJStyOszTrPXCEBG133HkSOr1MrKr33O6dE9MFVXD4IJJkoLvNoTT8ys4ewsTL4s/zwB0V1LCGMtfiLLZiGinLp4XfIkiO2d9QzCUATjRZe/3kOYdnjCUM1dJg16fVn9NarNY2iNni8tNPNostBYDuX870M8mPNabD7FCAosGuWXpzKDve8RkZIpGB3YZhOAGxfMU7/PyKSA361VOrS4GjYDrzuuVOX/vYXoM24M6jvn/UylnVfoQacyN5Uyt6DqbHWT1m44BrEETuag8ApD2GQkpkVFYftn+nmdfLrY88d9H/cif04XMpdexkyMTMWBC5JmotNjFPzl09akBr2Knn78G21X4L/XeJO0judu2Wbs4E6x+JZhCqIJnefkVReI/3Z/UmMpMy/5Qieajvd3D3U/NFTORmnibxA1h8nGN8Bc5t7eGBktGmolXuiwuVun7Xnqx+wcxf2YvS2sf7xoz5UvkhsnTNxlDK86elNn5Gdt7spRu9mDfK1a+4hS+FN/Bzqgqf0w pIwXYJqF ziiPWwFXDEp0jdBmBHxNjfh1DMxIldF24Bkt+Wz2SZCgEmhBCI4Iv3nPA5GJ/h0XCfD9JWHrzt1vtVmj7KocOyiXeJ7CYGLuVTQqVxk0WV7apy774IX5f7DYX3bQDu29AMe7sF8B15IRsJeBbfMBEZcfjIxARk0PBR4P3VuytH+MNtKzNTSfhRe4/vbtC1nu9jfsWrrUXVcwFiLrEelnjtVhj6MrgP188SQW6gJSsDNRoV00PSB1Ry2XF1ht7g9lV4P8AG6ti/z1m2jFNeH1NGkIaVBChZKi2f8u098xSiBCNuXi3lfh65wU77t/RZE8Xjt7za5g3MCr8PuQz+W5n/ZfXbbtnHXFwtEyGpgRItqSRQXX5owWEGrSroCkCVaNiq0WPdoxFrpxxw0I= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: - Remove dupentry, reusing entry works just fine. - Rename pool to shrink_pool, as this one actually is confusing. - Remove page, use folio_nid() and kmap_local_folio() directly. - Set entry->swpentry in a common path. - Move value and src to local scope of use. Signed-off-by: Johannes Weiner Reviewed-by: Chengming Zhou Acked-by: Yosry Ahmed Reviewed-by: Nhat Pham --- mm/zswap.c | 28 +++++++++++++--------------- 1 file changed, 13 insertions(+), 15 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index f9b9494156ba..cde309c539b3 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1542,14 +1542,11 @@ bool zswap_store(struct folio *folio) { swp_entry_t swp = folio->swap; pgoff_t offset = swp_offset(swp); - struct page *page = &folio->page; struct zswap_tree *tree = swap_zswap_tree(swp); struct zswap_entry *entry, *dupentry; struct obj_cgroup *objcg = NULL; struct mem_cgroup *memcg = NULL; - struct zswap_pool *pool; - unsigned long value; - u8 *src; + struct zswap_pool *shrink_pool; VM_WARN_ON_ONCE(!folio_test_locked(folio)); VM_WARN_ON_ONCE(!folio_test_swapcache(folio)); @@ -1567,10 +1564,10 @@ bool zswap_store(struct folio *folio) * the tree, and it might be written back overriding the new data. */ spin_lock(&tree->lock); - dupentry = zswap_rb_search(&tree->rbroot, offset); - if (dupentry) { + entry = zswap_rb_search(&tree->rbroot, offset); + if (entry) { + zswap_invalidate_entry(tree, entry); zswap_duplicate_entry++; - zswap_invalidate_entry(tree, dupentry); } spin_unlock(&tree->lock); objcg = get_obj_cgroup_from_folio(folio); @@ -1598,17 +1595,19 @@ bool zswap_store(struct folio *folio) } /* allocate entry */ - entry = zswap_entry_cache_alloc(GFP_KERNEL, page_to_nid(page)); + entry = zswap_entry_cache_alloc(GFP_KERNEL, folio_nid(folio)); if (!entry) { zswap_reject_kmemcache_fail++; goto reject; } if (zswap_same_filled_pages_enabled) { - src = kmap_local_page(page); + unsigned long value; + u8 *src; + + src = kmap_local_folio(folio, 0); if (zswap_is_page_same_filled(src, &value)) { kunmap_local(src); - entry->swpentry = swp; entry->length = 0; entry->value = value; atomic_inc(&zswap_same_filled_pages); @@ -1637,9 +1636,8 @@ bool zswap_store(struct folio *folio) if (!zswap_compress(folio, entry)) goto put_pool; - entry->swpentry = swp; - insert_entry: + entry->swpentry = swp; entry->objcg = objcg; if (objcg) { obj_cgroup_charge_zswap(objcg, entry->length); @@ -1684,9 +1682,9 @@ bool zswap_store(struct folio *folio) return false; shrink: - pool = zswap_pool_last_get(); - if (pool && !queue_work(shrink_wq, &pool->shrink_work)) - zswap_pool_put(pool); + shrink_pool = zswap_pool_last_get(); + if (shrink_pool && !queue_work(shrink_wq, &shrink_pool->shrink_work)) + zswap_pool_put(shrink_pool); goto reject; } From patchwork Tue Jan 30 01:36:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536675 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1C39C47DA9 for ; Tue, 30 Jan 2024 01:42:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9D2296B00B2; Mon, 29 Jan 2024 20:42:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 95C4A6B00B3; Mon, 29 Jan 2024 20:42:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7AD116B00B4; Mon, 29 Jan 2024 20:42:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 65EB46B00B2 for ; Mon, 29 Jan 2024 20:42:27 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 3C31080263 for ; Tue, 30 Jan 2024 01:42:27 +0000 (UTC) X-FDA: 81734277534.08.769C596 Received: from mail-qv1-f42.google.com (mail-qv1-f42.google.com [209.85.219.42]) by imf23.hostedemail.com (Postfix) with ESMTP id 789A7140019 for ; Tue, 30 Jan 2024 01:42:25 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=R36OVo0D; spf=pass (imf23.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.42 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578945; a=rsa-sha256; cv=none; b=X27WeaCPeHm0Fo8zNWnCrdM8RsRq3f7pv14aDZUjkC2Z8h8aXCX93SoVb3cDmAiwqymAR/ SKW6qlYyhXqJ6v5c5IxkW8iRQIfpzQ7yH3f0rg6k6fIyRld5hPtuttgodRW/LvxJwBccao iV1Y6ag8Wzn1nlStnVcDUlUUH+9X0e8= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=R36OVo0D; spf=pass (imf23.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.42 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578945; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YF/Bz5keaw8EKm3mlmWlxcJJmelaMJ52iqxIOq55K14=; b=ZWHBOdyNP7/7jlqMV44yKGkNYLm6ogETsJUDj479WaJQfgRNLQOSw9vJkw35AQyB9TySYa W19mY/bGR1XeZAAMsbEGL+DgTUqixEWsQW9hRAlfH87nk1fs4+TjkCGCgmP+hbbU1ynh/j PFRMyfxskcxRaSZNg5y2jXRPgLDj5l4= Received: by mail-qv1-f42.google.com with SMTP id 6a1803df08f44-68873473ce6so27582946d6.0 for ; Mon, 29 Jan 2024 17:42:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578944; x=1707183744; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YF/Bz5keaw8EKm3mlmWlxcJJmelaMJ52iqxIOq55K14=; b=R36OVo0DctXCGSafskRwptguWYaVOHSVYQFxUT5TRhSAJ6WRUaf2eAqaDXlVM1Giym NPjWhYIcUUbEP0kln1H7mD8qq6uj61nRBS5K92v0e6wC4WP2fOlyLqDNKzkpcvQocdgo Wy+bGy8L7CxpmbHBOTdwGmnAdNC3qqs/C74xnBqJfzFWdeLVjhIKl9QX44hosE/0jPa+ AmmjRKLHlVVJ+dcZdqKDE27/yX4xdbXURg39hxets2x4h0MgrjcxXLnhTlly29m8XRlu efk8VzxRuUvRA2qbGPBN6r35qokGZFRcC9GpsG7TIw1MWlVtq1c/agjIkWp2Zs9MEekD Z5+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578944; x=1707183744; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YF/Bz5keaw8EKm3mlmWlxcJJmelaMJ52iqxIOq55K14=; b=in2ZmJFfisNrAqfgCX1LRM9YM8g1oAGYkxmIth7auAgPYmQLb5O4Knsx2P/UYx9T7W FdG+9/xZcwZPF6/e+z25cC6KIUumJso7Jj/lx0blfKmpUbzkxexrEFRUr8HLP/JjMKs2 vF2uo/CwSpeMzAwezviBdyj/QQDsHe/nmSF/XKumLGAiehj/03h/Av2BxPg5mYigwozV hPXL4+kvZ7t58tjo6ss3WxGcBFx9x1oaqyFpzOaNmI6nrUuyUa1PUEiIoqHdDlajCRDc r/xGmWAORGndLME59oIa2wFNdNhUUhRvhUGAlbUdAYBrowj0LaO1AWNwUaOIz7URJWY0 TdEw== X-Gm-Message-State: AOJu0YxXArt54I8yM4awL3Edhl/KgycduarrnV/z6Hvvcp18KuX0aXGB bn3p8FsgFtZg5zdU216lV1124CjgasrrLMvq65IJ6xUAFN9Kl3Y6Ogt+h697nvy2UxUqiO6wMRp R X-Google-Smtp-Source: AGHT+IEJUv991aH24M+mVS5b+IJCVkoWklj7Vk7Ouj1HKmo2haOgY/W/6O3BRf6ukVaZRAeczwm31g== X-Received: by 2002:a05:6214:1804:b0:68c:623d:13d3 with SMTP id o4-20020a056214180400b0068c623d13d3mr24326qvw.68.1706578944683; Mon, 29 Jan 2024 17:42:24 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id kf4-20020a056214524400b006837a012417sm3992019qvb.51.2024.01.29.17.42.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:24 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 09/20] mm: zswap: simplify zswap_invalidate() Date: Mon, 29 Jan 2024 20:36:45 -0500 Message-ID: <20240130014208.565554-10-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 789A7140019 X-Stat-Signature: 9tb4d13h6b4k76qeueinwgkxsgxq51dm X-Rspam-User: X-HE-Tag: 1706578945-220352 X-HE-Meta: U2FsdGVkX19ydKuwlJmeOxo/FD5WWbOxc47VJ/vQiByLgviimqaddnqeP1eFxiSpwiwTbDSbAmOGl9inL1n8hthSBH/iFhciNYvOBWRUeU46daz6dwdOu/j3LncHK1QdV9oP/Mmq2nAoxrFvK4FlB+oKFm7bXjp6pY9sxPdKQMuo63KPb/JA01NWWYNguOG0gUNDr8dhFuUMzKpbOefuUhf32peW0I4I5+ad5VaA91Iz3rhhqqPYghilOU6DE37Q2U8ipbPoG94RIOxM8qin85jC7e8Z5EW0vbrC8hDEShJj79owXMvSVt/GSQ+hNUWjFjitjKuA7D4swdbii4a2WoiVVtwxa4zYseYrtHi1n/HqPfp//mA+z20Djp/rJ14KzxkVFQYNPYrCdG+5FldAc5L5JJASHZRBh8RtEnhdRzp76fNlcKp1dPqlRNGadvLDhTEwZp5knLjQjQ3etxwGXxsOf/G4n3+JoaJ1k7RZeRzfs8p5hxk6rqFfRTL8wru+ofeHcbSNBUdlImsULg1OqnwTKJQ+C7aGv+9R36JFmJz3jqxjP4+mF3NNeqqTXtbUuszjDIdi0gkOoKNCH4sfdrDVETVpeOfd8uLc/akBxP/O3crfDj3JubuGb/Th0T/ZcPRRL5EFW4EY7TzlsZWZT5ws6nMSRu/9ifwxdsqduAhZL28HpFOjGyILxz2N/AXZvx0cdFskx8oUYykJLb6L+uQ7B9JrozbKDBmJ+PiFOFJDfZ8BkKwC+njueDY6hePJudX3WwFfNSI/P7Aa7JVuVrgM/Xv0y9xKgN7yl/9xlEkXvxdIH873deEsjc+7zuolkc/wbY3OEsUA0mT/jQJ5h6oJgu4adWq239KjZ9uMZi1lkDcyIypM4Ju8XCY6iahJxsocdJ6+hgVUu0iq7BjsPsjtK3Rkf220Ra1Pb+ZfZQ2SXgOKwM11rOMadlpUI6dswC26WnFR6vxv+x7VTHQ dXBD4/Sk hJijqNJdgcOre4OnmDCPiXgS/dYF0BXIPgisnLLHRkql5tmrmTZHPZ21bOQ6FYS1BTpIPPN1j9feLp5ubHvmWXQKcmC/3731Rc7FzNY/YVj6UTPVNp89rl0GO98kYUX6j8dHxufjQm/7Z7kg36uZCI/m+NkLQGgUbenZf8pjTcR3fhEHq0S//t25Xn8rR/lFuaNAeNXdVen0/uwqWBgDFz4fl5rSTLWM+35FK1nnfI2KfVz3PATivYcWstpvjt0jGG71RATnmKqtIzmA30jFjRrfha1U1t/t/iogxvboSLMvqCF/D87L5iU8zU2qQ5eu6ySCn6XH3sRqPrkPMlYgaSSmtFWyGO/5Wet3pouAoozkPlY11lbAGt3gKU6XHnOZOoKW3Lc97IhZi5e8JS/pKQMpQQTKFHNNxLCtA9t4cBjbT4JeRa8tCJd/XqjSo/2dIl1WNgcSqejcLQM+NXgT2rLV8TA1ja2NrdDC+3owLLV+dbBU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000103, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The branching is awkward and duplicates code. The comment about writeback is also misleading: yes, the entry might have been written back. Or it might have never been stored in zswap to begin with due to a rejection - zswap_invalidate() is called on all exiting swap entries. Signed-off-by: Johannes Weiner Reviewed-by: Chengming Zhou Acked-by: Yosry Ahmed Reviewed-by: Nhat Pham --- mm/zswap.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index cde309c539b3..082d076a758d 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1739,15 +1739,10 @@ void zswap_invalidate(int type, pgoff_t offset) struct zswap_tree *tree = swap_zswap_tree(swp_entry(type, offset)); struct zswap_entry *entry; - /* find */ spin_lock(&tree->lock); entry = zswap_rb_search(&tree->rbroot, offset); - if (!entry) { - /* entry was written back */ - spin_unlock(&tree->lock); - return; - } - zswap_invalidate_entry(tree, entry); + if (entry) + zswap_invalidate_entry(tree, entry); spin_unlock(&tree->lock); } From patchwork Tue Jan 30 01:36:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536676 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55E18C47DB3 for ; Tue, 30 Jan 2024 01:42:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 011806B00A3; Mon, 29 Jan 2024 20:42:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id ED8D26B00B3; Mon, 29 Jan 2024 20:42:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D51B76B00B4; Mon, 29 Jan 2024 20:42:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C29346B00A3 for ; Mon, 29 Jan 2024 20:42:28 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 9BE42140651 for ; Tue, 30 Jan 2024 01:42:28 +0000 (UTC) X-FDA: 81734277576.28.27D3A77 Received: from mail-qv1-f50.google.com (mail-qv1-f50.google.com [209.85.219.50]) by imf20.hostedemail.com (Postfix) with ESMTP id D12DC1C000D for ; Tue, 30 Jan 2024 01:42:26 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=b77cHJX4; spf=pass (imf20.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.50 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578946; a=rsa-sha256; cv=none; b=awUzE4fbXjjX9zIVlGUm59iR3cd334QAIa22x8+MeJi3a577M5E6BsK4Uod0YhRTtrKq/o zkhJSNgDNSbeECzS8BveUrEU0JuHf3ojWPWJXxU2R+lrrWLsUP8/6yPIzAGnReehTWxJEr R1DkT98T1tCRR/QUxy4pf4M9dCRMfz0= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=b77cHJX4; spf=pass (imf20.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.50 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578946; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rkkvsf4pStxQTi2gKgo4CCnvmALR1LUjxBEOZSgiwUU=; b=qF/Pp7RCoe5Au/kHUMBiIkKohQg+zO2GtR9lMuR7agL+1Q1/iBhGMecrXPwuCN18Utm5bF xiX7DbLJsWFQdbCsqelvtaFkUcj+yaqEgoU28IwSJGehG1gQZ7dpyYGGcUYgKquH5Sj/VE JvWJK4UV/5B90+1jnCy003rA9LO4DMo= Received: by mail-qv1-f50.google.com with SMTP id 6a1803df08f44-68c4fb9e7e0so11622886d6.2 for ; Mon, 29 Jan 2024 17:42:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578946; x=1707183746; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rkkvsf4pStxQTi2gKgo4CCnvmALR1LUjxBEOZSgiwUU=; b=b77cHJX4WfEqaTl0B3i6uVN9mfwNnc4Wo+ERDV6THJofsASjqZD3tx6NdVBeQKLLe8 wRQoDosR+ZLOKRqIk8fbSSehRSSOKe+NwAav0qiYVmea5nJuMe9YREENqDl2Dlfa7R10 9XPYrJSSLPdMFqiNdFRqT8fz6GLAQrK3n8fU/LrFD6Od2nsAPJzGgdOQxnfthne2fRnt OJMGSTtOPnaaPKOh22g1DoKIH3WPbEgasAzZ2v77SCrBRo4fGl1Kqw7gyzgb4NNAN+FD Mb/z9HyQD5+Ru007ZZGfvQfbqBUFoOp0vPUkSGfXonCsHL3BPXfSMn5KSvrBfzwhX/L/ 3TfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578946; x=1707183746; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rkkvsf4pStxQTi2gKgo4CCnvmALR1LUjxBEOZSgiwUU=; b=DF+Jt4gQNoiZymoPlNFYH/NmAf1D7AnGrEgH1kyx++DS3Cc1uSxjXLP26GktYaNysL qwxhFh0xcyXWnQgQj70m/EAdWTAPIn/dy3mId4aShveMqKTV0Udfpf9L54xeRpfGiDYv GiSTvIN/wQ/g9nLEXesJkupqcz2gvSLjKY35vv+WVQz0T2uTfKFmIyL16GA8mI/6hhTi hcxrDDbSuUS1mVsdShWO2mdl7rza6ApvdQ6YMLF+/01FoHRsHV2q/STylYyistE/XzPG geBhOTqR6lNggW3twlMXvOd5TE565JI5593uG8RE9ymqxQCvOz5mPnRowSmST5hbqFLS uFMA== X-Gm-Message-State: AOJu0Yy/O5IOilhFUc9p/IcTYmoFdjUcagfYuK16S3wn44bH2fn6bCl7 d4dPdqQFay+i1Lnq1gfoxVkDo6K9Kr31vlJTsaY01wNevEcC+T3dkufftPSKXa0= X-Google-Smtp-Source: AGHT+IEupJwBS6PzRmr4d1gT3dBmbqwm+T5GT7wGrB81djH8pkmwnFJL+V394X2JsA3CiQUdBjgjdA== X-Received: by 2002:ad4:5f89:0:b0:68c:5d15:25c1 with SMTP id jp9-20020ad45f89000000b0068c5d1525c1mr790999qvb.107.1706578946041; Mon, 29 Jan 2024 17:42:26 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id vz9-20020a05620a494900b007833c4dddfdsm3587691qkn.53.2024.01.29.17.42.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:25 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 10/20] mm: zswap: function ordering: pool alloc & free Date: Mon, 29 Jan 2024 20:36:46 -0500 Message-ID: <20240130014208.565554-11-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: D12DC1C000D X-Stat-Signature: 39qme45xzqgny346wx4bcb11kap6455k X-Rspam-User: X-HE-Tag: 1706578946-226261 X-HE-Meta: U2FsdGVkX1+5IV5rRUixLz6r97zBJpM2dlPcrydPn6wgn5qmEw/d6Chge97wI6ZlcDCQ26HSprrP2b8IkRhj+737pJaI7vUOhXPBbFKxkOSpvWRq08vlE7E12TC1jrvOn52sd9Va6AaCJ0i4DAfzM7UHKVh+gqhGnYJdCqgN8txJUTbNg/X+15gMjZqMjtg1SXd+DTp7UKJsSbjxT1Y4FGVeYfQ3MLtIBMqLCmfgT1wGzzKjEI5W46R11cS7yFJRf68a9Iw1sMuq4BfWmsgWWsW6nbbzniUjadnAwdLDkQd3TBCvHJkXoYWduA3NjshCJ1VMX0byLuNngpx85bHHv0PFmkRkkht84d3rG1BfLwbGOiBc/fFCr+tStgC2GYl+3uPptMBApGdwh0IEzcg8w5kdDWvES0TQG2Fp/uHQb43RPXAj9v8GhMwcCGQVLc/rnJs87ExS5Xo3thH2TmYUzsowAPnheq+Lsiw1psMf3tyctnM8Sypg8Lxch6fDaDZBVCO1B6IzJsLXDvAJ++oVKgiWVq06+NQbw0uhTClHNy0i6ZEQDbHmMOoxfxHcKiQa9b/DLacw3JMuRkanxiJEt0968InbcOyvWJbRjd1lZdm/lrC4FvNdhwOC20t6ftCLUVkXPO8yCMz3/S3wqqHoGy9HMWjNPH9uEOfqpaRjrlXMHgl4gBM0+mkO8u5uhclT3GXUauE60FAJ5GecExMZgzH2Ec2C8BpaiHicDe7lLkIZRH8wv8b+GThvTO3BdsNf9CfEv48YnKTgPYAfzWL8bR3yujzmnEvIaN4lgZo0k2eOV5/qo8DPwUjcsRgqZj1UhiYvebn8ndBOoQHoaXCpcU7TALGIOC0UDo0WV+YgLvznCXNoQGYqy2madKh2JRwPB208gJaw6RTjRxAILvLsjQ2BOBYN4Lef1gr+p1jJQ9cEIpJ3kM8f7SIuhWdgYl7/YQpkreXFYfNfrPttde1 6uH3Ezma TbUbXouzDQtCfO/m4Zgv9hK1y2/nYr7fKCxS/+LmDZiXLNVfJTtDkgCiYLO4ufUQYAqtLOzY8lH9Fa/bY2o8IMHCkbykIsLcE9ftxoqVJsFp49HnPTPl2tyWArCSHWLGnCBmSLxz/tW8Lp8qTufRy5QK72g8T86mqVRVDAUbQTWSZzG5BHKSx//al1qtHk1gDZmp5+Q6mDe4+h1xU1gY+tYEKVJ8ajm77wy2HafZY7fgDzoxf1oKT2fo7jUp3THtOjpIyVNCA9tODFKq2o59KUYMcjPrw3AkWQrb292rR65mU/OOcotNhXvr09cY1W00mcL6YvWiORRAJROFa+pTkGtjF7xaWt8Re1NbyKi5NNLd2DstbNNSAuUhX1VYf7RDuwzkrh8iUr3u4aS2NX2dEjYSAXWicY61m93Ps/ve5dVapLBU/g7xJQfYsG3k1j/LX2cT4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The function ordering in zswap.c is a little chaotic, which requires jumping in unexpected directions when following related code. This is a series of patches that brings the file into the following order: - pool functions - lru functions - rbtree functions - zswap entry functions - compression/backend functions - writeback & shrinking functions - store, load, invalidate, swapon, swapoff - debugfs - init But it has to be split up such the moving still produces halfway readable diffs. In this patch, move pool allocation and freeing functions. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 297 +++++++++++++++++++++++++++-------------------------- 1 file changed, 152 insertions(+), 145 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 082d076a758d..805d9a35f633 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -320,6 +320,158 @@ static void zswap_update_total_size(void) zswap_pool_total_size = total; } +/********************************* +* pool functions +**********************************/ + +static void zswap_alloc_shrinker(struct zswap_pool *pool); +static void shrink_worker(struct work_struct *w); + +static struct zswap_pool *zswap_pool_create(char *type, char *compressor) +{ + int i; + struct zswap_pool *pool; + char name[38]; /* 'zswap' + 32 char (max) num + \0 */ + gfp_t gfp = __GFP_NORETRY | __GFP_NOWARN | __GFP_KSWAPD_RECLAIM; + int ret; + + if (!zswap_has_pool) { + /* if either are unset, pool initialization failed, and we + * need both params to be set correctly before trying to + * create a pool. + */ + if (!strcmp(type, ZSWAP_PARAM_UNSET)) + return NULL; + if (!strcmp(compressor, ZSWAP_PARAM_UNSET)) + return NULL; + } + + pool = kzalloc(sizeof(*pool), GFP_KERNEL); + if (!pool) + return NULL; + + for (i = 0; i < ZSWAP_NR_ZPOOLS; i++) { + /* unique name for each pool specifically required by zsmalloc */ + snprintf(name, 38, "zswap%x", + atomic_inc_return(&zswap_pools_count)); + + pool->zpools[i] = zpool_create_pool(type, name, gfp); + if (!pool->zpools[i]) { + pr_err("%s zpool not available\n", type); + goto error; + } + } + pr_debug("using %s zpool\n", zpool_get_type(pool->zpools[0])); + + strscpy(pool->tfm_name, compressor, sizeof(pool->tfm_name)); + + pool->acomp_ctx = alloc_percpu(*pool->acomp_ctx); + if (!pool->acomp_ctx) { + pr_err("percpu alloc failed\n"); + goto error; + } + + ret = cpuhp_state_add_instance(CPUHP_MM_ZSWP_POOL_PREPARE, + &pool->node); + if (ret) + goto error; + + zswap_alloc_shrinker(pool); + if (!pool->shrinker) + goto error; + + pr_debug("using %s compressor\n", pool->tfm_name); + + /* being the current pool takes 1 ref; this func expects the + * caller to always add the new pool as the current pool + */ + kref_init(&pool->kref); + INIT_LIST_HEAD(&pool->list); + if (list_lru_init_memcg(&pool->list_lru, pool->shrinker)) + goto lru_fail; + shrinker_register(pool->shrinker); + INIT_WORK(&pool->shrink_work, shrink_worker); + atomic_set(&pool->nr_stored, 0); + + zswap_pool_debug("created", pool); + + return pool; + +lru_fail: + list_lru_destroy(&pool->list_lru); + shrinker_free(pool->shrinker); +error: + if (pool->acomp_ctx) + free_percpu(pool->acomp_ctx); + while (i--) + zpool_destroy_pool(pool->zpools[i]); + kfree(pool); + return NULL; +} + +static struct zswap_pool *__zswap_pool_create_fallback(void) +{ + bool has_comp, has_zpool; + + has_comp = crypto_has_acomp(zswap_compressor, 0, 0); + if (!has_comp && strcmp(zswap_compressor, + CONFIG_ZSWAP_COMPRESSOR_DEFAULT)) { + pr_err("compressor %s not available, using default %s\n", + zswap_compressor, CONFIG_ZSWAP_COMPRESSOR_DEFAULT); + param_free_charp(&zswap_compressor); + zswap_compressor = CONFIG_ZSWAP_COMPRESSOR_DEFAULT; + has_comp = crypto_has_acomp(zswap_compressor, 0, 0); + } + if (!has_comp) { + pr_err("default compressor %s not available\n", + zswap_compressor); + param_free_charp(&zswap_compressor); + zswap_compressor = ZSWAP_PARAM_UNSET; + } + + has_zpool = zpool_has_pool(zswap_zpool_type); + if (!has_zpool && strcmp(zswap_zpool_type, + CONFIG_ZSWAP_ZPOOL_DEFAULT)) { + pr_err("zpool %s not available, using default %s\n", + zswap_zpool_type, CONFIG_ZSWAP_ZPOOL_DEFAULT); + param_free_charp(&zswap_zpool_type); + zswap_zpool_type = CONFIG_ZSWAP_ZPOOL_DEFAULT; + has_zpool = zpool_has_pool(zswap_zpool_type); + } + if (!has_zpool) { + pr_err("default zpool %s not available\n", + zswap_zpool_type); + param_free_charp(&zswap_zpool_type); + zswap_zpool_type = ZSWAP_PARAM_UNSET; + } + + if (!has_comp || !has_zpool) + return NULL; + + return zswap_pool_create(zswap_zpool_type, zswap_compressor); +} + +static void zswap_pool_destroy(struct zswap_pool *pool) +{ + int i; + + zswap_pool_debug("destroying", pool); + + shrinker_free(pool->shrinker); + cpuhp_state_remove_instance(CPUHP_MM_ZSWP_POOL_PREPARE, &pool->node); + free_percpu(pool->acomp_ctx); + list_lru_destroy(&pool->list_lru); + + spin_lock(&zswap_pools_lock); + mem_cgroup_iter_break(NULL, pool->next_shrink); + pool->next_shrink = NULL; + spin_unlock(&zswap_pools_lock); + + for (i = 0; i < ZSWAP_NR_ZPOOLS; i++) + zpool_destroy_pool(pool->zpools[i]); + kfree(pool); +} + /* should be called under RCU */ #ifdef CONFIG_MEMCG static inline struct mem_cgroup *mem_cgroup_from_entry(struct zswap_entry *entry) @@ -970,151 +1122,6 @@ static void shrink_worker(struct work_struct *w) zswap_pool_put(pool); } -static struct zswap_pool *zswap_pool_create(char *type, char *compressor) -{ - int i; - struct zswap_pool *pool; - char name[38]; /* 'zswap' + 32 char (max) num + \0 */ - gfp_t gfp = __GFP_NORETRY | __GFP_NOWARN | __GFP_KSWAPD_RECLAIM; - int ret; - - if (!zswap_has_pool) { - /* if either are unset, pool initialization failed, and we - * need both params to be set correctly before trying to - * create a pool. - */ - if (!strcmp(type, ZSWAP_PARAM_UNSET)) - return NULL; - if (!strcmp(compressor, ZSWAP_PARAM_UNSET)) - return NULL; - } - - pool = kzalloc(sizeof(*pool), GFP_KERNEL); - if (!pool) - return NULL; - - for (i = 0; i < ZSWAP_NR_ZPOOLS; i++) { - /* unique name for each pool specifically required by zsmalloc */ - snprintf(name, 38, "zswap%x", - atomic_inc_return(&zswap_pools_count)); - - pool->zpools[i] = zpool_create_pool(type, name, gfp); - if (!pool->zpools[i]) { - pr_err("%s zpool not available\n", type); - goto error; - } - } - pr_debug("using %s zpool\n", zpool_get_type(pool->zpools[0])); - - strscpy(pool->tfm_name, compressor, sizeof(pool->tfm_name)); - - pool->acomp_ctx = alloc_percpu(*pool->acomp_ctx); - if (!pool->acomp_ctx) { - pr_err("percpu alloc failed\n"); - goto error; - } - - ret = cpuhp_state_add_instance(CPUHP_MM_ZSWP_POOL_PREPARE, - &pool->node); - if (ret) - goto error; - - zswap_alloc_shrinker(pool); - if (!pool->shrinker) - goto error; - - pr_debug("using %s compressor\n", pool->tfm_name); - - /* being the current pool takes 1 ref; this func expects the - * caller to always add the new pool as the current pool - */ - kref_init(&pool->kref); - INIT_LIST_HEAD(&pool->list); - if (list_lru_init_memcg(&pool->list_lru, pool->shrinker)) - goto lru_fail; - shrinker_register(pool->shrinker); - INIT_WORK(&pool->shrink_work, shrink_worker); - atomic_set(&pool->nr_stored, 0); - - zswap_pool_debug("created", pool); - - return pool; - -lru_fail: - list_lru_destroy(&pool->list_lru); - shrinker_free(pool->shrinker); -error: - if (pool->acomp_ctx) - free_percpu(pool->acomp_ctx); - while (i--) - zpool_destroy_pool(pool->zpools[i]); - kfree(pool); - return NULL; -} - -static struct zswap_pool *__zswap_pool_create_fallback(void) -{ - bool has_comp, has_zpool; - - has_comp = crypto_has_acomp(zswap_compressor, 0, 0); - if (!has_comp && strcmp(zswap_compressor, - CONFIG_ZSWAP_COMPRESSOR_DEFAULT)) { - pr_err("compressor %s not available, using default %s\n", - zswap_compressor, CONFIG_ZSWAP_COMPRESSOR_DEFAULT); - param_free_charp(&zswap_compressor); - zswap_compressor = CONFIG_ZSWAP_COMPRESSOR_DEFAULT; - has_comp = crypto_has_acomp(zswap_compressor, 0, 0); - } - if (!has_comp) { - pr_err("default compressor %s not available\n", - zswap_compressor); - param_free_charp(&zswap_compressor); - zswap_compressor = ZSWAP_PARAM_UNSET; - } - - has_zpool = zpool_has_pool(zswap_zpool_type); - if (!has_zpool && strcmp(zswap_zpool_type, - CONFIG_ZSWAP_ZPOOL_DEFAULT)) { - pr_err("zpool %s not available, using default %s\n", - zswap_zpool_type, CONFIG_ZSWAP_ZPOOL_DEFAULT); - param_free_charp(&zswap_zpool_type); - zswap_zpool_type = CONFIG_ZSWAP_ZPOOL_DEFAULT; - has_zpool = zpool_has_pool(zswap_zpool_type); - } - if (!has_zpool) { - pr_err("default zpool %s not available\n", - zswap_zpool_type); - param_free_charp(&zswap_zpool_type); - zswap_zpool_type = ZSWAP_PARAM_UNSET; - } - - if (!has_comp || !has_zpool) - return NULL; - - return zswap_pool_create(zswap_zpool_type, zswap_compressor); -} - -static void zswap_pool_destroy(struct zswap_pool *pool) -{ - int i; - - zswap_pool_debug("destroying", pool); - - shrinker_free(pool->shrinker); - cpuhp_state_remove_instance(CPUHP_MM_ZSWP_POOL_PREPARE, &pool->node); - free_percpu(pool->acomp_ctx); - list_lru_destroy(&pool->list_lru); - - spin_lock(&zswap_pools_lock); - mem_cgroup_iter_break(NULL, pool->next_shrink); - pool->next_shrink = NULL; - spin_unlock(&zswap_pools_lock); - - for (i = 0; i < ZSWAP_NR_ZPOOLS; i++) - zpool_destroy_pool(pool->zpools[i]); - kfree(pool); -} - static int __must_check zswap_pool_get(struct zswap_pool *pool) { if (!pool) From patchwork Tue Jan 30 01:36:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536677 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2165C47DB3 for ; Tue, 30 Jan 2024 01:42:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 628876B00B3; Mon, 29 Jan 2024 20:42:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5D6C26B00B4; Mon, 29 Jan 2024 20:42:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 402456B00B5; Mon, 29 Jan 2024 20:42:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 2C9836B00B3 for ; Mon, 29 Jan 2024 20:42:30 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id ED0ADC0239 for ; Tue, 30 Jan 2024 01:42:29 +0000 (UTC) X-FDA: 81734277618.19.F44AA39 Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com [209.85.222.181]) by imf07.hostedemail.com (Postfix) with ESMTP id 1907940017 for ; Tue, 30 Jan 2024 01:42:27 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=KrNTCHdW; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf07.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.181 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578948; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nW+93M2Q9nWxSPgR21ybfgSivHJtX7f76WUEXvkAumg=; b=hjTer1qfWXQp60OBywQNRGWmwgaqCVB++zywASWRCY5lYyFB0627ltHtNzbv/jpoZpmKna lm9OHkUtOcAS0kZ5C4g+3PvB5rLThNEz36EaFrL7Yu8y5wVKSsnnxUb6ZpMObSJe0nVCeK sXTtO3wqEEKjOfDPpS7mcZnjjMvyTmg= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=KrNTCHdW; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf07.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.181 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578948; a=rsa-sha256; cv=none; b=Ex1Rl8SwZ1jHUktniRGFWfRLjuQBnrG70J1V0UX3aMVD+RUeZweYzUqMVys6IeD+lGVFP3 MzmgfcHIqgmqYWXg8qHBR4eGShcG/mGzsoNr+5+KMK2fHuS+4iDLi9czuHEEz3JddIsMzP Hc/BW2TPmCv1/xfZ4vB7VOC+eZ9HGQ4= Received: by mail-qk1-f181.google.com with SMTP id af79cd13be357-783da26489aso280796185a.3 for ; Mon, 29 Jan 2024 17:42:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578947; x=1707183747; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nW+93M2Q9nWxSPgR21ybfgSivHJtX7f76WUEXvkAumg=; b=KrNTCHdWI5yNYFfblX+EPD9Wt4htph+YbNYGgvbOn7f/HhJvBl/NbkPHZDwOONe5n4 cYPkXhLNAOntUgw7UkgZ4NztCBUfICrfILjmuQpQBl9sJyEhYejpdTpcexja7FyLuVTE SLSwCordhVxqwhjGK1kxPcRIDHVTPr2iJaZOOi7XaExAAnWWFuiH/rmVEKcV5qIRcFcS hOA3U4De7R0Gbxa/WMfM4ULp5U0Jhg7VbABdx+Ia50lFlVR7HrY43ofAYj605Qw9ifGV V/sw10JcwJHRfRUtYbOCFnllHH76WK3vleaVxv2u21zQTkQU4weZOFi7yets71uS2xqV LRgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578947; x=1707183747; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nW+93M2Q9nWxSPgR21ybfgSivHJtX7f76WUEXvkAumg=; b=dlbAhqnSp2jRpx7bVuV3jO/LiMbwCQvpzWLpUXtIXCGPZ81L14BBGjXGmhPtW8byGy aQcTgDS/bD5scVu+9ARREy9u/fvn8wIuOS+dyu0OPsVXrMCPBe3iWopwH4rvAEOu4lsb nYmzxRAxpntf84Ij5kV173YJWUivQoBS9Gvvw/6izYmb+zfGwKnGM4Tb/AhPPVtsS6g1 5DFRym8P5S8QP4xYcPP2ogViHiBxorAaPpS/6/LZu1ugwCBKb0vN/zLlcR3Oe2xmGRve nMVxpVk6sZNwaEU5YaqD5Re2wyISAUMs+VW5NCKwdADL4cMqRYRWp/vyNEjj4ujPBhgU YhDg== X-Gm-Message-State: AOJu0YzFz1O+SO/zHcxB9bs2lAA4s0h+N4PQnECMiCaJYT4+CGqUhc3U RJaYMuZE0/JC+WvMsVdufGItyorv2QZbrjfYDEi1cxdb2MgmVqk+7B01hAgynno= X-Google-Smtp-Source: AGHT+IGbQQQZq+kfQrrdhSmxdR44XoO3pzuQK7Yh8fRiEenD92oHac9H3b2LTsWjU2hqhbKGsaWTag== X-Received: by 2002:a05:6214:cab:b0:686:9441:ac30 with SMTP id s11-20020a0562140cab00b006869441ac30mr8699046qvs.22.1706578947264; Mon, 29 Jan 2024 17:42:27 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id lr9-20020a0562145bc900b006819bb31533sm3993392qvb.99.2024.01.29.17.42.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:26 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 11/20] mm: zswap: function ordering: pool refcounting Date: Mon, 29 Jan 2024 20:36:47 -0500 Message-ID: <20240130014208.565554-12-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 1907940017 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: hydy48a4oiyxsi9s4pabruh88maqd5fg X-HE-Tag: 1706578947-960855 X-HE-Meta: U2FsdGVkX1+vzuYM6I2ir6RBt18bwqlFatcr7bNb5YSOWCxCS8gJ98hoQ9nrAuhN1Ta5WQvQECZr3rhL7hfV1Oa888Fj4usfmj1QS/0Y9GWHa/hfVBh8mDal5lsp0v1umsPtxOCah5QjcvAmB+taz6TTST+heE+rMtJQYK1BA5AGh/DqAOzMSyvstzC7AG3AhTnbEuEyNeOB+UH14oZHuJUlMGZ6HuI0OGkl9Ii2/DY9mJl+l8Idqr/SKQTCGKsIIbz5VEgOcN/QV0dHnffMYiMqsD49BkeoRY0oCz09LvjLCcQRtkKaO+wCT+aIIw/pTGWziMDNFMWEywYIHzcQVWlnTgivtjgrW9N6W9QtPRTuc+c488bMq62NX5X7by8q3enM2O00+puqKUCBps0XYJVeJVRiRwLeV1lPv+8QuBSuFov3BvQk1iA2fLAMv3EOAuw0o2o18I3AOSi22hVWd12DoOiKLjqu1wT+SiQSUywCqnkuGF5l2vyRUCA9YKxoBQ13yHS75E8iSLUksKnOBi0xrDFbcdGbbF8AqyqjiCjHmpp8jCkoqCW7SGYhG0kYvKLG/tlY7C1vaR1nfntIntAf8UOD7y1jNr5fMzsKQ241G14qBGmyC4eH6lCRXyLB0E6nl6pIbqGQPUY6qNGJJ+/Th9dX/7Mld5U7iWXUVL1QSbX1HBvyPrs6gzWyZQPQUk3//nTAeSlZMonCR6Sbtt4RNUUJSZJe0BGEJZUR/5Uccd3+ZvLmEQKegmkWymtFbURgW0YE/AWhvj+PiGhehFSJmVksoHN0oaCf+31xSeqqxx+QNWPgNCfevYwb3kvZhfoVW3GzYkucMtDD3ppmd2oxwGPgSOqmL7KQAtaK+slw4kQxL1H+V0wpxb/4bdMHvWyxI6m9w4ac8OiM/H3uVRvV7mPqKT4u+MKiLSPMJIfNukGc3oKuxHMoX5mz0i3cECownjbinUMwEkmd/YO L3p6KhtD c9yqpnOr6MXfoTTcHJpPkLNIRgZwiJu/ynfYPAKdUQM0RuI/mvwQLGeQi/2UuIf5Ze21gosXoknESibDD7O9Lo6fwyexkpEgGc2PT5Bvis0TH1lDxcOvTggMj585/ewYF+R3cLQtAaO4gx5F0DPVLAUr1wumYyB/nLWjhvoIGnIkm/pfoo/BBU7vfX6yN1BwH+6oJbo94hZk6PFk+ZCWc0l4pR0zBf3jsywOk+9DJzraTn/VYNzhXDu0nsp68vL0tMgW81skDyf0tM4Kj7LBe3GxKGHF5p7uiWe9MeTWnHFNKBhFQsraV6s1c4UBoJsVbczvWapG05PmDl4YZccsvigRRsbkVsQkI4MBpHFKexsZtyuT8BQwnBYHr+UiPkqLQWqc7gZOZwWzV+nTpYGWPqzCtK2aHwO4Iw7FmwxLz2990wLb8AV68nwI+pA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Move pool refcounting functions into the pool section. First the destroy functions, then the get and put which uses them. __zswap_pool_empty() has an upward reference to the global zswap_pools, to sanity check it's not the currently active pool that's being freed. That gets the forward decl for zswap_pool_cuyrrent(). This puts the get and put function above all callers, so kill the forward decls as well. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 94 +++++++++++++++++++++++++++--------------------------- 1 file changed, 47 insertions(+), 47 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 805d9a35f633..33775f2224b7 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -278,8 +278,6 @@ static inline struct zswap_tree *swap_zswap_tree(swp_entry_t swp) static int zswap_writeback_entry(struct zswap_entry *entry, swp_entry_t swpentry); -static int zswap_pool_get(struct zswap_pool *pool); -static void zswap_pool_put(struct zswap_pool *pool); static bool zswap_is_full(void) { @@ -472,6 +470,53 @@ static void zswap_pool_destroy(struct zswap_pool *pool) kfree(pool); } +static void __zswap_pool_release(struct work_struct *work) +{ + struct zswap_pool *pool = container_of(work, typeof(*pool), + release_work); + + synchronize_rcu(); + + /* nobody should have been able to get a kref... */ + WARN_ON(kref_get_unless_zero(&pool->kref)); + + /* pool is now off zswap_pools list and has no references. */ + zswap_pool_destroy(pool); +} + +static struct zswap_pool *zswap_pool_current(void); + +static void __zswap_pool_empty(struct kref *kref) +{ + struct zswap_pool *pool; + + pool = container_of(kref, typeof(*pool), kref); + + spin_lock(&zswap_pools_lock); + + WARN_ON(pool == zswap_pool_current()); + + list_del_rcu(&pool->list); + + INIT_WORK(&pool->release_work, __zswap_pool_release); + schedule_work(&pool->release_work); + + spin_unlock(&zswap_pools_lock); +} + +static int __must_check zswap_pool_get(struct zswap_pool *pool) +{ + if (!pool) + return 0; + + return kref_get_unless_zero(&pool->kref); +} + +static void zswap_pool_put(struct zswap_pool *pool) +{ + kref_put(&pool->kref, __zswap_pool_empty); +} + /* should be called under RCU */ #ifdef CONFIG_MEMCG static inline struct mem_cgroup *mem_cgroup_from_entry(struct zswap_entry *entry) @@ -1122,51 +1167,6 @@ static void shrink_worker(struct work_struct *w) zswap_pool_put(pool); } -static int __must_check zswap_pool_get(struct zswap_pool *pool) -{ - if (!pool) - return 0; - - return kref_get_unless_zero(&pool->kref); -} - -static void __zswap_pool_release(struct work_struct *work) -{ - struct zswap_pool *pool = container_of(work, typeof(*pool), - release_work); - - synchronize_rcu(); - - /* nobody should have been able to get a kref... */ - WARN_ON(kref_get_unless_zero(&pool->kref)); - - /* pool is now off zswap_pools list and has no references. */ - zswap_pool_destroy(pool); -} - -static void __zswap_pool_empty(struct kref *kref) -{ - struct zswap_pool *pool; - - pool = container_of(kref, typeof(*pool), kref); - - spin_lock(&zswap_pools_lock); - - WARN_ON(pool == zswap_pool_current()); - - list_del_rcu(&pool->list); - - INIT_WORK(&pool->release_work, __zswap_pool_release); - schedule_work(&pool->release_work); - - spin_unlock(&zswap_pools_lock); -} - -static void zswap_pool_put(struct zswap_pool *pool) -{ - kref_put(&pool->kref, __zswap_pool_empty); -} - /********************************* * param callbacks **********************************/ From patchwork Tue Jan 30 01:36:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536678 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90525C47DB3 for ; Tue, 30 Jan 2024 01:42:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 628F16B00B4; Mon, 29 Jan 2024 20:42:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 589276B00B5; Mon, 29 Jan 2024 20:42:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3DF326B00B6; Mon, 29 Jan 2024 20:42:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 28E7D6B00B4 for ; Mon, 29 Jan 2024 20:42:31 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0661E140651 for ; Tue, 30 Jan 2024 01:42:31 +0000 (UTC) X-FDA: 81734277702.12.EF99E40 Received: from mail-qt1-f175.google.com (mail-qt1-f175.google.com [209.85.160.175]) by imf20.hostedemail.com (Postfix) with ESMTP id 32A651C000D for ; Tue, 30 Jan 2024 01:42:29 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b="C3H9L/PT"; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf20.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.175 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578949; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6SQ/aPKt2RcQwCRNk0SkwceZy4++62pAYToaUxPVif8=; b=xFibc/+4QqSvIOE0Xe+y2BrIQPh/5Kew6qFA5+1jpuu04UxTi73eo3TmQT8WQnlEdRHCUk Nl5GA1Xh1DAQPN28T3uM7xnt34uWBSczmE8/AOhiWmudOnY25+LAFniVssMkoFHrIzE4yc IhijPh3e3RsesggK9kTcxHljQ/uXse8= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b="C3H9L/PT"; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf20.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.175 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578949; a=rsa-sha256; cv=none; b=F1jWE3kHx2EI4sjf1B74z2C7lqtUCjq1XctU2iBm3uJNfDjd82Xh4XhFRCz6Daha82evrv 6swCi6TTYzPMotO+JM3g5wzNzf/cFuOQhgWHEYyd8a/Qn7HfzUJxG6j42x1ChlFYC3RBFa 5ZQK/wLC8M7vZRRPyu5wuYuhVzu6jRc= Received: by mail-qt1-f175.google.com with SMTP id d75a77b69052e-42993124fa1so32817451cf.3 for ; Mon, 29 Jan 2024 17:42:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578948; x=1707183748; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6SQ/aPKt2RcQwCRNk0SkwceZy4++62pAYToaUxPVif8=; b=C3H9L/PT0XzpqK0vd/ULe5zwWR0aLptwHd/Rwa8z6D9cg9XE8f/1BJHeKfYUPOIU6G 9Vl1GpWGTrTnNk4iUgAhJQUQkKN3j9GQBeKNi0Ol+qy941RaRq7PCwCsllKILPE7TLbX POvKvvSPXL2y6dJslLYMXJvzu6bHNSmdYjhYvFtmKzDQxsofnKUqz4Gc0Htl+GjEw5og ASADwU/KzIR8631ST/naMeFdM9bZ+bfnXFVvhohZXs+JF6+kHoTPW/spGvb11Yt3adPH KwLZYEWdxtUl0N3p1zMJD/BNcT9z6U1TeiPwNFLZVinJB95hprUPgPU56eBTTAB6K5aF lCJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578948; x=1707183748; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6SQ/aPKt2RcQwCRNk0SkwceZy4++62pAYToaUxPVif8=; b=s059i5S0Wo3SLZfhpQ80udbuZurlHGCzyuL5luXJZpbgDZ96GnFf1Btqp/gRhBJAlW 9qIcPuxxilZ7eAaidA20ayRlAgAVMhhnGRlLgsa9p4rIkH/bqnOGc/Xgj7PRIvAlAo5Z QtRBvITFmwZ3XwaAh7Ttnknw817CL83M+Edh6VjTJyAqbCFlMONuZA9fT5hj/bloy8ge AJ1Q5cn9/HA9EENRqUnV4Bm40Np3IFWFkYsqH155LgqM4oBzpm6kaY4q8OU9tMoP1/ce zpi5ciSDM2VEluHB+ofrESkfIFI+YWcifCJuLMKoDKMyTlDoqsfuoWcLG/O1RYj+XiXJ CArg== X-Gm-Message-State: AOJu0YyDcF0iGgHSTYNn14c7adRO16L0OrywXulzGxaiTehXqCEefXxH SANvwFCsKAsBLr9m84XhIkG9mGFsYEf34ajIUlWs2LvCACLV04fj1ig1a/3FHGk= X-Google-Smtp-Source: AGHT+IHN9SfcdSpe35E+NdhAeZAJBp2Tlz+ZnOTskDGGlYQh7cy5auJKHnbJEqJ5ObxjfCI7Rxu3Cw== X-Received: by 2002:ac8:5994:0:b0:42a:b152:c0fe with SMTP id e20-20020ac85994000000b0042ab152c0femr1228897qte.31.1706578948436; Mon, 29 Jan 2024 17:42:28 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id ie7-20020a05622a698700b00427fabefe3bsm4023363qtb.52.2024.01.29.17.42.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:28 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 12/20] mm: zswap: function ordering: zswap_pools Date: Mon, 29 Jan 2024 20:36:48 -0500 Message-ID: <20240130014208.565554-13-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: fj5xd9d1ayzo3q556gbnjfmdxrk9h7cb X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 32A651C000D X-HE-Tag: 1706578949-325659 X-HE-Meta: U2FsdGVkX195QGDs53SaDL+rZFpfnY4lS48mYweGZQQAVKRw9Z48a+lx2r/e2Q9Rk5i2S+ua6y2050asXSOTVYUTpcXbyaPgZbJoONmtpjVLUtslZkkkC3BcNLRe9HVydjI26SoOXT4PTwfY6U16csGvJB8nbhdpslq3szug0+1V2nd0Yz60okf0fF6Iz/aY56mfbQAa0qqP0m2XE1u38tocgQ4yqlr6fwUPKY5Zzz+pXVkDQ9l0AABfa4+xYoWgUExiYeb8ZKJo6+rlMcSvd02/eeQz0xtL3nZqQfvSMGSkZo44MADop4RVoFFU5L7OUC5dXBeup6tuRi0uJm4gLZliiqV6BFYlpfEdDWDVdCwXOv3iA8n8lAGY04A4iHKMKSLoktWVwVOmxcOOfL4nC+N+QMfjou1MFZTt5/WJREEq9ETbkjgaMjTuW5nM0NRUItyD0Sv4HC4DMuwzL7TH/+b6mRkDnzyJiVIey5Wh/+a/zPrk/Cf5dkcFv+xDhdC3uzN3Gy/gCd5/P5X1++jKLlUCfsRZRrktgvucChigTU2kMqZmjD9iw+e4TVJx7ScC2cO0CziXqSb/PGltSREhdui5+LC9EoTQU+WyLx62SxwDDGKtP2K/8Gfa9XMdZ/CNo4nR4uFk4A3Mj71bQxN6HkvdEc8ox05+d/fYIfdkkHRZmNhHF+jhSqeXcyjRhiGDZXqUeWFscVdqgV0S4xjss8s3RZ8ihWQZc022k2x0bzvdeaEiK5o6kO+5bY6FpcOBVqz1ktBmMF3eYaIih7UJP5H04peztuMRDSnd96mSVROScq9EtxH8Ksc91ONPaP2o7tOMSm1bktfiA30hog5GL7XEGiaxWUvNAaqyWOUIn0RTJRAk1LM+oWMoieimrSekJN/qyL0ftRKIyP/TQvNi2ZjTypXMrNIXl1yRu0FT+3d6s5fSID6SKxrcpquTvWGUKElfIWDD65qBZ25cDdF s4Mee+H1 a8REYi8Oc4MFpYJP6QfqiLngJlAwFcnBOCQqwy+4eTPumvUQhLGyBKyM3unKP4TOxUdaI+ACaDCsbOwR8JC+20xrqG1ieF0v097Y5rQD2kqaTiju2bVCxnxXGJ+KELveOpR7nUm51cu69/CDxDVoXn/zzIstItu9kDUO73/c8H/dkQ+cjRbTcJR6R0fESg77Wzmacoie9G2teNYLNyQmR5iqSpjobZTbbBmTbEQGdVINJ2liVJ822i7YxJXTbIZC/0pWVJfL4RTN7rDoaBd0adVKXgMCWs8i6tO2s5fPBz9kkVM+8zdMZylJOJo8qHqR89f5GeJDDOoCJW4T0Y/Ekov/OmMWKIrmvY1DpkGRoBNzxcT5WFXNL/KFwRChEh8KVlox2Oth78iWrLY7P9plku5BDX+LO9NcfmGje12Clrk4wGlyRmzc0cuarGdAwocLHqJ1syWPoSQPyHemM3c6rsTUp5la+keUJiR0pX3CgHHzPikI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Move the operations against the global zswap_pools list (current pool, last, find) to the pool section. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 150 ++++++++++++++++++++++++++--------------------------- 1 file changed, 73 insertions(+), 77 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 33775f2224b7..168afd6767b3 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -517,6 +517,79 @@ static void zswap_pool_put(struct zswap_pool *pool) kref_put(&pool->kref, __zswap_pool_empty); } +static struct zswap_pool *__zswap_pool_current(void) +{ + struct zswap_pool *pool; + + pool = list_first_or_null_rcu(&zswap_pools, typeof(*pool), list); + WARN_ONCE(!pool && zswap_has_pool, + "%s: no page storage pool!\n", __func__); + + return pool; +} + +static struct zswap_pool *zswap_pool_current(void) +{ + assert_spin_locked(&zswap_pools_lock); + + return __zswap_pool_current(); +} + +static struct zswap_pool *zswap_pool_current_get(void) +{ + struct zswap_pool *pool; + + rcu_read_lock(); + + pool = __zswap_pool_current(); + if (!zswap_pool_get(pool)) + pool = NULL; + + rcu_read_unlock(); + + return pool; +} + +static struct zswap_pool *zswap_pool_last_get(void) +{ + struct zswap_pool *pool, *last = NULL; + + rcu_read_lock(); + + list_for_each_entry_rcu(pool, &zswap_pools, list) + last = pool; + WARN_ONCE(!last && zswap_has_pool, + "%s: no page storage pool!\n", __func__); + if (!zswap_pool_get(last)) + last = NULL; + + rcu_read_unlock(); + + return last; +} + +/* type and compressor must be null-terminated */ +static struct zswap_pool *zswap_pool_find_get(char *type, char *compressor) +{ + struct zswap_pool *pool; + + assert_spin_locked(&zswap_pools_lock); + + list_for_each_entry_rcu(pool, &zswap_pools, list) { + if (strcmp(pool->tfm_name, compressor)) + continue; + /* all zpools share the same type */ + if (strcmp(zpool_get_type(pool->zpools[0]), type)) + continue; + /* if we can't get it, it's about to be destroyed */ + if (!zswap_pool_get(pool)) + continue; + return pool; + } + + return NULL; +} + /* should be called under RCU */ #ifdef CONFIG_MEMCG static inline struct mem_cgroup *mem_cgroup_from_entry(struct zswap_entry *entry) @@ -938,83 +1011,6 @@ static int zswap_cpu_comp_dead(unsigned int cpu, struct hlist_node *node) return 0; } -/********************************* -* pool functions -**********************************/ - -static struct zswap_pool *__zswap_pool_current(void) -{ - struct zswap_pool *pool; - - pool = list_first_or_null_rcu(&zswap_pools, typeof(*pool), list); - WARN_ONCE(!pool && zswap_has_pool, - "%s: no page storage pool!\n", __func__); - - return pool; -} - -static struct zswap_pool *zswap_pool_current(void) -{ - assert_spin_locked(&zswap_pools_lock); - - return __zswap_pool_current(); -} - -static struct zswap_pool *zswap_pool_current_get(void) -{ - struct zswap_pool *pool; - - rcu_read_lock(); - - pool = __zswap_pool_current(); - if (!zswap_pool_get(pool)) - pool = NULL; - - rcu_read_unlock(); - - return pool; -} - -static struct zswap_pool *zswap_pool_last_get(void) -{ - struct zswap_pool *pool, *last = NULL; - - rcu_read_lock(); - - list_for_each_entry_rcu(pool, &zswap_pools, list) - last = pool; - WARN_ONCE(!last && zswap_has_pool, - "%s: no page storage pool!\n", __func__); - if (!zswap_pool_get(last)) - last = NULL; - - rcu_read_unlock(); - - return last; -} - -/* type and compressor must be null-terminated */ -static struct zswap_pool *zswap_pool_find_get(char *type, char *compressor) -{ - struct zswap_pool *pool; - - assert_spin_locked(&zswap_pools_lock); - - list_for_each_entry_rcu(pool, &zswap_pools, list) { - if (strcmp(pool->tfm_name, compressor)) - continue; - /* all zpools share the same type */ - if (strcmp(zpool_get_type(pool->zpools[0]), type)) - continue; - /* if we can't get it, it's about to be destroyed */ - if (!zswap_pool_get(pool)) - continue; - return pool; - } - - return NULL; -} - static enum lru_status shrink_memcg_cb(struct list_head *item, struct list_lru_one *l, spinlock_t *lock, void *arg) { From patchwork Tue Jan 30 01:36:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536679 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0A9FC47DA9 for ; Tue, 30 Jan 2024 01:42:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AE4386B00B6; Mon, 29 Jan 2024 20:42:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A1FB56B00B7; Mon, 29 Jan 2024 20:42:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 849136B00B8; Mon, 29 Jan 2024 20:42:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6E0C26B00B6 for ; Mon, 29 Jan 2024 20:42:32 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 4ADCEC015F for ; Tue, 30 Jan 2024 01:42:32 +0000 (UTC) X-FDA: 81734277744.05.02CF116 Received: from mail-qt1-f177.google.com (mail-qt1-f177.google.com [209.85.160.177]) by imf13.hostedemail.com (Postfix) with ESMTP id A93B120007 for ; Tue, 30 Jan 2024 01:42:30 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=P2Fal7DA; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf13.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.177 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578950; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ivn7Y9AQrDvvewOffcfUAdTab4Q3w5wTeKa9SOf1gRc=; b=JRJEf8LEhqlrGGTgohap0p2/E/XtaTlpcnIMzm3nFy3EJNqvscv90SsYiBJ5jE55v5q2AT lUqzCJLr0RJSr80zos0i7h3PeIlXz3dOf0bNNRwQEYpOXpwmKAaL5nMfHlNTjyrF9bLxc2 j5Ibv2QK4IvcoHsIP5hxkJOPjfXK0VI= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=P2Fal7DA; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf13.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.177 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578950; a=rsa-sha256; cv=none; b=tH6ebsaq8XALj5OgBSYUjp53nuQ0HZyWbed4B/CJaIrL7TYTs+8HQFoo8R4UfS+stxFECk BuEFQ9px/ozMWkereGLWDqw9Reevkc91+OLTp7xxJiO597+g2ied9Qq4fyoXMOUhUcIGuC UdHmshPmrDIMWUCaoe+mnX+R5Q/HNPM= Received: by mail-qt1-f177.google.com with SMTP id d75a77b69052e-429d7896d35so35391971cf.3 for ; Mon, 29 Jan 2024 17:42:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578950; x=1707183750; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ivn7Y9AQrDvvewOffcfUAdTab4Q3w5wTeKa9SOf1gRc=; b=P2Fal7DAZRD7ivMrPLaqdnchFCojMSt4jD/bmv4OfU+Y38yVYnjdD/74HK6x2yAuis ytvNxw9WHMC/bnYp+i7/xl8Pay61IBvb1rcG02zIEa28W83TTBB4+eBoK35/B8JdhwO5 a7EhkwOkwvZi1+KmDl5lo6mVq5XQ4g1isvZP3U1Fxmti9I4JYOBnMxBrD6VPf+ZgpIPG o645Tsr0NBHSvJdvXYwktaIrHfahEZXr6xLXzJVW7zRavSwG3lzm2/DxIn+M/aB4opDi PI1OyvdtNjCHxZYa+hupRLaZFa10hoUbfFgaBhvb8/QVU+LK8xtPYjDJtflngRbr0XqY z9AA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578950; x=1707183750; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ivn7Y9AQrDvvewOffcfUAdTab4Q3w5wTeKa9SOf1gRc=; b=C/ASQ+Ptlvd/1r2OXrLcaFgUDAnkLkNEA5B69XvgbYm/Yi54UQ2QFXkSYKjcc77k49 Zp6Vn0vq9xYPzHm3IU/669OSk90dL4pNODUIm8WJXfoRbUBZQGDpZqLFJ8DNiszUFpsy iBqTZSWY+qLz7fep47cmklCXQHM+jnxKqmLidR3KJCctNoFcDQ/fIrYIYidE7LgRwRiK 8vLtPTbbkzb8Wa99n7sR5vnv3TzNbT9IaBwLzFZyJLi7MM1B1Gxtwb08u9k7+LsRWOmN gLyAI37SujupepZ7uLBUM/rCiVMsZqC6+REeGjScWz7iAihksDd2SHocq9T3rfJyJ6k8 lsyw== X-Gm-Message-State: AOJu0Yz030huPT02moYosZ1wS8vxtWrFUJaHXBc/FwsyfW3JyX7hYMcl m0Ioag4lHYpH9bcX+9ZJ+T4uPhi0ESrhlRCv0OmREzvpaMdZVH+Xl0H1/vmlTkI= X-Google-Smtp-Source: AGHT+IFfZ8rsmJoVVOZQjiDjqNXCo9B7vayXBf6ACzYtz9VPfP8IlcslQtRqPIN1gd49Ezcam+jmmQ== X-Received: by 2002:a05:622a:1e09:b0:42a:600c:2d7e with SMTP id br9-20020a05622a1e0900b0042a600c2d7emr9275227qtb.51.1706578949706; Mon, 29 Jan 2024 17:42:29 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id cp7-20020a05622a420700b0042a6859ff3esm3078179qtb.61.2024.01.29.17.42.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:29 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 13/20] mm: zswap: function ordering: pool params Date: Mon, 29 Jan 2024 20:36:49 -0500 Message-ID: <20240130014208.565554-14-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: A93B120007 X-Stat-Signature: 34bhy91uwe8ys46p6e1tqk1f5z64gkaw X-HE-Tag: 1706578950-606909 X-HE-Meta: U2FsdGVkX19sO3Kghf42ZCjP1axUN7uc/262X+f/2b/TAGQv201THfozuUBDignNGBAqCJSHNWPqE0fCW7+A+IbzQIBR3NgHqq3tdn6IR6yskYDIKgz3uS4SrWdz3Q8oxaoj6C7+koyJfA5zvtfIq75faNKU7NFdWfeeNWLpqDS9rIHHmhnUdj/ip1l4ZkoPgRKnNgtArU5EPbfFhwPj6RUcDddWXmPg9Rq0utup6HDIl6qZnI7fLwadMPwXLbLiMKTRTKx+PUrvxLNX//7umyFU0KeUUTexNutqJ64nVbeg3dyJZ1g25Vj7umnI9J8JiINfpUkQZxugx0q/FxWGIIfUMz6LNwAC6WelypDTS4qZ+vn3n7iTmQCbQ/Ygmwwooj8GA4j6Q486Uo9YJ9tUTO5JpRJOb1OY0HDJvM3H4sYrpxyucct0tHSVQK2Xv91FWSlzJK8vbYp4mgtIm4DW7lLznrW18y4GJl6UzPCFFTdcMGVOCXp9i9xKJLuEpiW3GHvpc/TxxTp0BJM3/NtDfA7aOslasg5mT+a3S/ZLuEgDzNbe03YyQ6S5tSr2Di7WCK4q612kSZpE8pjQRXE5uyI70rtYMGVsmzTKQkcgP8A1uKOha1dVfr+0doiaDXWQktUO8qpETdUgz4BxY36LmYXStRoYNLIWtVwSMKgw97CXVCMKw+R4RLB138ceTyw04/E05bHwb1k0O69hTRBaYb1aiAptM+M1jnxx9uZAq/EwYP2SHp3p7y2A+y4nk6SuOEtD2TPbkmSaW6jLhQxyrZuUzTP46e43gsNjUI4G+BQa6znDZbQ3ToX5L8awRKYU8PPvEovKNE9YpPsQXNCq0ST5UGWsICUPpTOED16NvwLJAPcx8CGaEGzAKt9ADQXKErKSt9/70BftwFdOilGchWavV+4N9hqhmoeR9BCorEUn+rEDjt+T1UtLp/s8zPjh8HLMX6OxsxyWqjepiDb 39ycMag6 bK+Wl4a6ypxkEUPhmGIC3ns1BgRjgj2tgD2gMh5eDUOeO7RIROv083wnxeBtEGVRu+9cQtU3DYRck6I1snXaCmq/M6LhTCBPtGqXNIwLLF8Y4AXbOSOWYQaiIHAUxlwtOMPSXvWYzm4B70c7hyE3kdenYY4M8QoT52tm2oZLG7YzS10C/Q3lUL7xeFQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The parameters primarily control pool attributes. Move those operations up to the pool section. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 312 ++++++++++++++++++++++++++--------------------------- 1 file changed, 156 insertions(+), 156 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 168afd6767b3..e650fc587116 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -590,6 +590,162 @@ static struct zswap_pool *zswap_pool_find_get(char *type, char *compressor) return NULL; } +/********************************* +* param callbacks +**********************************/ + +static bool zswap_pool_changed(const char *s, const struct kernel_param *kp) +{ + /* no change required */ + if (!strcmp(s, *(char **)kp->arg) && zswap_has_pool) + return false; + return true; +} + +/* val must be a null-terminated string */ +static int __zswap_param_set(const char *val, const struct kernel_param *kp, + char *type, char *compressor) +{ + struct zswap_pool *pool, *put_pool = NULL; + char *s = strstrip((char *)val); + int ret = 0; + bool new_pool = false; + + mutex_lock(&zswap_init_lock); + switch (zswap_init_state) { + case ZSWAP_UNINIT: + /* if this is load-time (pre-init) param setting, + * don't create a pool; that's done during init. + */ + ret = param_set_charp(s, kp); + break; + case ZSWAP_INIT_SUCCEED: + new_pool = zswap_pool_changed(s, kp); + break; + case ZSWAP_INIT_FAILED: + pr_err("can't set param, initialization failed\n"); + ret = -ENODEV; + } + mutex_unlock(&zswap_init_lock); + + /* no need to create a new pool, return directly */ + if (!new_pool) + return ret; + + if (!type) { + if (!zpool_has_pool(s)) { + pr_err("zpool %s not available\n", s); + return -ENOENT; + } + type = s; + } else if (!compressor) { + if (!crypto_has_acomp(s, 0, 0)) { + pr_err("compressor %s not available\n", s); + return -ENOENT; + } + compressor = s; + } else { + WARN_ON(1); + return -EINVAL; + } + + spin_lock(&zswap_pools_lock); + + pool = zswap_pool_find_get(type, compressor); + if (pool) { + zswap_pool_debug("using existing", pool); + WARN_ON(pool == zswap_pool_current()); + list_del_rcu(&pool->list); + } + + spin_unlock(&zswap_pools_lock); + + if (!pool) + pool = zswap_pool_create(type, compressor); + + if (pool) + ret = param_set_charp(s, kp); + else + ret = -EINVAL; + + spin_lock(&zswap_pools_lock); + + if (!ret) { + put_pool = zswap_pool_current(); + list_add_rcu(&pool->list, &zswap_pools); + zswap_has_pool = true; + } else if (pool) { + /* add the possibly pre-existing pool to the end of the pools + * list; if it's new (and empty) then it'll be removed and + * destroyed by the put after we drop the lock + */ + list_add_tail_rcu(&pool->list, &zswap_pools); + put_pool = pool; + } + + spin_unlock(&zswap_pools_lock); + + if (!zswap_has_pool && !pool) { + /* if initial pool creation failed, and this pool creation also + * failed, maybe both compressor and zpool params were bad. + * Allow changing this param, so pool creation will succeed + * when the other param is changed. We already verified this + * param is ok in the zpool_has_pool() or crypto_has_acomp() + * checks above. + */ + ret = param_set_charp(s, kp); + } + + /* drop the ref from either the old current pool, + * or the new pool we failed to add + */ + if (put_pool) + zswap_pool_put(put_pool); + + return ret; +} + +static int zswap_compressor_param_set(const char *val, + const struct kernel_param *kp) +{ + return __zswap_param_set(val, kp, zswap_zpool_type, NULL); +} + +static int zswap_zpool_param_set(const char *val, + const struct kernel_param *kp) +{ + return __zswap_param_set(val, kp, NULL, zswap_compressor); +} + +static int zswap_enabled_param_set(const char *val, + const struct kernel_param *kp) +{ + int ret = -ENODEV; + + /* if this is load-time (pre-init) param setting, only set param. */ + if (system_state != SYSTEM_RUNNING) + return param_set_bool(val, kp); + + mutex_lock(&zswap_init_lock); + switch (zswap_init_state) { + case ZSWAP_UNINIT: + if (zswap_setup()) + break; + fallthrough; + case ZSWAP_INIT_SUCCEED: + if (!zswap_has_pool) + pr_err("can't enable, no pool configured\n"); + else + ret = param_set_bool(val, kp); + break; + case ZSWAP_INIT_FAILED: + pr_err("can't enable, initialization failed\n"); + } + mutex_unlock(&zswap_init_lock); + + return ret; +} + /* should be called under RCU */ #ifdef CONFIG_MEMCG static inline struct mem_cgroup *mem_cgroup_from_entry(struct zswap_entry *entry) @@ -1163,162 +1319,6 @@ static void shrink_worker(struct work_struct *w) zswap_pool_put(pool); } -/********************************* -* param callbacks -**********************************/ - -static bool zswap_pool_changed(const char *s, const struct kernel_param *kp) -{ - /* no change required */ - if (!strcmp(s, *(char **)kp->arg) && zswap_has_pool) - return false; - return true; -} - -/* val must be a null-terminated string */ -static int __zswap_param_set(const char *val, const struct kernel_param *kp, - char *type, char *compressor) -{ - struct zswap_pool *pool, *put_pool = NULL; - char *s = strstrip((char *)val); - int ret = 0; - bool new_pool = false; - - mutex_lock(&zswap_init_lock); - switch (zswap_init_state) { - case ZSWAP_UNINIT: - /* if this is load-time (pre-init) param setting, - * don't create a pool; that's done during init. - */ - ret = param_set_charp(s, kp); - break; - case ZSWAP_INIT_SUCCEED: - new_pool = zswap_pool_changed(s, kp); - break; - case ZSWAP_INIT_FAILED: - pr_err("can't set param, initialization failed\n"); - ret = -ENODEV; - } - mutex_unlock(&zswap_init_lock); - - /* no need to create a new pool, return directly */ - if (!new_pool) - return ret; - - if (!type) { - if (!zpool_has_pool(s)) { - pr_err("zpool %s not available\n", s); - return -ENOENT; - } - type = s; - } else if (!compressor) { - if (!crypto_has_acomp(s, 0, 0)) { - pr_err("compressor %s not available\n", s); - return -ENOENT; - } - compressor = s; - } else { - WARN_ON(1); - return -EINVAL; - } - - spin_lock(&zswap_pools_lock); - - pool = zswap_pool_find_get(type, compressor); - if (pool) { - zswap_pool_debug("using existing", pool); - WARN_ON(pool == zswap_pool_current()); - list_del_rcu(&pool->list); - } - - spin_unlock(&zswap_pools_lock); - - if (!pool) - pool = zswap_pool_create(type, compressor); - - if (pool) - ret = param_set_charp(s, kp); - else - ret = -EINVAL; - - spin_lock(&zswap_pools_lock); - - if (!ret) { - put_pool = zswap_pool_current(); - list_add_rcu(&pool->list, &zswap_pools); - zswap_has_pool = true; - } else if (pool) { - /* add the possibly pre-existing pool to the end of the pools - * list; if it's new (and empty) then it'll be removed and - * destroyed by the put after we drop the lock - */ - list_add_tail_rcu(&pool->list, &zswap_pools); - put_pool = pool; - } - - spin_unlock(&zswap_pools_lock); - - if (!zswap_has_pool && !pool) { - /* if initial pool creation failed, and this pool creation also - * failed, maybe both compressor and zpool params were bad. - * Allow changing this param, so pool creation will succeed - * when the other param is changed. We already verified this - * param is ok in the zpool_has_pool() or crypto_has_acomp() - * checks above. - */ - ret = param_set_charp(s, kp); - } - - /* drop the ref from either the old current pool, - * or the new pool we failed to add - */ - if (put_pool) - zswap_pool_put(put_pool); - - return ret; -} - -static int zswap_compressor_param_set(const char *val, - const struct kernel_param *kp) -{ - return __zswap_param_set(val, kp, zswap_zpool_type, NULL); -} - -static int zswap_zpool_param_set(const char *val, - const struct kernel_param *kp) -{ - return __zswap_param_set(val, kp, NULL, zswap_compressor); -} - -static int zswap_enabled_param_set(const char *val, - const struct kernel_param *kp) -{ - int ret = -ENODEV; - - /* if this is load-time (pre-init) param setting, only set param. */ - if (system_state != SYSTEM_RUNNING) - return param_set_bool(val, kp); - - mutex_lock(&zswap_init_lock); - switch (zswap_init_state) { - case ZSWAP_UNINIT: - if (zswap_setup()) - break; - fallthrough; - case ZSWAP_INIT_SUCCEED: - if (!zswap_has_pool) - pr_err("can't enable, no pool configured\n"); - else - ret = param_set_bool(val, kp); - break; - case ZSWAP_INIT_FAILED: - pr_err("can't enable, initialization failed\n"); - } - mutex_unlock(&zswap_init_lock); - - return ret; -} - static bool zswap_compress(struct folio *folio, struct zswap_entry *entry) { struct crypto_acomp_ctx *acomp_ctx; From patchwork Tue Jan 30 01:36:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536680 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 936B2C47DA9 for ; Tue, 30 Jan 2024 01:42:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DF0606B00B7; Mon, 29 Jan 2024 20:42:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D9E506B00B8; Mon, 29 Jan 2024 20:42:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C3FB06B00B9; Mon, 29 Jan 2024 20:42:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B16A36B00B7 for ; Mon, 29 Jan 2024 20:42:33 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 8A3F9C0A8A for ; Tue, 30 Jan 2024 01:42:33 +0000 (UTC) X-FDA: 81734277786.25.9769495 Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) by imf04.hostedemail.com (Postfix) with ESMTP id C6E394000B for ; Tue, 30 Jan 2024 01:42:31 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=v9cIJeci; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf04.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.176 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578951; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IOr65lcwE97/+IL040qC2nQXATF0TLQoSMjJ/TctotE=; b=xLQUvuJI0iQkd/WLwwVvUA+JECQk8gUESOkiabPTdG2smLcoDZSy5kmtHoanr1czSxI05X bKAGb37xfxy2U39ubkNOzufYy075+9w2QyHqvKJMAMjgY0V+Ga5rcH/d46/O7tTB3lboSh 1lVebWsouxwp85KsIiyyJ8MkmcMj5oI= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=v9cIJeci; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf04.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.176 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578951; a=rsa-sha256; cv=none; b=YjMvtSUqpho4FkWKV6JkRImC9UB5UZCEykWzyM6ccerElRs7LLsllJn1xkyj6fbXyc1kTQ ENUAcZo2mDL8WcFJ9l37HdhYM8N+11SAv4VXl6xp3s8hhl1R9jkbYMEnt3zJGSWGK1+jXK ZXVh4Fo5/olGWNe0w8i2utVK6i5qitg= Received: by mail-qk1-f176.google.com with SMTP id af79cd13be357-783045e88a6so349543785a.0 for ; Mon, 29 Jan 2024 17:42:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578951; x=1707183751; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IOr65lcwE97/+IL040qC2nQXATF0TLQoSMjJ/TctotE=; b=v9cIJecigJKKCTm6SOKkmZu7hSrHnwJLFaR+6kxHml0hbv4KybYsVknpqZgk87ZNJ5 VSY3W9HHOAWBBP0aSm2/l6AhXiCtmjtWeWhxzfUzxC1uWoF3gcc2TV+4jXofJtqKjz+4 sWMqX0D1TsHQ1D8Y/OAM8lid6eazKlmGGljmWC7W1OJYZ7r2lP2IWEcQLo21QC6yx9C2 ebBaiSYikAWgd3QiUbw3cKIiCkQT+ibkXcvC4+cVC7iiNtQJQnUg5UGmwUt6HK0Smhdi Gi35yAhu0ZfLqZhrwQ1fHMDi3iwQRwGmwi+bVlcB+wPDiJRiopgypA2r6wY1nbYNWCWx emnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578951; x=1707183751; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IOr65lcwE97/+IL040qC2nQXATF0TLQoSMjJ/TctotE=; b=Ld5XsdUPCr/ZXMtStOYJux5UK7ny5WME+AnpgCERh8Rq6Hkl+f1AUVJYlUxzIHD+92 KgOOtLUo1860q+sDRx79nCFi1pJZbQ53XL9RNN4HomBS9fr/YdGV0laugd0RUMrcVf/U 6cZHoEHNr3Xk8zyNvbzdHZ541i9vd6nyEP6mzlplfyl/Mj732J/loqEAIt6Gn2/npBgQ YOXIn+dq4skXjgvT7vfToRMDpJsafU8YkcQxs1JyaDzDnMEAEulIhUdQL1X4rUmzba5x roR92F8/OdbdV2kf6mT9CCiFA71MOI6GkwNq5F7I6RIuQzbU+q/0LqNGEbkAfvJ18tCf OjJw== X-Gm-Message-State: AOJu0YwYKl8MJU+/bkY/3iZZyKMmV71XisxAAFQa00roUNuHF+lRAJ97 c82qRmlJFUudOPY5laLDBGf9f69OZPDqpIQl7CFENU8M0P8ozHGQ2iO3IcOInn0= X-Google-Smtp-Source: AGHT+IGBNUuoDAQL0pkLCMyHWCaO2KYdN3eRRjbq+JcAUmpZl/6o0Vki6wyVQPOrxoVvBtwDFwQh9g== X-Received: by 2002:a05:6214:20ce:b0:67a:a721:b1b5 with SMTP id 14-20020a05621420ce00b0067aa721b1b5mr9104092qve.112.1706578951030; Mon, 29 Jan 2024 17:42:31 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id b6-20020a05620a0f8600b007832575779esm3580991qkn.52.2024.01.29.17.42.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:30 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 14/20] mm: zswap: function ordering: public lru api Date: Mon, 29 Jan 2024 20:36:50 -0500 Message-ID: <20240130014208.565554-15-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: C6E394000B X-Stat-Signature: bo7ebg179t9b34j447o8tha93c4eo7j1 X-HE-Tag: 1706578951-242369 X-HE-Meta: U2FsdGVkX1/me+QWp8OElBdpxM5/jHV4t2GQ6VBzYuSBcFB6x0JL0g6vpHnK9d/JbkczEf3DCKfw45ZXrbRZpjJN52ZeoZiLAoGPmhNKt2DmY0AmvC4nhkN0Oq0JrIyK+mtyyyxxcnQTXblYvje2nV5ZYHYz8rev6Ftqsh8HNAvsMW+Ozd7A904+8cLssH2DYgwVY3+YZpimbK7HNpPtCi46cBQt5iobBMjP65hHDPYgDU2I3LWVCYaEZI1gfql5ZKVD2QC5znlN1Jwyn34tWybRte8S1yhocbUuZY8i+eQhra2mdtS+NE5eBppmjWBd9Cx/DjGskI+7eecF/0jgY345UBZmUijeC0D1/FbcfF7eXyJb957Kt3obOdH+E4P0BNfq5+k6VWdPKhH5mO7aow/QTUnTveaTqJN4sEl/JajpvL3q6hkH/gZcK5Sl9SDbcgtx1sfj6/VNUK/iPqqX9w7BEf6oJ6olzqqVguJYSN3Ey09yTr8Fx1MWA35c846OcNgtfB+uE10PhIAGgi2fiwD6qT8mz4ijJy2cc8hbfMHlS99r0qWykOEe8INH5WHDJ5kdexmWuDK0tlIPfbPEpnRgtoSXgJTzpFg/EspSKt1Nkl22R7DOiWOgBo70SX8TQNnbogeaFk55CZvZA0QCuQC+cjCaE9zjdhnzX9Kda2Y78HOKVqnEybkz1GBwC01C7jnNHp5iG1dZxMRQMmQojMR2D8CWHJ8wxy20A29gja3eq2fsDRAC4QF6xPpbX4vsR5IaAWTxQ3/fKOmyJn9JzYlJaxctSyMKKqz29KlBw0f8vdoBKa4Y6dHLxN2ZXQ5LFusYRaXelcvRNlR67JdO8y7nxCgV/dwGTeAL+7ltz4XsN3R1YmTD5o6Tc40EQNrXLrDf4hXXvsCCrfCKw6dp4oI125Qk7HiAgfQiBz2RqVmOg/8VzVzQTRByVRtz4XHDTDIc+9hyWA47zOECrnk AFXBcDII 6wYcZoAl+bYNPP0UljfSyqCkCxAKzOyzMNzS/WT7D7ELrudEe6Bb1gXrXergp/ciU7Ws0zxT57fE90eZeD9x37jypBbklWLIc2uU41yvM27cIo/vWeMk4pyBerRUXy7QcJnWJF1RLDTIBcKY9s5RI1Un2QRfZSQTs/TSSt9RRBY334yP/dxSx6GjOBwh7Cr3R9VSW+ZZb6MbEj2qqdR8nwzQgUoi2R+ihf8d7nDZXvegLwfYnS9QRgdfTGOQf9Q4rZ8JYx7ZMSQnw62m4lY1Im6cPCUJaaQ+azGHCxxgBwuhCF4XLIHGWttiayuATpYXxTgehmmOrmJmDZH5RK5m2tMW9ify85K4keFL/6CR1DpjZvA9jUck9GVeNA8T7JDXYuz8LiSzX+yaVH21WU8zkHXKsYwTMHDP+z95r7d+j2dsqu1YQTcLivEpGg2jqKvRc6Ehm X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The zswap entry section sits awkwardly in the middle of LRU-related functions. Group the external LRU API functions first. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 37 +++++++++++++++++++------------------ 1 file changed, 19 insertions(+), 18 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index e650fc587116..511bfafc1456 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -746,6 +746,10 @@ static int zswap_enabled_param_set(const char *val, return ret; } +/********************************* +* lru functions +**********************************/ + /* should be called under RCU */ #ifdef CONFIG_MEMCG static inline struct mem_cgroup *mem_cgroup_from_entry(struct zswap_entry *entry) @@ -764,6 +768,21 @@ static inline int entry_to_nid(struct zswap_entry *entry) return page_to_nid(virt_to_page(entry)); } +void zswap_lruvec_state_init(struct lruvec *lruvec) +{ + atomic_long_set(&lruvec->zswap_lruvec_state.nr_zswap_protected, 0); +} + +void zswap_folio_swapin(struct folio *folio) +{ + struct lruvec *lruvec; + + if (folio) { + lruvec = folio_lruvec(folio); + atomic_long_inc(&lruvec->zswap_lruvec_state.nr_zswap_protected); + } +} + void zswap_memcg_offline_cleanup(struct mem_cgroup *memcg) { struct zswap_pool *pool; @@ -798,24 +817,6 @@ static void zswap_entry_cache_free(struct zswap_entry *entry) kmem_cache_free(zswap_entry_cache, entry); } -/********************************* -* zswap lruvec functions -**********************************/ -void zswap_lruvec_state_init(struct lruvec *lruvec) -{ - atomic_long_set(&lruvec->zswap_lruvec_state.nr_zswap_protected, 0); -} - -void zswap_folio_swapin(struct folio *folio) -{ - struct lruvec *lruvec; - - if (folio) { - lruvec = folio_lruvec(folio); - atomic_long_inc(&lruvec->zswap_lruvec_state.nr_zswap_protected); - } -} - /********************************* * lru functions **********************************/ From patchwork Tue Jan 30 01:36:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536681 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23D02C47DB3 for ; Tue, 30 Jan 2024 01:42:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2F6086B00BA; Mon, 29 Jan 2024 20:42:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2A43E6B00BC; Mon, 29 Jan 2024 20:42:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 144866B00BB; Mon, 29 Jan 2024 20:42:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id F0A6C6B00B9 for ; Mon, 29 Jan 2024 20:42:35 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id BEDFEA09B3 for ; Tue, 30 Jan 2024 01:42:35 +0000 (UTC) X-FDA: 81734277870.12.7FB1A35 Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com [209.85.222.181]) by imf02.hostedemail.com (Postfix) with ESMTP id 063B580022 for ; Tue, 30 Jan 2024 01:42:32 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=lwBD+DaL; spf=pass (imf02.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.181 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578953; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yxxBC8E+iSnomrO4M1Zu6Eym4Wq/S4pyguWdd0McoRg=; b=kN0lJuJ5reaGbSzqGY2pZJbI4FoP1O/R8+iDhfs1ePo1Q64tyeZom8E9zYJB5fo30LtoP1 g7jIgGkxDvLOpk1MR7LFB1t4MVJYOcJNQpg9E2n5F0r4grUlWh51+T9en2EfEVesji4Yly 5Cq6TefTl3CM+4wfmGd0ghmxHHasfIE= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=lwBD+DaL; spf=pass (imf02.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.181 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578953; a=rsa-sha256; cv=none; b=Cl9EoxFv8WwqevJkk+HQ/aez+jNMB7y8bMxD6smhV1hfPaTAJFyQq2HfJUhtecEWcSxTkE PKTH+brFoLt1ESlstNpESLn8Ew2dnyuO1LLUh00cqzqTSvYwR+CToU4WXgYnOExervI47u Gbh6TXG4EI1dIaLub8Glr0QXLlbufHY= Received: by mail-qk1-f181.google.com with SMTP id af79cd13be357-783ced12f9bso236068285a.3 for ; Mon, 29 Jan 2024 17:42:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578952; x=1707183752; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yxxBC8E+iSnomrO4M1Zu6Eym4Wq/S4pyguWdd0McoRg=; b=lwBD+DaLsNagwEQiD1x7nPZl9Z7uM8R+ctk0GVq437TjSsMD5K1rOpJo5QNudPYqxz FrJWWsrOS2T1WbhZTCKjWd0pToZ1wxY7PGfRTnTxDcCmqHcWq3Y0c1x7vcpKPLJucOSx FTpe+P3z7xV0c64fuE2TbT/dIq+X4MsXJdKxuODUbfL644wIbp7d+mOG6Hn1vWbk9Qeg De9NnK7qI/ZsOyMl1hpAzBfgzAsuGZYus6QtreaQblA58BiuwoFrUqarEsoVadDG4znT 83r9oTFZJbFB5uB+1+5wUXLod2NwPPbfpaNq8a+daQrkbUvj9Rr40B1H4HhHsFGkd9Ed vnIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578952; x=1707183752; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yxxBC8E+iSnomrO4M1Zu6Eym4Wq/S4pyguWdd0McoRg=; b=P7sI4UQymuUi/LocesBTafTKv0uyaNB/4AJew4ji0arsfQFJG/MLSGp4VGlEX+Kw2v Tvz5SgjJhK3JNGYHulY2FbiS7J7Q/zI9kulFXIBBv7mAHfCIpl1AoNjffJ1rkhVHT4QP NouhcfFvgY7PdBF1b1Wk7rlib7KN3ako3FBQX8ewVlBLsDv8d1yWGoq+uqlnbq94oDmQ +sgds5Myb4PJ1XsDU7jAu9vM7BZixUa+qVFwNZd7uo87iwPHiC01Px7AqCpESEPu51Nt jX6+jqlyRl+2eLUk4tsIxfezBONEGyqftCtH1SUAdAHoshlYIIV63lBY7+6ZOjYLB3Yq 9/iA== X-Gm-Message-State: AOJu0YzdGYDXBOS0spo74rKeRwdYywz48igSm4v3im3R1A4e02lEQ/8K NbW9jhciNCF3Zg17Vc8WWMjRUjNT6kwUxXuWcCQIXYnBDNvAy9zLzBIPS2VfL1M= X-Google-Smtp-Source: AGHT+IH0Ej2IVjNTByv8ncwfD8JySniOOvwcT3ZZLF3iZpkW+vzHER0j5wJzpzxzw9guVH1KYf0/nw== X-Received: by 2002:a05:6214:2681:b0:681:98bd:644 with SMTP id gm1-20020a056214268100b0068198bd0644mr8490769qvb.119.1706578952162; Mon, 29 Jan 2024 17:42:32 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id ow9-20020a0562143f8900b0068692ea038esm3974062qvb.91.2024.01.29.17.42.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:31 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 15/20] mm: zswap: function ordering: move entry sections out of LRU section Date: Mon, 29 Jan 2024 20:36:51 -0500 Message-ID: <20240130014208.565554-16-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 063B580022 X-Rspam-User: X-Stat-Signature: f1hwp77qnocjyj53uowq3t93q1dyhxtn X-Rspamd-Server: rspam01 X-HE-Tag: 1706578952-113219 X-HE-Meta: U2FsdGVkX1/u5qULEJ7dk/bm2bu0JhCmuBJkANmSUmmhP06bEUGH1CvZOKrCgOleBH5S75+kcPNN98qDyIDKFKjelKHS9p1R8bTj5kvYa1eXubf962PGWAZQugbFFoKzopPJ59ys7HDW294F+Lll/gTIaj7u0YIR5ZIfK+aVMzLziVIjQe9ljfF403eeMEwpP+7oUZPtd+pU0kM2HdB6tj0gdJwZFL130ut7NqaUM29pWpArTcXuuXlq68JVazUbDSNsje2ljOqq/2nDNuypoTplphVCDvVTXyUksqwBFzILQpmiPmHY+mStKBST6g8G/h/M+OF6eulJXzEnGmSYSvXMko7oySneMTAu2/V3R7vuXTMfYBgRWD2AwpvLQwHSxoDETnbcI4uoqmBHf9tTKRsHdH5JzTnHt4SSTyrPINmbQ3heO5UmNMqeEF72VUOLYad6H8uhkcGpCqfzqJed5A8BCz2GqF7wkQEaIumwT2Il/070WNYBSVOx3CVj+BAFmy7L4wDZZx5vvYeXUT3XHWv6pZULAl2sG2qSiDQS6dM3T3YO3ogfVR4pr8+66fWEVtjZ/2ssSXkCBGaMu5NJ5N1K+CNRfQQN/M5MWHssCqS4mzA/sh63zVx+6536dlSCuHBXveuPCB0h7mvBd8FqzRj8qNhfV6SS9qh8ClOE1pmF4ihlsOyA0D7jOXbqSwf4ZrrTPOL5jurVjjZk+5OwdHR+9f79+vExi9H3qgBYuiNvs8rMAWtiPQB/3yRhe6MF8oR1XfBVIfx8UKtWhIDSSziKOmO24elvCk7vGIc9gBhLhQV5KlBr7ve1yjqLMJXNTl6HOsM1lzvkEz/lrs002xNW+D0MPF8xb3DxrHpqKw5v3pAedbhKBeuJ/x+ghWJz4L11/hcUyIRIaK0e86kFSWs1tfbfyj6z2ROIXCwGrFtnYKpTFc+e9ZsDbhTuwSLrIiz0RpxbdXd46zMKh7R TyGAPDAc 8DHcE6SJiM8YdjfH/YUoSkItKdvsleNzp3puJvmMZ6k09UEnrvTqzBUMuE3XytC6a/qCr2lg2dytem7N8SuXYWnv4kC3xihWu1/YOnGgB2sJrrlRo5EHxJq+TYWc0HrvMhTGjUyuciUE28YCybih+YYUf2gp7ZoJvgufxntO8pAVOd3BSeHU4PMnyVa+hjC5jTCCoO49vxW3tTX7IILEix7YANz9RWpd67y0BI2TAf7RIypzICPjMxlMjd6w4mj99dKD9KYEiRbu3cM7AmLCC8jFFqcpHcI/DsfVPUK5XgOgD0HONeP+WWkxKO4Ugzz6Hmty/fokqJ8Mab6hUDkGKsHDzr7P4TR8jHkpVllF7hFU2KZzZD6dChvAi1Yg2HvpCO7K836hf444PArZmdaTiTsnxeuCcRSRAZ8YiqUlVC6rpJ/HQSpkHt9NYYcxEHH2131kA X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This completes consolidation of the LRU section. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 101 ++++++++++++++++++++++++++--------------------------- 1 file changed, 49 insertions(+), 52 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 511bfafc1456..756d4d575efe 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -768,58 +768,6 @@ static inline int entry_to_nid(struct zswap_entry *entry) return page_to_nid(virt_to_page(entry)); } -void zswap_lruvec_state_init(struct lruvec *lruvec) -{ - atomic_long_set(&lruvec->zswap_lruvec_state.nr_zswap_protected, 0); -} - -void zswap_folio_swapin(struct folio *folio) -{ - struct lruvec *lruvec; - - if (folio) { - lruvec = folio_lruvec(folio); - atomic_long_inc(&lruvec->zswap_lruvec_state.nr_zswap_protected); - } -} - -void zswap_memcg_offline_cleanup(struct mem_cgroup *memcg) -{ - struct zswap_pool *pool; - - /* lock out zswap pools list modification */ - spin_lock(&zswap_pools_lock); - list_for_each_entry(pool, &zswap_pools, list) { - if (pool->next_shrink == memcg) - pool->next_shrink = mem_cgroup_iter(NULL, pool->next_shrink, NULL); - } - spin_unlock(&zswap_pools_lock); -} - -/********************************* -* zswap entry functions -**********************************/ -static struct kmem_cache *zswap_entry_cache; - -static struct zswap_entry *zswap_entry_cache_alloc(gfp_t gfp, int nid) -{ - struct zswap_entry *entry; - entry = kmem_cache_alloc_node(zswap_entry_cache, gfp, nid); - if (!entry) - return NULL; - entry->refcount = 1; - RB_CLEAR_NODE(&entry->rbnode); - return entry; -} - -static void zswap_entry_cache_free(struct zswap_entry *entry) -{ - kmem_cache_free(zswap_entry_cache, entry); -} - -/********************************* -* lru functions -**********************************/ static void zswap_lru_add(struct list_lru *list_lru, struct zswap_entry *entry) { atomic_long_t *nr_zswap_protected; @@ -872,6 +820,55 @@ static void zswap_lru_del(struct list_lru *list_lru, struct zswap_entry *entry) rcu_read_unlock(); } +void zswap_lruvec_state_init(struct lruvec *lruvec) +{ + atomic_long_set(&lruvec->zswap_lruvec_state.nr_zswap_protected, 0); +} + +void zswap_folio_swapin(struct folio *folio) +{ + struct lruvec *lruvec; + + if (folio) { + lruvec = folio_lruvec(folio); + atomic_long_inc(&lruvec->zswap_lruvec_state.nr_zswap_protected); + } +} + +void zswap_memcg_offline_cleanup(struct mem_cgroup *memcg) +{ + struct zswap_pool *pool; + + /* lock out zswap pools list modification */ + spin_lock(&zswap_pools_lock); + list_for_each_entry(pool, &zswap_pools, list) { + if (pool->next_shrink == memcg) + pool->next_shrink = mem_cgroup_iter(NULL, pool->next_shrink, NULL); + } + spin_unlock(&zswap_pools_lock); +} + +/********************************* +* zswap entry functions +**********************************/ +static struct kmem_cache *zswap_entry_cache; + +static struct zswap_entry *zswap_entry_cache_alloc(gfp_t gfp, int nid) +{ + struct zswap_entry *entry; + entry = kmem_cache_alloc_node(zswap_entry_cache, gfp, nid); + if (!entry) + return NULL; + entry->refcount = 1; + RB_CLEAR_NODE(&entry->rbnode); + return entry; +} + +static void zswap_entry_cache_free(struct zswap_entry *entry) +{ + kmem_cache_free(zswap_entry_cache, entry); +} + /********************************* * rbtree functions **********************************/ From patchwork Tue Jan 30 01:36:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536682 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 604E3C47DA9 for ; Tue, 30 Jan 2024 01:42:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 711ED6B00B9; Mon, 29 Jan 2024 20:42:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6C6206B00BC; Mon, 29 Jan 2024 20:42:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 381C06B00B9; Mon, 29 Jan 2024 20:42:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 2191D6B00B9 for ; Mon, 29 Jan 2024 20:42:36 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id E98E580738 for ; Tue, 30 Jan 2024 01:42:35 +0000 (UTC) X-FDA: 81734277870.01.91FA754 Received: from mail-qt1-f178.google.com (mail-qt1-f178.google.com [209.85.160.178]) by imf28.hostedemail.com (Postfix) with ESMTP id 3A676C000A for ; Tue, 30 Jan 2024 01:42:34 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=ZUX1zZBM; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf28.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.178 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578954; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zBr/4NzQcxrA4rNO53TUlaG7d5zRLMT2JlV2BMAMPQs=; b=h7Ufz9WCrfiv9YYPAqejoqw9ACvcrzReSSkSyLUnRBJzPlD8vWRQ3KZwevOueKcCvSSECQ HbxoT0SKN1LRHk8zTIMGXBEZ2sBSRkipOOD+hAIWFJZNgnDy9HIcfRBPiuVFa0OxLzIq5f U7YAJQ74O2VE4gzD9jCg9oLBSRKqHic= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=ZUX1zZBM; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf28.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.178 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578954; a=rsa-sha256; cv=none; b=7Hl5AoBFqZG1GmeNYEt7LwmqYNFNz0ef20trZ2oI1lykz6gpIKeyWr3AZkrR4ZkXsGj2Oe Wv5/jT++fbQbp32rOUpzRCwPU/aKGQkMIhfDOO6n4Ps4CbwV0EPjbSvBDpo7XytAVd5g8Y /E8aPv91H5rClqNOZbzaUAwLmz0PH38= Received: by mail-qt1-f178.google.com with SMTP id d75a77b69052e-42a9f4935a6so12530651cf.1 for ; Mon, 29 Jan 2024 17:42:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578953; x=1707183753; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zBr/4NzQcxrA4rNO53TUlaG7d5zRLMT2JlV2BMAMPQs=; b=ZUX1zZBMYoQTsCXQ+ZyEAPb+H0ydTt/vd4Sh5a5NKAc6cJ+iFErnInAg5NQJpOmRGp DCF4TSEkVnJrmthTBwVlDvmIaoG0OSryMi/Ac9momNPhcbYcd4MexKIhuKA/HeNcKZ1K PGqCx+uuqWa1RQo2JMPLJb/R/PQavTi0kpaNPbvwB7+POv29jx04ddLaJ/gvJ6bl414y YhFV8iT8jtKUQPOlJpCM1Pwct/8bI2oQw2euxVPg1DYViLGDSBqrcSksOIcvuWt74vE/ GNxvoT3Fzfm2SY5iSZSinVl65fZK1Fjp70CblB2NQqyROndH7V4F52H6APl5GAQtjlSO xFgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578953; x=1707183753; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zBr/4NzQcxrA4rNO53TUlaG7d5zRLMT2JlV2BMAMPQs=; b=A2lm7SuVCcKt140GZkqN5DRWle1Qfg5X9aKrvkQUpLMkG+K9dzQaBdSx99dXopvO6b N2HOHI0y1jZvb40QlniDrDfK2OBSO4OdHjvjN2qyaP4Q61EhNR/9c158091LhYfXE7y1 kq5lwyBL4Q/uZv3nGDbKXiVbpX4iLQuPlerhysdNqdlGKKC7t13qB682zL/Y/aef26FK jRms59fDTaK5gnGW0e1L8TF8pYlmc5hHBq0Xsfu/j7lD1IKxzv+bLPE5uCKEu1t6EFX9 PPiYNNKKnMOGREJV6RvqkBbww4YjXI3masDRCSDGE0on9jRvQrpj/1GV8rY55d2JeMfr pgxQ== X-Gm-Message-State: AOJu0YzLKaGV4AcoMvLqlyAAW+hGjn1XS+5bz6AyxV387By3pMiflNVL OibbHZXxYZ1O4MjecflnQrxqxt2EKnBP0mpHo4tv+ZTyAfb3Xx3DxoSshAvtOzA= X-Google-Smtp-Source: AGHT+IFnFy8t74gPrfD3nUEsIc6ZJrr+zQk6hXnBVgVJPyD5DXHBfSOXtvC3SYSy8pD3LO1UcyYHNA== X-Received: by 2002:ac8:59cc:0:b0:42a:ad25:21a2 with SMTP id f12-20020ac859cc000000b0042aad2521a2mr340871qtf.16.1706578953364; Mon, 29 Jan 2024 17:42:33 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id bb36-20020a05622a1b2400b0042ab4a129easm236641qtb.73.2024.01.29.17.42.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:33 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 16/20] mm: zswap: function ordering: move entry section out of tree section Date: Mon, 29 Jan 2024 20:36:52 -0500 Message-ID: <20240130014208.565554-17-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 3A676C000A X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 4u3f1fj7qu5tnwi4epfyt5ogjwe9gs3u X-HE-Tag: 1706578954-923695 X-HE-Meta: U2FsdGVkX19gA4m9uqubBk5chlquqWJo6sR1u4bUZqbjt0eKYgTf4UFPYpWdylKZpjjPnfzucQ7Uv21oInEzGnmTszBZTI2jW1pVFmh/hczi+Z++hMBqICn73VF9qmU4yuiNDjuKtqdSSp5BT/QpIp0Igzz+vLBf+ilpGO7gxF+4/BkxKSbdRxKBT+CJXA6T1574ERvlaLVDoeuTD8lYR2uwo/bXXNls7IWU9jSR1tCUV+xl9RNk1vzfnlo2h4DUctD07CJc6yWnsLSSeMiSgtUT3hYI6pouH88BrR3zCzw8tFJ+DcgF7Wdu7zYN/36J+YuHtqS5jDxpu5ASL9aJXxGY7FFouUynqGKmZsWxcyFAKUlrB57o6wETFMR5+Gq1bibA7C0pbNWzDbfMkRj0b1NR/Oa0xrBLjRBRB9K8QoLui3JX/ZOkK8bDuGPQr83hyi+P3QuzXWAOU0aw51llQJPa3bdqeEr8mMVf6o7oIHagt/s14htw/NDkTl08EKLXOlkl2iswR0Ybwhke/gfwESMqkgBhrrixWLq1CIjFz985HDiXNU6dWC7pXBGEXpPPuuEnRkiUOq1GLZxZ/KwLMMJmpx6edSPS77ZlTZbHF79IEcHvg1i+GfvPj45KvqZ+gTgYZHzLmy2iT0VPY4QxxwNc8R8XhZVwHqsgLUiUwUmqLojYYOUGv5PR6qZ8o2LEhvMY24M4mgvP0Eg+KTh4E1KiIil0YZJ5FVwOO05u3X+FLdF04y6RsR/Uyhvqu4GYVpvJMG+ZDH+p9YfuKZh8aljdf7xOTqUdSWUh9DfvWdaeOzuvH4PlDNldfBHmhgKs99pYrjDx1wVMLu6bvmQh2f3s502KcyXYzZYd9OTrCG4YQRDMKP+LKiXy2ggWURy6HYDlJng3TLDqFA1qXWll7V756I3WJapNuN4MXlOUR7erRbO1NqkPomDlpPESCf1JSlmLhf4ol2pFGTGtK68 0/dt2qEN 6jJOLL945a2jRdM3DAdDWzV6BgjqIQZVozX3F7wOnqLe7MOPZ1xavQldZhkPRoad378XbbBoJtDpxi4Ci4xsfVU6fdEocaKSmQInC+L7iashD6FxrA72PFJ2sXIwVBk4SPxsmKAbpkQyhnC0k2PHPWWkChGyNmg15DksD/GChoZL5NsJ9fDw2aiTWBEWVOCSDpxgJrPyl2fVOPvSmUpgUPJj8bO09sZVVAH8lyDqremxMBAFbWI6z9BVwhGv9NzOl95aC8bFuXalt0xlBdchhm45Fo3qObJC8fZUrIVAx2wpFlJhvJ3Qw/cTmXyodpVZI+nSQ8WlO9GN7V5Y6fPYrO5GVG07Uu6fwQVP2XbP0CegJlWUm7gIvrDLGiPGUlSdfDrU3Vo3kkKh12QPhiLM6eZjTUYkgTr6MI3wP6aiAgrB1+ZkFST+tvFhQxg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The higher-level entry operations modify the tree, so move the entry API after the tree section. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 42 +++++++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 756d4d575efe..80adc2f7d1a2 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -848,27 +848,6 @@ void zswap_memcg_offline_cleanup(struct mem_cgroup *memcg) spin_unlock(&zswap_pools_lock); } -/********************************* -* zswap entry functions -**********************************/ -static struct kmem_cache *zswap_entry_cache; - -static struct zswap_entry *zswap_entry_cache_alloc(gfp_t gfp, int nid) -{ - struct zswap_entry *entry; - entry = kmem_cache_alloc_node(zswap_entry_cache, gfp, nid); - if (!entry) - return NULL; - entry->refcount = 1; - RB_CLEAR_NODE(&entry->rbnode); - return entry; -} - -static void zswap_entry_cache_free(struct zswap_entry *entry) -{ - kmem_cache_free(zswap_entry_cache, entry); -} - /********************************* * rbtree functions **********************************/ @@ -930,6 +909,27 @@ static bool zswap_rb_erase(struct rb_root *root, struct zswap_entry *entry) return false; } +/********************************* +* zswap entry functions +**********************************/ +static struct kmem_cache *zswap_entry_cache; + +static struct zswap_entry *zswap_entry_cache_alloc(gfp_t gfp, int nid) +{ + struct zswap_entry *entry; + entry = kmem_cache_alloc_node(zswap_entry_cache, gfp, nid); + if (!entry) + return NULL; + entry->refcount = 1; + RB_CLEAR_NODE(&entry->rbnode); + return entry; +} + +static void zswap_entry_cache_free(struct zswap_entry *entry) +{ + kmem_cache_free(zswap_entry_cache, entry); +} + static struct zpool *zswap_find_zpool(struct zswap_entry *entry) { int i = 0; From patchwork Tue Jan 30 01:36:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536683 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2982C47DB3 for ; Tue, 30 Jan 2024 01:43:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7B0366B00BC; Mon, 29 Jan 2024 20:42:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 738F76B00BD; Mon, 29 Jan 2024 20:42:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 589906B00BE; Mon, 29 Jan 2024 20:42:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 41BEF6B00BC for ; Mon, 29 Jan 2024 20:42:37 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 1ECA41A0A02 for ; Tue, 30 Jan 2024 01:42:37 +0000 (UTC) X-FDA: 81734277954.29.D4D9181 Received: from mail-qv1-f49.google.com (mail-qv1-f49.google.com [209.85.219.49]) by imf11.hostedemail.com (Postfix) with ESMTP id 57E674000E for ; Tue, 30 Jan 2024 01:42:35 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=sNnnWUVe; spf=pass (imf11.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.49 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578955; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=F1iB6scB+xehMKrtYira4F0YAgxikUXsHI43Fnlp9WY=; b=bDwiAIxlEueCjKb2tC6xBIZuwGk3VSRavEdtWsYmEd7FB0WwH6t1o/GfoJHYTAI/5p/EBo azU3m1beLKURV1Q3Yc5vFyWK2i+6ltLz6N2HtN6XWaJhnpe7+NEmUVCWN+RjWPr3Tw+kTh qrxlim4Fg/QFWxVHCLsPLY0SGApolFM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578955; a=rsa-sha256; cv=none; b=KZ+gHTPZruOtot1hvmOKQnpx5pjfGrnTDOTRylTReFJ8boL6K9R8v2Fe5VE27QWZ+ISet6 dIynrE86Qa0eh5FdaQlKjGBEJY3wYNdEeCGrEZOrB/HqcSZXRHXCevgxX3Ge4WaTeC8GKv QG2607uy6VpVYOlvOYNZ67xCQOiO4Ek= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=sNnnWUVe; spf=pass (imf11.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.49 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org Received: by mail-qv1-f49.google.com with SMTP id 6a1803df08f44-68c2f4c3282so26793026d6.3 for ; Mon, 29 Jan 2024 17:42:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578954; x=1707183754; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=F1iB6scB+xehMKrtYira4F0YAgxikUXsHI43Fnlp9WY=; b=sNnnWUVesOLl0H8VFn6CKm9Ud3+iOKTF7XqGksktxJAt6F3VC8TCBmilMmoR5Jms5l zVgUVTBlD2RUFa28yfLIimfHUx7FSuwUIQo5blEYUg15gVfLCYgUgsDCowcNRFSSFzk4 JNzMs0XfyVd5wyKW2OpDjd0M+YMk0fe+DAkVVjK1VFeS3F0tmRrP+KUKnVTkS1+FQVgQ Y5BJ1kvlmgbrsnXDiZiRpBz4ZkRqy+FO7hgia71VKBqAAumstNu9Ot+eOCatERupZPOx mJhrqc5h+mZENoQ8gASWUGoiZOVCEx5yUIWfQifVyfvQ/KtoUVfFIhEqiPlr1SsPzVyf DqsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578954; x=1707183754; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=F1iB6scB+xehMKrtYira4F0YAgxikUXsHI43Fnlp9WY=; b=iwD9yPz/fcY1t/g37hw0l07H4LYN1wkcaEhYELvqzKBtdoo/mrlx0TKiJ06giiRcHb 2MoLdOddF2YNEmfKWTlz4SUlK8L9Em4QSk2TU3+tGTo+YMSpZfNmEBuImyJgThX7eb9n Vaw/Rj1onPcRDcuJCRpjF14DDAjnZrpT40NN9t65Ty6+pT3AEAp37Peu9OF8R3UVcdHV 6qAVYr4pNR7i7OMdu5H1QlHEVrbSRChxYLw3X3yMe+ETWSJnaYz14dJ8e323KHQhdpBN P8I9+lEcvqvziaWAcpBtmhGdBGUDkQ9gnlgbbjf6q6UnLOdo8KEp/zCdrRbJQSNeqx+h 9MLA== X-Gm-Message-State: AOJu0YxnYHfZLfcm3sUGzBmkw0lYYiUnwALcBZfsQkUCgr/syQGwE88s 7fsIGgBDacEfyNZfX1NTY5d9QPtlT10QceyF9v/oeBGmT3LHeqUvQVm5BLxg1ZQ= X-Google-Smtp-Source: AGHT+IGzesXTSc5km00kKTgaDrZtfj8iYTZdT/vrOqGJQTpSYvP8oFDnRJlKxNfj+JGUQtlQVzjEvg== X-Received: by 2002:a05:6214:628:b0:681:5800:9f23 with SMTP id a8-20020a056214062800b0068158009f23mr8224751qvx.115.1706578954521; Mon, 29 Jan 2024 17:42:34 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id mw2-20020a05621433c200b006891e1f098csm3996652qvb.78.2024.01.29.17.42.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:34 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 17/20] mm: zswap: function ordering: compress & decompress functions Date: Mon, 29 Jan 2024 20:36:53 -0500 Message-ID: <20240130014208.565554-18-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Stat-Signature: i46pwy5xgc9jbu7izosx7qzfxmo6grew X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 57E674000E X-Rspam-User: X-HE-Tag: 1706578955-14923 X-HE-Meta: U2FsdGVkX1+f9chI60UkRTIr/Q0Tfb7rJR+wu3qAMcjU0BRFDAgrRJtUcRl3InJiTfqBL3yGf5rpNiuL9RAn02Rmhl7OMiKuginzN+4LRqOA70ePMpSZ3Nj22ugEtmnVI1CMe+lnxSMdjyIJcW+pQVu8kjoAwfvUL6oqqtqURTayYaQD99ezIRg3jUlr9kSy8Tgv1Bw3JyUWO2jEVFyKFE4Q8TxUyDNvDHyNMSGYYUCrTaBvBwsTZ0UTJWlkaJEVp3P24qdREw0dYAwhAq2EdQMOWeYcnH2TfaEAJxoiwjnWKhhga5l5Y1fQFbwn2r8giq4tI6ETf8jFha72TFENuLN2VQPpg6ghovCYXoQYHT8bccL2JcGbvufVENd3OF/gbyNxVCMbU+DDqzqrBPOuDl2hvJGjBV/4VMUtW4wXVBV892bFHv3FOs2KVa91aKluiILyJ105ExGls+YlYt1zOr5dPrXtFscPcERJNi//fniSV7Ry/hEZfxqX6Opsbd60VxhvAgjEamnZoDdxGdtCc+/P0sOqEz9cHAGKLS8HgkINkBJ07ApT/CMFhbrRdgIAlYovPf9+lqE6xL4mHp5XzWACptr7B+l8xRie450N+yMzgFOhp2++as+kBenY7EzarWEIniay7IKh1hr5le9DlNmDYcF3IbhOl7j0ehhu3ZvTwl8cbvIvkussg3bWu3OZK9TgrKz0mcqgHUjC978J6vEOVxHZ5SmjeQsM/PniuNDderP3WkolDeZD6oCFURzlXoMHMBVZB4p2drMcW+/k5z+4mWZr8IXaKah24qhBwrb9itEQz6bUrEOPRBxLZNq+Fduoh64ysbsgv83GvBQWKp9gi/ltLc9Vc+QLXdIhWGUNGazsEkN8DCyy5Si4Gn9WqNmgdR0WRYXYN/6e4aAJWha9lzIZp4NcLhfXt5pUcUp4uk73UzZC5p+fZ+L1Hf4+ePeC6YIeHjSq0sy/0+Q G6rYx5ot iBvim0DlRiKCOknKkJOn78wpZm9Xl8YL6e0uMr7t9Gz5sobY8ba6UBd03Rg4p4UR2bVTxvy6Gf4E6lG3plhfKfpptS/D9yLtvFiwPOZmiEXJz4D/Avh89zqs7/DwGlnAl5DlQmWLxsgli85ByWHUkv4XpJlz/6v0PFhdh12x+5p7wjferNa9w+wNwcG2pWADZ/Hdi9myfEpY9ExKOFzlyz0hdMP1UzEH3NJiTmyd/+OmauRAtAugIg0Ssu/0ncCnYAuTHQhPbaAT1qNL5BPn6rYXQ/QsDZQEEicnphOccyjdIGFRe61jyjRcqTCWWvDBIm5kUATXUxt2VYhP05gvrMG3jbSYtN1VdActDUK7RUjlq+ZGR2CSrUIaUbUcHS+RwxJcG+hxguB2CTr6Ca1bXLNitcz2rfP8Y85o1cLIxVp2whmvwCr1jTV5XGn2P8kiEIcnAHUa05i70hVetK2jXloiwaA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Writeback needs to decompress. Move the (de)compression API above what will be the consolidated shrinking/writeback code. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 207 +++++++++++++++++++++++++++-------------------------- 1 file changed, 105 insertions(+), 102 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 80adc2f7d1a2..17356b2e35c2 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -992,6 +992,111 @@ static void zswap_invalidate_entry(struct zswap_tree *tree, zswap_entry_put(entry); } +/********************************* +* compressed storage functions +**********************************/ +static bool zswap_compress(struct folio *folio, struct zswap_entry *entry) +{ + struct crypto_acomp_ctx *acomp_ctx; + struct scatterlist input, output; + unsigned int dlen = PAGE_SIZE; + unsigned long handle; + struct zpool *zpool; + char *buf; + gfp_t gfp; + int ret; + u8 *dst; + + acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); + + mutex_lock(&acomp_ctx->mutex); + + dst = acomp_ctx->buffer; + sg_init_table(&input, 1); + sg_set_page(&input, &folio->page, PAGE_SIZE, 0); + + /* + * We need PAGE_SIZE * 2 here since there maybe over-compression case, + * and hardware-accelerators may won't check the dst buffer size, so + * giving the dst buffer with enough length to avoid buffer overflow. + */ + sg_init_one(&output, dst, PAGE_SIZE * 2); + acomp_request_set_params(acomp_ctx->req, &input, &output, PAGE_SIZE, dlen); + + /* + * it maybe looks a little bit silly that we send an asynchronous request, + * then wait for its completion synchronously. This makes the process look + * synchronous in fact. + * Theoretically, acomp supports users send multiple acomp requests in one + * acomp instance, then get those requests done simultaneously. but in this + * case, zswap actually does store and load page by page, there is no + * existing method to send the second page before the first page is done + * in one thread doing zwap. + * but in different threads running on different cpu, we have different + * acomp instance, so multiple threads can do (de)compression in parallel. + */ + ret = crypto_wait_req(crypto_acomp_compress(acomp_ctx->req), &acomp_ctx->wait); + dlen = acomp_ctx->req->dlen; + if (ret) { + zswap_reject_compress_fail++; + goto unlock; + } + + zpool = zswap_find_zpool(entry); + gfp = __GFP_NORETRY | __GFP_NOWARN | __GFP_KSWAPD_RECLAIM; + if (zpool_malloc_support_movable(zpool)) + gfp |= __GFP_HIGHMEM | __GFP_MOVABLE; + ret = zpool_malloc(zpool, dlen, gfp, &handle); + if (ret == -ENOSPC) { + zswap_reject_compress_poor++; + goto unlock; + } + if (ret) { + zswap_reject_alloc_fail++; + goto unlock; + } + + buf = zpool_map_handle(zpool, handle, ZPOOL_MM_WO); + memcpy(buf, dst, dlen); + zpool_unmap_handle(zpool, handle); + + entry->handle = handle; + entry->length = dlen; + +unlock: + mutex_unlock(&acomp_ctx->mutex); + return ret == 0; +} + +static void zswap_decompress(struct zswap_entry *entry, struct page *page) +{ + struct zpool *zpool = zswap_find_zpool(entry); + struct scatterlist input, output; + struct crypto_acomp_ctx *acomp_ctx; + u8 *src; + + acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); + mutex_lock(&acomp_ctx->mutex); + + src = zpool_map_handle(zpool, entry->handle, ZPOOL_MM_RO); + if (!zpool_can_sleep_mapped(zpool)) { + memcpy(acomp_ctx->buffer, src, entry->length); + src = acomp_ctx->buffer; + zpool_unmap_handle(zpool, entry->handle); + } + + sg_init_one(&input, src, entry->length); + sg_init_table(&output, 1); + sg_set_page(&output, page, PAGE_SIZE, 0); + acomp_request_set_params(acomp_ctx->req, &input, &output, entry->length, PAGE_SIZE); + BUG_ON(crypto_wait_req(crypto_acomp_decompress(acomp_ctx->req), &acomp_ctx->wait)); + BUG_ON(acomp_ctx->req->dlen != PAGE_SIZE); + mutex_unlock(&acomp_ctx->mutex); + + if (zpool_can_sleep_mapped(zpool)) + zpool_unmap_handle(zpool, entry->handle); +} + /********************************* * shrinker functions **********************************/ @@ -1317,108 +1422,6 @@ static void shrink_worker(struct work_struct *w) zswap_pool_put(pool); } -static bool zswap_compress(struct folio *folio, struct zswap_entry *entry) -{ - struct crypto_acomp_ctx *acomp_ctx; - struct scatterlist input, output; - unsigned int dlen = PAGE_SIZE; - unsigned long handle; - struct zpool *zpool; - char *buf; - gfp_t gfp; - int ret; - u8 *dst; - - acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); - - mutex_lock(&acomp_ctx->mutex); - - dst = acomp_ctx->buffer; - sg_init_table(&input, 1); - sg_set_page(&input, &folio->page, PAGE_SIZE, 0); - - /* - * We need PAGE_SIZE * 2 here since there maybe over-compression case, - * and hardware-accelerators may won't check the dst buffer size, so - * giving the dst buffer with enough length to avoid buffer overflow. - */ - sg_init_one(&output, dst, PAGE_SIZE * 2); - acomp_request_set_params(acomp_ctx->req, &input, &output, PAGE_SIZE, dlen); - - /* - * it maybe looks a little bit silly that we send an asynchronous request, - * then wait for its completion synchronously. This makes the process look - * synchronous in fact. - * Theoretically, acomp supports users send multiple acomp requests in one - * acomp instance, then get those requests done simultaneously. but in this - * case, zswap actually does store and load page by page, there is no - * existing method to send the second page before the first page is done - * in one thread doing zwap. - * but in different threads running on different cpu, we have different - * acomp instance, so multiple threads can do (de)compression in parallel. - */ - ret = crypto_wait_req(crypto_acomp_compress(acomp_ctx->req), &acomp_ctx->wait); - dlen = acomp_ctx->req->dlen; - if (ret) { - zswap_reject_compress_fail++; - goto unlock; - } - - zpool = zswap_find_zpool(entry); - gfp = __GFP_NORETRY | __GFP_NOWARN | __GFP_KSWAPD_RECLAIM; - if (zpool_malloc_support_movable(zpool)) - gfp |= __GFP_HIGHMEM | __GFP_MOVABLE; - ret = zpool_malloc(zpool, dlen, gfp, &handle); - if (ret == -ENOSPC) { - zswap_reject_compress_poor++; - goto unlock; - } - if (ret) { - zswap_reject_alloc_fail++; - goto unlock; - } - - buf = zpool_map_handle(zpool, handle, ZPOOL_MM_WO); - memcpy(buf, dst, dlen); - zpool_unmap_handle(zpool, handle); - - entry->handle = handle; - entry->length = dlen; - -unlock: - mutex_unlock(&acomp_ctx->mutex); - return ret == 0; -} - -static void zswap_decompress(struct zswap_entry *entry, struct page *page) -{ - struct zpool *zpool = zswap_find_zpool(entry); - struct scatterlist input, output; - struct crypto_acomp_ctx *acomp_ctx; - u8 *src; - - acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); - mutex_lock(&acomp_ctx->mutex); - - src = zpool_map_handle(zpool, entry->handle, ZPOOL_MM_RO); - if (!zpool_can_sleep_mapped(zpool)) { - memcpy(acomp_ctx->buffer, src, entry->length); - src = acomp_ctx->buffer; - zpool_unmap_handle(zpool, entry->handle); - } - - sg_init_one(&input, src, entry->length); - sg_init_table(&output, 1); - sg_set_page(&output, page, PAGE_SIZE, 0); - acomp_request_set_params(acomp_ctx->req, &input, &output, entry->length, PAGE_SIZE); - BUG_ON(crypto_wait_req(crypto_acomp_decompress(acomp_ctx->req), &acomp_ctx->wait)); - BUG_ON(acomp_ctx->req->dlen != PAGE_SIZE); - mutex_unlock(&acomp_ctx->mutex); - - if (zpool_can_sleep_mapped(zpool)) - zpool_unmap_handle(zpool, entry->handle); -} - /********************************* * writeback code **********************************/ From patchwork Tue Jan 30 01:36:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536684 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F61DC47DA9 for ; Tue, 30 Jan 2024 01:43:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7FE626B00BD; Mon, 29 Jan 2024 20:42:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7AF916B00BE; Mon, 29 Jan 2024 20:42:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5B0056B00BF; Mon, 29 Jan 2024 20:42:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 46B976B00BD for ; Mon, 29 Jan 2024 20:42:38 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 2E54DA0164 for ; Tue, 30 Jan 2024 01:42:38 +0000 (UTC) X-FDA: 81734277996.18.5D93E5A Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) by imf10.hostedemail.com (Postfix) with ESMTP id 8F29AC0016 for ; Tue, 30 Jan 2024 01:42:36 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=xuALguqK; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf10.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.176 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578956; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pWfrSpQCJ73gDMIVUCgMZppF172UOPzzTZlrxOiNavU=; b=N+dRJyC897HMgyfphYBIR1DbHibo9vNVGOAc2bXIM9gPZnxv/aK2zm5llkKM+JoMGg6Dgf 6W7563gxpKAE+wvZg3+YJVkevpcxKArYk1rVxWwsSbzBI8HLxT59SmuAqWapnZ3XRiC2TP oCjlmLfXWIlT/4soQx2MK1+eOwSrKek= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=xuALguqK; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf10.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.176 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578956; a=rsa-sha256; cv=none; b=Xqfc+pchxsvcLonH7Sfd7uJ5V99/8c7ti0cfxW1xYLTksRoMDMyTXbDGKskRrNXoDb/eF2 wuzmH/6juX6prjJRUARK/uHWMjZ/M+5hp61BwzTsKePXHZmRhzSauhQbQuxklpS1rBcrI7 IEz4YALbEWT2LQ3Kwl0c15NlQR9OXjs= Received: by mail-qk1-f176.google.com with SMTP id af79cd13be357-7838af983c1so281285685a.3 for ; Mon, 29 Jan 2024 17:42:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578956; x=1707183756; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pWfrSpQCJ73gDMIVUCgMZppF172UOPzzTZlrxOiNavU=; b=xuALguqKNfX8MS40M/WOs4T83ccp+zuOwqekWoa7oln13bDtDRDOG7sJdt3MDkUzTX QRQaVikAjqgJVa76RP+2EHpLcCHtZAJd+5Xj/U1WJUFesKkkGnCpbWLg0RHHcY2cy/DM PTT1PpZ7xCccldWECINkFH4EX2X5ly+LSk5uD7/KY+RpOAtN5SBBVBzGaoNK8IIt/CGM wRvhUkwn6gCikokVjd9mtVL63jZiu5BnJtpQIcKR086DzLSWNM90rUcF5AhAI9fk5vV7 2uYx9VgXX2B0BjGix7GX/7Lrxs6T9G1ibXox621zB0xXFfiMrqSWS7sEOojUc8xYqmyI aPyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578956; x=1707183756; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pWfrSpQCJ73gDMIVUCgMZppF172UOPzzTZlrxOiNavU=; b=LfMWnbDuLQIFQSjXb7vL8PeZvewZARoxu5YGj9ytkwIgbyvVHsU6IKbzqbUSNpg3WV ITtcqOWq/cGqzmFgWTlzQ/CoVUiUe72UuVMOYsHFnXvywuZIo8swd8tQpbjETY4pXnWv XhbyS1vGD//JyB1lfcuDYy6pxBiqHkHwfxmMEM+tgEhnix4PqA7KC7eJ2YiI+yqhwjjL mkG9YgtQORjssufXK+6F9zmFaWg69veeelwOjKbbbz+4z5CKXHwmPyzhrdHDAehiwg8k o/aUn3OgwQlY9rXEdn//pQx+OFYoUlg17p4Bon/ixkkmCOkBrRzif6+tATDCbOeJgxHp HJGw== X-Gm-Message-State: AOJu0Yzf1LkLE3KhmF+Jz92frhfekWpznRALmD57WovKv8xvJxWZuKpW LZ1WXo0uPY8UDKQ4EJVjyWKtrNTa5ZmpYl15EivVNoOcPgLpAKAbyyv022/8HpE= X-Google-Smtp-Source: AGHT+IEXs8SJCuuIZtfIF+y9ddQXhUqdNoy7qZLBVXx9Tn8aMjnAw4Zw+Z4ifp80mQOXs7/58wge/w== X-Received: by 2002:a05:6214:400c:b0:68c:5ae6:de8d with SMTP id kd12-20020a056214400c00b0068c5ae6de8dmr1717725qvb.39.1706578955770; Mon, 29 Jan 2024 17:42:35 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id df12-20020a056214080c00b0068c4542eb0fsm2419516qvb.137.2024.01.29.17.42.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:35 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 18/20] mm: zswap: function ordering: per-cpu compression infra Date: Mon, 29 Jan 2024 20:36:54 -0500 Message-ID: <20240130014208.565554-19-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 8F29AC0016 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: c1bt6sp48te3uarea5cubgboq7xhs7qq X-HE-Tag: 1706578956-660152 X-HE-Meta: U2FsdGVkX1874qxW8utB2hOVwMO35tqkArAdivZ3YRLKhtnExAxnrt62TMOZKD2pe7m2/qbgxKvHssXbR7muc5PdfNKP3mXW8IjfGgFWU1eMDrdqjx390doYTX7f1zzCh92hKfRLhmccSAJFTj+7uDe69Zvs5RWDIodUvvdGnSssDaJYYSwgFJalr2rJoKpbjmr8rNHQo0VBP50/jftdxIKF29CvqyoITEvu1LCGbEwSIJ6vr/blMVQalus8QlMNbg2VsT/TXAaP4ku8t5PPfoelRvi24EL6VVu8OujN5KFYALWtrleQEXa0bSuGb9bxeYPGIsx/HvCEYmzTas0+qmj1RRPYWQjWLVGVLfI6zRpLcFuHyuCT53GAttU4PFNMwQ/o24RAhQos/MDW/yhnScSeX3yI4WnwTMbSC0acVQb1Qk6I7ReFGsd1CvOTqbNRwwqzsmG364SGRY5VUmSnkKdDstoFkNB57cTqgsx5OarX860iCxUMsvc1uESUI/QsZsR2JHyI82QBHLLzhWgs7fw85f6MzasdX0jivzhkYhw5aDMfr821kqpn4LHzaIT6tUKVwgBKgCMmEOqA1Osy78PF+9Aa7dWPNrawrpZOmIcDN6vMZfFnQpudQGF7bEKW0WYo6ljSRl3pwM26nNQZ0217EWERBg9gX4E38CHlqPcd7EpsrlSDWw+V6nuFNnosJZ9xod4flfwj9TNAJJgk9QCxqk3j0TyTbsQIC6fo4Lqfx3rkES2uLr/gWYQxlv8DhRAFs0U4I918A7UdE2fzemq43fxq7aXcsY5tjDIRKZuMEJ9ioITQZFwXyhc9mC9Q0irQCBklDsm3+LTtmBEsTALgG65cwL9hGmKHIz9ppfBhvop/mnv6s/Y+1AoRCjjeMfJV+PG8mHeMfgysV69KOq4DwgzmdmSm3shdVKw+QudnU9OC5SEeQSb8DxE4a35FyDS6Yh7h2nk9hcWxbSk aN/UoU9O uDPj3bIbR5ZbgLgyBgdGKxG5kiNQd9jgtFtpsFmQthfldQ/iN07tlMEQ1ewL6+WoqgRktT5rUFH2RC8M/AN0JZJoT6vtQ7Gcfjii8wvzJAvAA/AZ59tiZJmhHKXajaPm1xvWrlEeiV4LPsWmmb7cpmwWlqTzK2q5+WhUwIOZw6Tip5oDkq6sHkzUX3g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The per-cpu compression init/exit callbacks are awkwardly in the middle of the shrinker code. Move them up to the compression section. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 135 ++++++++++++++++++++++++++--------------------------- 1 file changed, 66 insertions(+), 69 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 17356b2e35c2..acd7dcd1e0f2 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -995,6 +995,72 @@ static void zswap_invalidate_entry(struct zswap_tree *tree, /********************************* * compressed storage functions **********************************/ +static int zswap_cpu_comp_prepare(unsigned int cpu, struct hlist_node *node) +{ + struct zswap_pool *pool = hlist_entry(node, struct zswap_pool, node); + struct crypto_acomp_ctx *acomp_ctx = per_cpu_ptr(pool->acomp_ctx, cpu); + struct crypto_acomp *acomp; + struct acomp_req *req; + int ret; + + mutex_init(&acomp_ctx->mutex); + + acomp_ctx->buffer = kmalloc_node(PAGE_SIZE * 2, GFP_KERNEL, cpu_to_node(cpu)); + if (!acomp_ctx->buffer) + return -ENOMEM; + + acomp = crypto_alloc_acomp_node(pool->tfm_name, 0, 0, cpu_to_node(cpu)); + if (IS_ERR(acomp)) { + pr_err("could not alloc crypto acomp %s : %ld\n", + pool->tfm_name, PTR_ERR(acomp)); + ret = PTR_ERR(acomp); + goto acomp_fail; + } + acomp_ctx->acomp = acomp; + + req = acomp_request_alloc(acomp_ctx->acomp); + if (!req) { + pr_err("could not alloc crypto acomp_request %s\n", + pool->tfm_name); + ret = -ENOMEM; + goto req_fail; + } + acomp_ctx->req = req; + + crypto_init_wait(&acomp_ctx->wait); + /* + * if the backend of acomp is async zip, crypto_req_done() will wakeup + * crypto_wait_req(); if the backend of acomp is scomp, the callback + * won't be called, crypto_wait_req() will return without blocking. + */ + acomp_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG, + crypto_req_done, &acomp_ctx->wait); + + return 0; + +req_fail: + crypto_free_acomp(acomp_ctx->acomp); +acomp_fail: + kfree(acomp_ctx->buffer); + return ret; +} + +static int zswap_cpu_comp_dead(unsigned int cpu, struct hlist_node *node) +{ + struct zswap_pool *pool = hlist_entry(node, struct zswap_pool, node); + struct crypto_acomp_ctx *acomp_ctx = per_cpu_ptr(pool->acomp_ctx, cpu); + + if (!IS_ERR_OR_NULL(acomp_ctx)) { + if (!IS_ERR_OR_NULL(acomp_ctx->req)) + acomp_request_free(acomp_ctx->req); + if (!IS_ERR_OR_NULL(acomp_ctx->acomp)) + crypto_free_acomp(acomp_ctx->acomp); + kfree(acomp_ctx->buffer); + } + + return 0; +} + static bool zswap_compress(struct folio *folio, struct zswap_entry *entry) { struct crypto_acomp_ctx *acomp_ctx; @@ -1201,75 +1267,6 @@ static void zswap_alloc_shrinker(struct zswap_pool *pool) pool->shrinker->seeks = DEFAULT_SEEKS; } -/********************************* -* per-cpu code -**********************************/ -static int zswap_cpu_comp_prepare(unsigned int cpu, struct hlist_node *node) -{ - struct zswap_pool *pool = hlist_entry(node, struct zswap_pool, node); - struct crypto_acomp_ctx *acomp_ctx = per_cpu_ptr(pool->acomp_ctx, cpu); - struct crypto_acomp *acomp; - struct acomp_req *req; - int ret; - - mutex_init(&acomp_ctx->mutex); - - acomp_ctx->buffer = kmalloc_node(PAGE_SIZE * 2, GFP_KERNEL, cpu_to_node(cpu)); - if (!acomp_ctx->buffer) - return -ENOMEM; - - acomp = crypto_alloc_acomp_node(pool->tfm_name, 0, 0, cpu_to_node(cpu)); - if (IS_ERR(acomp)) { - pr_err("could not alloc crypto acomp %s : %ld\n", - pool->tfm_name, PTR_ERR(acomp)); - ret = PTR_ERR(acomp); - goto acomp_fail; - } - acomp_ctx->acomp = acomp; - - req = acomp_request_alloc(acomp_ctx->acomp); - if (!req) { - pr_err("could not alloc crypto acomp_request %s\n", - pool->tfm_name); - ret = -ENOMEM; - goto req_fail; - } - acomp_ctx->req = req; - - crypto_init_wait(&acomp_ctx->wait); - /* - * if the backend of acomp is async zip, crypto_req_done() will wakeup - * crypto_wait_req(); if the backend of acomp is scomp, the callback - * won't be called, crypto_wait_req() will return without blocking. - */ - acomp_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG, - crypto_req_done, &acomp_ctx->wait); - - return 0; - -req_fail: - crypto_free_acomp(acomp_ctx->acomp); -acomp_fail: - kfree(acomp_ctx->buffer); - return ret; -} - -static int zswap_cpu_comp_dead(unsigned int cpu, struct hlist_node *node) -{ - struct zswap_pool *pool = hlist_entry(node, struct zswap_pool, node); - struct crypto_acomp_ctx *acomp_ctx = per_cpu_ptr(pool->acomp_ctx, cpu); - - if (!IS_ERR_OR_NULL(acomp_ctx)) { - if (!IS_ERR_OR_NULL(acomp_ctx->req)) - acomp_request_free(acomp_ctx->req); - if (!IS_ERR_OR_NULL(acomp_ctx->acomp)) - crypto_free_acomp(acomp_ctx->acomp); - kfree(acomp_ctx->buffer); - } - - return 0; -} - static enum lru_status shrink_memcg_cb(struct list_head *item, struct list_lru_one *l, spinlock_t *lock, void *arg) { From patchwork Tue Jan 30 01:36:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536685 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B79C3C47DA9 for ; Tue, 30 Jan 2024 01:43:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F004E6B00BE; Mon, 29 Jan 2024 20:42:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E894E6B00BF; Mon, 29 Jan 2024 20:42:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C8E786B00C0; Mon, 29 Jan 2024 20:42:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B3F5E6B00BE for ; Mon, 29 Jan 2024 20:42:39 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 8A99BA1D03 for ; Tue, 30 Jan 2024 01:42:39 +0000 (UTC) X-FDA: 81734278038.08.FA9B1EA Received: from mail-qv1-f53.google.com (mail-qv1-f53.google.com [209.85.219.53]) by imf28.hostedemail.com (Postfix) with ESMTP id C6CCEC000B for ; Tue, 30 Jan 2024 01:42:37 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=3HavyYuJ; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf28.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.53 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578957; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nthkOb34nC29m9Z7rdKEYXbNdpH1xuatn2MKs2M9M7s=; b=FOhj2RVOfJuGIclEYYTbOcf7pxeFDjsJ5RaZ/Ow0XbjBq/YtIU74u5WMeA6ali4Glpq2ey 41+3oWLwcTRLVioID+9WJiUidadUx+S/Rtp+i89MoiXHh0TDt1mluVGE3IlC/0179sYKEC AnR9DmwdJQVCY+LbbcjmYcHP/hM7AjU= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=3HavyYuJ; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf28.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.53 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578957; a=rsa-sha256; cv=none; b=D5nIsh5+dp226sSkAh6pVs9IG4fJ0K2CPOeWMZ9nn5xML3aw4lJSZ5VuHg2DIPvbNmNUIM v6rrSlbRZLSq/2Z4jW7orawq6zQ3nglZueFFMxKMaCHETmqh7SLBLP3xgSrmzj6XzhAb3V mc7tShKwjnyFNrgxN/tWEbSV+JGkxmk= Received: by mail-qv1-f53.google.com with SMTP id 6a1803df08f44-68c4fb9e7e0so11623896d6.2 for ; Mon, 29 Jan 2024 17:42:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578957; x=1707183757; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nthkOb34nC29m9Z7rdKEYXbNdpH1xuatn2MKs2M9M7s=; b=3HavyYuJvTUY8zmJzN73Wjnc9MhGh+bCEu2BsnckYot5oq5uCYlFU8haPfzSy1N50n lFQRqbCuk1VG5xPZ2sq2zA4kGXex6yfxrlWD95S3PvJmCCFnXZTmuG3KArqoefC56bIA B/V1RcjQFZC1YmrgI619TRhUgWp7W/17RqW7XtOpQVWtQH4L5/gTijnkZp26hSVvBf3f WZMIJfAgBfiI7yKoWFlkZ52BiXagqumEpWB+I82PtM3Tp5Ka89AJIs+7YVB9hEu0ivyK cK+Ekx3vBwaIDAc30/KahvV1g3EslRC0omqLx2QgVR+/P8W9FmEC/ZzLytuXYS0By6v7 ZHsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578957; x=1707183757; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nthkOb34nC29m9Z7rdKEYXbNdpH1xuatn2MKs2M9M7s=; b=wSXQ0tGT3t7WCwpMZEvx5AQ8faSSWkcPz9nKs8KWnHgG+lARVNUmDerBchRsJ4MJM6 +7ZUq1DFSRbgCvK25Ob2Fv8Jw2+FFevGmSE9SR25sgPaoutkVcCDz8x5oaWmbP7KIjFx gy/7eifgVMEMIF6BjsRvrs8XsXrivQQnYLDVRerwXMIYoNzH8B4AW9lFeCjgork3YQTH eA1RkRTiPQS3iRFV9eMyTo+0Lgujz2SuDPnT5atjPjv0nJ2XCP0SsGLgI/3ga1c22TPj 1LPkhVmxK+lEITmcYk0d/eHsheWUzZBVndm3GBtwapqymC4VBYJcat9+FeyiBKrmdpWB SVEg== X-Gm-Message-State: AOJu0YxpqxQ5l/IDixeEEWanbYZLZSdhSRFHkcAg8dClMtCbrVj7Sj8U ic95HyNb0/L46j01PZ4vjmiVMUBvop8vQD/E1y6lTwGV6tYPUtnmmb62o1zvs8Q= X-Google-Smtp-Source: AGHT+IGTKeUUuo7KoC1RLq1MuNgQsEnaX6DaZ/35944su1CBVjphvILdr8AWGuYvZXd46VFaBb9AjQ== X-Received: by 2002:a05:6214:c68:b0:68c:498e:d09f with SMTP id t8-20020a0562140c6800b0068c498ed09fmr4765609qvj.54.1706578956903; Mon, 29 Jan 2024 17:42:36 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id p16-20020a0cc3d0000000b00686ac3226ccsm3935915qvi.114.2024.01.29.17.42.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:36 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 19/20] mm: zswap: function ordering: writeback Date: Mon, 29 Jan 2024 20:36:55 -0500 Message-ID: <20240130014208.565554-20-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: C6CCEC000B X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: wd4qnji194j5ujqgoz5bkxniz3chxgis X-HE-Tag: 1706578957-905289 X-HE-Meta: U2FsdGVkX18iOQtoFHTQEd2xEfjWdVqBbC73lQ5+BxlHsZ2rOHGt7STB5CubN5bkcthWmVuQNe+denwA8FDHhe+BmYdo+KFjVXC0Xojcb6N0s9omQzz29U6fHjM/nT665UY5w9B6gtmZ5tYMO89wAeTRBsi1VwtsWeecgBbnzJH3rgbac8N9YInwnjVWE0nZn0I4GOsa812M0KGxxTdA8FgWj8E1dKhZlYSrnZC5++lFxLRpiuQ+oSZ7zgbO8ltedk7FX8yP6OvyjkSho8NyE9iLtrTuEz72C1x/P6B5e9NxyTEOCGiEDd0YeyeGK9R/mybVG33MMjMmxPRgMwm6buEAoa4iZp696O31LQ6KFeVj+ZUsWHORjQuZ+iynWXUPtST9Z3m19RGvdzxr910dyoyvca6fhJqiSKvwWBvJLuv+0hT966Ti3JerbinIF7AWUsaTylVefK4vBAzFgAF1i21/cJDdAIuJELYlu+9Cj2oPHY/cyOkdLlSSF26Ht8VYULrMK/yE+r6T6ucQ7cRAzRDXKt25CQcQ2sM8d/+rR2aVcYbgi+exgerna0+yCJW4exkf7vVC+UET9ycGxJ5kH4/ikJotUrTPArYDZi8BfSxQuC1UI2nGD80uw6dZNilGXJ4qNGiKNLvIenlPH8GeK6ABrHhsem7IQ6jyWY9V+jXR6csYUEbnNvxa/8Y+Yz7J56sE6QjbyVOX4Bs9iW96eoHbpHv+IbsZwUlgloTdO52xdBzRtGSECDX5D9oP0bi180kF0WLaygcuN86txwKDCrps2uXtuYOPX+BWKtYr1zmGMkPRagi6hGyMQhfZVkkjwYi9p6PueUMHHTP5+m4UtwBrcuqjYK5TxFlmFHzkdyfcqqjT9cj8ZNfkU/znddBq4jTcsudiqvehjTsaH0LoeUyqMaxpId9NNv4Orj/7+uxN/GZX5AM1uOklUTEvDQeoNzWHjxWePhpGsZH2BHH nVpKSSg1 MnF6PeT87Oln7GelZFsVUXSdbmlOt+4yn1VKFF0LsQTddPMJgrgfcNCRmvc2albw7ySCcu+trJkxjlHl9jHpq5vUPF3Y8b9L2knFyiWg5+YHWgqQP0FdG4hM8TTBZeUGuHF4MWiojR5kE1jZRYBowgGP/coDmgD4C+EFG40AeldT6Wv68jRHkAw1C5UB1HCxNk9uTvnui54hyuSoavB3MwaYM5JHu1s9mS2j1RPlsAIUh5y1OYqif4Bz4obOIU5poRdf9BEn4XFKrqtrA+mmNfyciw3ZIQjmqIkYka7jWHAZEt2NKrFuJloxsBnVW8LUj+kp7uTE117Q+5iXROjqYrFBrQOwuc8CcJ1yclsnGcNypwvtIrWdiaFs1VR9e7R8OkJTYlV0wUlqLsZWzg0bn94WFBX6nfhU8lGK2Ies++Pza5a2S+X3pmbCEMv+COrO1yACT X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Shrinking needs writeback. Naturally, move the writeback code above the shrinking code. Delete the forward decl. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 183 ++++++++++++++++++++++++++--------------------------- 1 file changed, 90 insertions(+), 93 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index acd7dcd1e0f2..0cb3437d47eb 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -276,9 +276,6 @@ static inline struct zswap_tree *swap_zswap_tree(swp_entry_t swp) pr_debug("%s pool %s/%s\n", msg, (p)->tfm_name, \ zpool_get_type((p)->zpools[0])) -static int zswap_writeback_entry(struct zswap_entry *entry, - swp_entry_t swpentry); - static bool zswap_is_full(void) { return totalram_pages() * zswap_max_pool_percent / 100 < @@ -1163,6 +1160,96 @@ static void zswap_decompress(struct zswap_entry *entry, struct page *page) zpool_unmap_handle(zpool, entry->handle); } +/********************************* +* writeback code +**********************************/ +/* + * Attempts to free an entry by adding a folio to the swap cache, + * decompressing the entry data into the folio, and issuing a + * bio write to write the folio back to the swap device. + * + * This can be thought of as a "resumed writeback" of the folio + * to the swap device. We are basically resuming the same swap + * writeback path that was intercepted with the zswap_store() + * in the first place. After the folio has been decompressed into + * the swap cache, the compressed version stored by zswap can be + * freed. + */ +static int zswap_writeback_entry(struct zswap_entry *entry, + swp_entry_t swpentry) +{ + struct zswap_tree *tree; + struct folio *folio; + struct mempolicy *mpol; + bool folio_was_allocated; + struct writeback_control wbc = { + .sync_mode = WB_SYNC_NONE, + }; + + /* try to allocate swap cache folio */ + mpol = get_task_policy(current); + folio = __read_swap_cache_async(swpentry, GFP_KERNEL, mpol, + NO_INTERLEAVE_INDEX, &folio_was_allocated, true); + if (!folio) + return -ENOMEM; + + /* + * Found an existing folio, we raced with swapin or concurrent + * shrinker. We generally writeback cold folios from zswap, and + * swapin means the folio just became hot, so skip this folio. + * For unlikely concurrent shrinker case, it will be unlinked + * and freed when invalidated by the concurrent shrinker anyway. + */ + if (!folio_was_allocated) { + folio_put(folio); + return -EEXIST; + } + + /* + * folio is locked, and the swapcache is now secured against + * concurrent swapping to and from the slot. Verify that the + * swap entry hasn't been invalidated and recycled behind our + * backs (our zswap_entry reference doesn't prevent that), to + * avoid overwriting a new swap folio with old compressed data. + */ + tree = swap_zswap_tree(swpentry); + spin_lock(&tree->lock); + if (zswap_rb_search(&tree->rbroot, swp_offset(swpentry)) != entry) { + spin_unlock(&tree->lock); + delete_from_swap_cache(folio); + folio_unlock(folio); + folio_put(folio); + return -ENOMEM; + } + + /* Safe to deref entry after the entry is verified above. */ + zswap_entry_get(entry); + spin_unlock(&tree->lock); + + zswap_decompress(entry, &folio->page); + + count_vm_event(ZSWPWB); + if (entry->objcg) + count_objcg_event(entry->objcg, ZSWPWB); + + spin_lock(&tree->lock); + zswap_invalidate_entry(tree, entry); + zswap_entry_put(entry); + spin_unlock(&tree->lock); + + /* folio is up to date */ + folio_mark_uptodate(folio); + + /* move it to the tail of the inactive list after end_writeback */ + folio_set_reclaim(folio); + + /* start writeback */ + __swap_writepage(folio, &wbc); + folio_put(folio); + + return 0; +} + /********************************* * shrinker functions **********************************/ @@ -1419,96 +1506,6 @@ static void shrink_worker(struct work_struct *w) zswap_pool_put(pool); } -/********************************* -* writeback code -**********************************/ -/* - * Attempts to free an entry by adding a folio to the swap cache, - * decompressing the entry data into the folio, and issuing a - * bio write to write the folio back to the swap device. - * - * This can be thought of as a "resumed writeback" of the folio - * to the swap device. We are basically resuming the same swap - * writeback path that was intercepted with the zswap_store() - * in the first place. After the folio has been decompressed into - * the swap cache, the compressed version stored by zswap can be - * freed. - */ -static int zswap_writeback_entry(struct zswap_entry *entry, - swp_entry_t swpentry) -{ - struct zswap_tree *tree; - struct folio *folio; - struct mempolicy *mpol; - bool folio_was_allocated; - struct writeback_control wbc = { - .sync_mode = WB_SYNC_NONE, - }; - - /* try to allocate swap cache folio */ - mpol = get_task_policy(current); - folio = __read_swap_cache_async(swpentry, GFP_KERNEL, mpol, - NO_INTERLEAVE_INDEX, &folio_was_allocated, true); - if (!folio) - return -ENOMEM; - - /* - * Found an existing folio, we raced with swapin or concurrent - * shrinker. We generally writeback cold folios from zswap, and - * swapin means the folio just became hot, so skip this folio. - * For unlikely concurrent shrinker case, it will be unlinked - * and freed when invalidated by the concurrent shrinker anyway. - */ - if (!folio_was_allocated) { - folio_put(folio); - return -EEXIST; - } - - /* - * folio is locked, and the swapcache is now secured against - * concurrent swapping to and from the slot. Verify that the - * swap entry hasn't been invalidated and recycled behind our - * backs (our zswap_entry reference doesn't prevent that), to - * avoid overwriting a new swap folio with old compressed data. - */ - tree = swap_zswap_tree(swpentry); - spin_lock(&tree->lock); - if (zswap_rb_search(&tree->rbroot, swp_offset(swpentry)) != entry) { - spin_unlock(&tree->lock); - delete_from_swap_cache(folio); - folio_unlock(folio); - folio_put(folio); - return -ENOMEM; - } - - /* Safe to deref entry after the entry is verified above. */ - zswap_entry_get(entry); - spin_unlock(&tree->lock); - - zswap_decompress(entry, &folio->page); - - count_vm_event(ZSWPWB); - if (entry->objcg) - count_objcg_event(entry->objcg, ZSWPWB); - - spin_lock(&tree->lock); - zswap_invalidate_entry(tree, entry); - zswap_entry_put(entry); - spin_unlock(&tree->lock); - - /* folio is up to date */ - folio_mark_uptodate(folio); - - /* move it to the tail of the inactive list after end_writeback */ - folio_set_reclaim(folio); - - /* start writeback */ - __swap_writepage(folio, &wbc); - folio_put(folio); - - return 0; -} - static int zswap_is_page_same_filled(void *ptr, unsigned long *value) { unsigned long *page; From patchwork Tue Jan 30 01:36:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13536686 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EBF2C47DA9 for ; Tue, 30 Jan 2024 01:43:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 121916B00C0; Mon, 29 Jan 2024 20:42:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0CE576B00C2; Mon, 29 Jan 2024 20:42:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E8B866B00C1; Mon, 29 Jan 2024 20:42:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id D50C16B00BF for ; Mon, 29 Jan 2024 20:42:40 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id B23C7A0170 for ; Tue, 30 Jan 2024 01:42:40 +0000 (UTC) X-FDA: 81734278080.01.3D09ED4 Received: from mail-qt1-f181.google.com (mail-qt1-f181.google.com [209.85.160.181]) by imf14.hostedemail.com (Postfix) with ESMTP id E94F5100010 for ; Tue, 30 Jan 2024 01:42:38 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=U6D5kgSz; spf=pass (imf14.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.181 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706578959; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jHRUZ67yNEKZS9FrhnY4ruPkQ8T6m4mDy3O5Fk7spbc=; b=TxR46PhEK3vlA6TnDhdC94lsJUaPn32S6xieFWsJIcKXP/PA4/wG2R8T31mCxqGk/DFS9J g9frjhPAdGMmZEIYCTBAgKgSiY2MKn5QJi0ITZMDLiLKi4w4XnePmBnsAc6xlDzcCS2jMU /7TUoB8z4Kg/V6DalhJYiWftyNPTQCs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706578959; a=rsa-sha256; cv=none; b=CK82ZMl8KAouyYKMWDe8Vaw7v3ri0VkWQoPYiqdoROTj8C38JukM0l2KaTyZP+EDt1SDhI DDyqZN81QigzdX3HmoNGdXt5Ug/8KZWAwHe/hAjSQ87yt9a6SmuemvQU3aWoWqlyIapsON +8GpSmcbysymaOo9qM9mdtbjOTrEQUQ= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=U6D5kgSz; spf=pass (imf14.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.160.181 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org Received: by mail-qt1-f181.google.com with SMTP id d75a77b69052e-42aa93264b1so6379101cf.2 for ; Mon, 29 Jan 2024 17:42:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578958; x=1707183758; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jHRUZ67yNEKZS9FrhnY4ruPkQ8T6m4mDy3O5Fk7spbc=; b=U6D5kgSzOnI4c4609Ra0cphIvgGZymUNmF9vLAwkCPfHqfb+n11WxDEO0ld1y3OPoZ 1YzT+3wfEhtrh0IkLRMAC7Atk3EYctbQ6aNALsOHBCcCjL5qasptVVTXaewj0l26i0KS rwBKlwtosZYmEdHLbAY3/RK7rnUKWWiNUO58yHY8eGhZ+8DuNkmUs/ijen5PihIytegK keSbPL2/7JgnnXC8wo8C9Vec8cSWFXT/J82Q5gxxHixd29iCmH4tj8PfeY4pqBj5pMDv vS4fDX2Nf+zHozO0jgwSVlyquH2NBllx3UEgUHJW9kvk+v0QzmALT43upgbFcWt+TKnx 9/Yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578958; x=1707183758; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jHRUZ67yNEKZS9FrhnY4ruPkQ8T6m4mDy3O5Fk7spbc=; b=f1zkon1VRIPv233rXpxYhDeWQZdn9m0jbvyHkcfxKriT6lcEOepxB5Zo5mFf2jcvxg hgj/PMl8rI9MVkbGAMTaDjtXeOY7PmS6sLSIh3OND529JbauvJvY5iZqteJ8+dXmZMWG P5Dv9XERBrAEh6Pc5J4Ii725kLgEDqW51KpEDXsHL6xQ+6Zq7YPNZMdVFAfjSS+pwz1u 2qYN86GM7RaNtWNO3EK9P+NmojM4eF+HeutWLRtx0BJi9pnmjfIifpz6+8oKdgbWYiDB bb7MKrfO41U1PLvS+N8jZvRQDMJHXs0dG8YeYWLT8O5OQP3pF2iaoGgOb479mQHKcf5f lE4A== X-Gm-Message-State: AOJu0YzM3V/CqfMXOSSAu3Ie2Dvs+gZ0nvZodxg+jlPYHEINlISJNUTk CF0GDIUIzG3F0dqqci8mGXUcIBl7bHJ53MC6ZswS8eYOQYP7+TwAsyRc/cviKcY= X-Google-Smtp-Source: AGHT+IFMteiVW0eCHxuGhuCuBCoCg6cS0vgJoS1KnkVEaCB7DQT4/zN8xIaB+z0TYxs1jXneazTs4Q== X-Received: by 2002:ac8:5bcc:0:b0:42a:9c39:9727 with SMTP id b12-20020ac85bcc000000b0042a9c399727mr5682189qtb.83.1706578958090; Mon, 29 Jan 2024 17:42:38 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id bl4-20020a05622a244400b004260b65b4f7sm1369468qtb.97.2024.01.29.17.42.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:37 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 20/20] mm: zswap: function ordering: shrink_memcg_cb Date: Mon, 29 Jan 2024 20:36:56 -0500 Message-ID: <20240130014208.565554-21-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: E94F5100010 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 5uid4w8ftwe4mh3n6ukosnbfrw17kyi4 X-HE-Tag: 1706578958-881184 X-HE-Meta: U2FsdGVkX18F5LJ/D8ZqcccbVc+VVWq5J/nq1qTpLkTBiythndE5J7H2OObMc5c5FjgBX7cOIcE+I5rMDM2GWMQo8Xvj3h+TLgNiwD2jgG/DqzjA7aCW6WjA0GHk8uAcDtHg/lekb9cjz5cRrxq5ZXTrNtvkICbC8BPpMcf7cHiUH1vS+sRztsC+1ntW24SdgyUvMCHUr+hcbD+5ChroaJXN9UsUsLdm76C6Ff/2zctpjBjaf72NOhA0C8iuqzMJ8NQNE7VEIaJqwi5YNoLDCI6eUgQtJlEhHoeHySLDtB17L2iHjtodh/rKR7PMlu/9s6ZtsUtdkly789WtgaKbjbrabxX5PQfdyLAKMU5x4y1MaZM3SjTGNdPr2QT2UDwpWnUrlBi+PJBzldaWphO69cyHcGwGSePqKArK7JPY5O0bfuosoi78DGnAfQTlIUmUz5nJNxoqBGbxA0B0HSOJR4+huRvIE4JR1KYBQfxGK2s+/Nmo6tFDv495VquIxLvrm/k3Es1IlSjdgArDZ7Mh67RPl2ZnR0W0t7XxHDm6RPIYmNDtUIb6/l6HQfg3/SJjNqjHjPVlU9HrWJWSPnMW74OYqEJsHZsvunO78nxjwl25ZXqA2Lj9oGysVF1yxmx2IVROj9F0Z0b3RROiV9lQ+VOi1KpEM2jhZZ0NVOD7/MeuZwQ0Xn+vnTO5gDg2svmli8nql+VrwNVkmqBn5wWbtwsxddkqBwTGoStOkFbtl3WRi+kWXHwwvVq6YkBam16EY2DBKLMTUJGWzAhIG1VgWZPIegm9oSPLi6RnChSDSduK24Tm3yQkNFfZdJndOQMxC5NYNnJs7KbMliTKwmfaXj/gtsPKvHgyIP8uwqoKhfP9uNWv2cRV7lfVPEm3jYCLGjEIL+du0cyLpT/f2Q7mLX+GqxDcESRqu8Q6hx5M8myOVM1Di8gW79XSgiz1x7AgF8P1FPbnDrquaHGIbA7 zKB+lbiL NQ0CccPrLHlDF5mLgPS4K58GRKWTTY0q/uuiN3Nc4W13nbBJh187ZOlAH2kNfucr7RTqJJn+buYHR3g5gmwrE6BvlGfekii3IO2lmPq1wfAhQciORJ3k4jS4N6ljaTowq7llMhXQETGMIOgEDBZXbX6EqI196kT7qndmta7s6QXyyVNjWG9c6yPxN7FuP699XgxiU2DGCg0ztDgSV/fY8XKfJKMX9T+IZLmTMVxpxVgMu0o4ZIjhvWhrEcnDQakPcMd424O0VdBfj3PZ9po/PCBndW7G39UaA1uTSStzZ+ttAdsWDc6+kaWQi1S3Ho3bbkmb5YBdQExoMLVdDPikctXFT4OG4lK0cs01dvnuSXtZsvfCQ1kycTakFQteqP9HU8nDouy6AoY304zrKMiekL+0nXfb2y7pqdC2jd18M/TcuJbSWG7caLSjoruWS6yFPvRLsnuP6w3n/TVPQYItLcJv5PQRn6wUPKsuPuo+z7KmBTds= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: shrink_memcg_cb() is called by the shrinker and is based on zswap_writeback_entry(). Move it in between. Save one fwd decl. Signed-off-by: Johannes Weiner Reviewed-by: Nhat Pham --- mm/zswap.c | 125 ++++++++++++++++++++++++++--------------------------- 1 file changed, 61 insertions(+), 64 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 0cb3437d47eb..4aea03285532 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -1254,7 +1254,67 @@ static int zswap_writeback_entry(struct zswap_entry *entry, * shrinker functions **********************************/ static enum lru_status shrink_memcg_cb(struct list_head *item, struct list_lru_one *l, - spinlock_t *lock, void *arg); + spinlock_t *lock, void *arg) +{ + struct zswap_entry *entry = container_of(item, struct zswap_entry, lru); + bool *encountered_page_in_swapcache = (bool *)arg; + swp_entry_t swpentry; + enum lru_status ret = LRU_REMOVED_RETRY; + int writeback_result; + + /* + * Rotate the entry to the tail before unlocking the LRU, + * so that in case of an invalidation race concurrent + * reclaimers don't waste their time on it. + * + * If writeback succeeds, or failure is due to the entry + * being invalidated by the swap subsystem, the invalidation + * will unlink and free it. + * + * Temporary failures, where the same entry should be tried + * again immediately, almost never happen for this shrinker. + * We don't do any trylocking; -ENOMEM comes closest, + * but that's extremely rare and doesn't happen spuriously + * either. Don't bother distinguishing this case. + * + * But since they do exist in theory, the entry cannot just + * be unlinked, or we could leak it. Hence, rotate. + */ + list_move_tail(item, &l->list); + + /* + * Once the lru lock is dropped, the entry might get freed. The + * swpentry is copied to the stack, and entry isn't deref'd again + * until the entry is verified to still be alive in the tree. + */ + swpentry = entry->swpentry; + + /* + * It's safe to drop the lock here because we return either + * LRU_REMOVED_RETRY or LRU_RETRY. + */ + spin_unlock(lock); + + writeback_result = zswap_writeback_entry(entry, swpentry); + + if (writeback_result) { + zswap_reject_reclaim_fail++; + ret = LRU_RETRY; + + /* + * Encountering a page already in swap cache is a sign that we are shrinking + * into the warmer region. We should terminate shrinking (if we're in the dynamic + * shrinker context). + */ + if (writeback_result == -EEXIST && encountered_page_in_swapcache) + *encountered_page_in_swapcache = true; + } else { + zswap_written_back_pages++; + } + + spin_lock(lock); + return ret; +} static unsigned long zswap_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc) @@ -1354,69 +1414,6 @@ static void zswap_alloc_shrinker(struct zswap_pool *pool) pool->shrinker->seeks = DEFAULT_SEEKS; } -static enum lru_status shrink_memcg_cb(struct list_head *item, struct list_lru_one *l, - spinlock_t *lock, void *arg) -{ - struct zswap_entry *entry = container_of(item, struct zswap_entry, lru); - bool *encountered_page_in_swapcache = (bool *)arg; - swp_entry_t swpentry; - enum lru_status ret = LRU_REMOVED_RETRY; - int writeback_result; - - /* - * Rotate the entry to the tail before unlocking the LRU, - * so that in case of an invalidation race concurrent - * reclaimers don't waste their time on it. - * - * If writeback succeeds, or failure is due to the entry - * being invalidated by the swap subsystem, the invalidation - * will unlink and free it. - * - * Temporary failures, where the same entry should be tried - * again immediately, almost never happen for this shrinker. - * We don't do any trylocking; -ENOMEM comes closest, - * but that's extremely rare and doesn't happen spuriously - * either. Don't bother distinguishing this case. - * - * But since they do exist in theory, the entry cannot just - * be unlinked, or we could leak it. Hence, rotate. - */ - list_move_tail(item, &l->list); - - /* - * Once the lru lock is dropped, the entry might get freed. The - * swpentry is copied to the stack, and entry isn't deref'd again - * until the entry is verified to still be alive in the tree. - */ - swpentry = entry->swpentry; - - /* - * It's safe to drop the lock here because we return either - * LRU_REMOVED_RETRY or LRU_RETRY. - */ - spin_unlock(lock); - - writeback_result = zswap_writeback_entry(entry, swpentry); - - if (writeback_result) { - zswap_reject_reclaim_fail++; - ret = LRU_RETRY; - - /* - * Encountering a page already in swap cache is a sign that we are shrinking - * into the warmer region. We should terminate shrinking (if we're in the dynamic - * shrinker context). - */ - if (writeback_result == -EEXIST && encountered_page_in_swapcache) - *encountered_page_in_swapcache = true; - } else { - zswap_written_back_pages++; - } - - spin_lock(lock); - return ret; -} - static int shrink_memcg(struct mem_cgroup *memcg) { struct zswap_pool *pool;