From patchwork Wed Jan 29 06:43:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13953464 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0522DC0218D for ; Wed, 29 Jan 2025 06:49:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3BA40280033; Wed, 29 Jan 2025 01:49:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 36921280032; Wed, 29 Jan 2025 01:49:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2090B280033; Wed, 29 Jan 2025 01:49:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id ED5FD280032 for ; Wed, 29 Jan 2025 01:49:06 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 9B6AB1A098C for ; Wed, 29 Jan 2025 06:49:06 +0000 (UTC) X-FDA: 83059562292.02.9A58384 Received: from mail-pj1-f45.google.com (mail-pj1-f45.google.com [209.85.216.45]) by imf28.hostedemail.com (Postfix) with ESMTP id CD882C0003 for ; Wed, 29 Jan 2025 06:49:04 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=U0vMH4fT; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf28.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.45 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738133344; a=rsa-sha256; cv=none; b=3G7dHXQoB0zSpz55olwpAmWvWUz4g7SDNmrGSQ3wTopBD13mQU+SDGFkiSQ+MQ1mbfRPor ncPH0Xc0+tewZMDQ21zVXw/0OA2awZyRBGuvfCXPQixKzc0gGS5BifoUyZpvr2yVmvCZ1x XzvtDGZQVcHrPJgXzRBmY7J3Ye1aaWg= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=U0vMH4fT; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf28.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.45 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738133344; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=nIHsH97JBfKpGoSoAFelRRbkGM380u6bs8Crr91xn1g=; b=erfO7IgsNj0W2NIp/kneJyqItqsrtcexv0CAlboVmjmUOFtzP+nawXIXxLz62omSPs05sD jlLMqVhBnkQpy89FfdZZEzA5cdMHuipRA6EqJTjCMoBGVt/jTB/qNl1mNBc5pxtW6nUpz/ s/AFa7Lr2LGhz5sM7Shm6WKCbjJZXDU= Received: by mail-pj1-f45.google.com with SMTP id 98e67ed59e1d1-2f42992f608so9056539a91.0 for ; Tue, 28 Jan 2025 22:49:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1738133343; x=1738738143; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=nIHsH97JBfKpGoSoAFelRRbkGM380u6bs8Crr91xn1g=; b=U0vMH4fTdj6KqyATXQY6z9jFuF2krC+DpAHfH8EG0hmyXAwIz4bDqihEtRtpyDFFoI B7kPGWzUS4tBtL+vLZ/r3jrg1iOpPuOmc1nTHRtLM3+XZLTkRuOW3VF98KXLeikRDE9G R6JrzETTc6bsZljWM74hCNGN4giFUFPv1zmk0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738133343; x=1738738143; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=nIHsH97JBfKpGoSoAFelRRbkGM380u6bs8Crr91xn1g=; b=ie//4RTFOuOsLaiC6Kuor2GAXEYgzfD4uMLpZz3+zjVN4SAOpg9P8JzYsl4O43B4Qj 4z2rXEJ1BwOufNL4/f1BEyTgV0QdEEtVvq5/kX445SDQj1Sz32JM+I/utYt8qIQm+wMt EkGgfchox/Gsxk+fbAhlhnYCkfy9cLXEQJ1B8SqtPgu0qLhoI0VwlR21l429UC6nhxCU is32N7+MQfx8ichGClr87w0JwX2By0jorLxQ1WC7s7LfrZB6dMlHvtwaSv0YuCHTMux6 MTbGqlv55t1XHX0j11dfjnt4N9Z319Kbt7K5TPLpGcUlMPEuHmXvE2bDwwnx5iHQyclM NHMw== X-Gm-Message-State: AOJu0Yyb/KrdkZIUpUpLzd0ipwqC/3uTN1lslCvZVme7v66UIJv/pybv ZdNRNZL9eKvMjKLLrOBW+Uv3YtvvMKL8Fp0DdcZPTsMhdGKwD5FU1ue+HAlbjw== X-Gm-Gg: ASbGncv0P+rFYKhyS3b6FPuKgtlSkxQc6f6QkI+CfH4lCew8ud8wttTHjstOge4a+p5 9itld7tqtwhLKUTbewNzyZGTt030V0raG6Nz3WBk1l5WHtZHY3HM2cdSFT/XaePHygLtnhX8n+X 9O5lOx5m/NnuUjPQhrBckxq+zDMNHkMXmejHWq0oPJQVZXArS8pjr48cHM3qx+uXec2rxQs7b9C xu5G6+V1ivQqUJVBVGG38P1BABCfR6C66ykLG9qTYAnamPYTc/YvEjbcf/xWdEsfgmk+WGCiJCH SWtGcLoV5ah+/K+/iw== X-Google-Smtp-Source: AGHT+IErOAjSVjGk9fjkjtR43tFtfz5SCHzPmvugMIl/Hrk34OZsMX6dTT7opahZJFMald/tlxJLvQ== X-Received: by 2002:a05:6a00:a1e:b0:725:b347:c3cc with SMTP id d2e1a72fcca58-72fd0c7bfadmr3234644b3a.23.1738133343328; Tue, 28 Jan 2025 22:49:03 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:b323:d70b:a1b8:1683]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-72f8a69ff81sm10805446b3a.11.2025.01.28.22.49.00 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 28 Jan 2025 22:49:02 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim , Johannes Weiner , Yosry Ahmed , Nhat Pham Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCHv1 0/6] zsmalloc: preemptible object mapping Date: Wed, 29 Jan 2025 15:43:46 +0900 Message-ID: <20250129064853.2210753-1-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: CD882C0003 X-Rspamd-Server: rspam10 X-Stat-Signature: ftz5jn4q9ssocjak5u449rm5h4816st5 X-HE-Tag: 1738133344-825053 X-HE-Meta: U2FsdGVkX1+wt1qlTSJT7j6w2wDzMqDg32ozNFaYuY1g5rTV9lDlUglQDoitQBJ9nIBSjf0++BqqJH0jRllcHcpQt3LZIHtjr/Yy5CmawJEANoUzKBbyDkPjLw8OMHyKs0rnP2JzPmX3w4tpXhRN+xDlMKUvUeV+4nVSPkRs+Q4QTujJPurL5HCqihhyyL0Tc9iLWPF3cvTf4lbSc0VYetkzOLfP1WNxTGXexp88fLFPZVg2qLQqNQ+B/xNY9TN1wDFLH5e+uZAZvKN3uh72kZo+oDw4+/bWPtgP92M7y2jwkBqvowqlX+dkRLyTh8zywPAWAdf8tQXzCHlgTi/QB4MFSrAji5xdtrkg1QupOaFL6Gz6J9MCDyW5ShOtSicyHByixjlBXesljnFrXC8lEXvwpWU0nj4XPCGDqmbx3QVvz3+VUUbWUCLy2B73uWHFwb4HZhh5/3yqeoQSGozG5lo5MzGBFWmh3AXz4Yv0EPYA8Q00IdhHXR2PF8WHLD08EQTcKoC9XId7DGQMj2CIk2Y7gAxw4eQLnQS2KuyO6lXrytX0wfySrxcd6XXh534hOfoYLARZD2pSMt1Rd1wDoyX0yter0/+S+a88gaIwJziIjyU8VETeyc8avo1aQttZ+lMItGaDTGirCk6g1D64Yq/R3BvEmqoT3Y4PFTBp/c7ub/RAHiA4+ybCPSkkjo88e/oXl//tg2B1ijiUzWpZS2yHu7KLYYHW9AlYHm/+TNL/qKW1iaArbeJB9XYYt3bYqvCiJ+iqdpFJXFTXaBDjU9rrswAc4c0ysAYSoNBe5NuPNLxfGOpru/LfHXlmETOlvIlFuhW37LD98Y0WrQ0lKqnrcWGdj6pU6ZJrUw4WSY21MjVV6Cyplxk8YoHC3zVLXcu4bEu8o7cCD0SreOISURzCflgdlvm3F3+dNwbY8AASxHJEeDB8shQTjHWfeug2qW2OUSUl2pJ0LtLZBaE dflQVnUg 0uJUWZRZVoSpxghtEcHOB4b9Up71AwQAHm2myk4NdYYPBd6B513V06S+voamX8hJIVwAAYvuLIenyVLdLeq+4FmpgMtN3292xkgoW4Br4CI4Vzedw2ETU+iODuvmD7jBSG8OhSBRdCnm4HD+KfaioZ4fE8aN3exB7k6RseyeGWGljI5FLiqE33rUqnSv/sQz6T+gQ4SD0UN3cdIKu9Hj+05Z42PhzdwSK8cJ1lv84Dz0LaohQyEl+KuzPYWRr0rhIsgTBsqOV2cN6ILMGX443khNws8aEMrn8NORRwSU4Vhmp9On+eFdE8BvsZP6pbd5lu38tqhg8Ygq1HpQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is Part II of the series [1] that makes zram read() and write() preemptible. This part focuses only zsmalloc because zsmalloc imposes atomicity restrictions on its users. One notable example is object mapping API, which returns with: a) local CPU lock held b) zspage rwlock held First, zsmalloc is converted to use sleepable RW-"lock" (it's atomic_t in fact) for zspage migration protection. Second, a new handle mapping is introduced which doesn't use per-CPU buffers (and hence no local CPU lock), does fewer memcpy() calls, but requires users to provide a pointer to temp buffer for object copy-in (when needed). Third, zram is converted to the new zsmalloc mapping API and thus zram read() becomes preemptible. [1] https://lore.kernel.org/linux-mm/20250127072932.1289973-1-senozhatsky@chromium.org RFC -> v1: - Only zspage->lock (leaf-lock for zs_map_object()) is converted to a preemptible lock. The rest of the zspool locks remain the same (Yosry hated with passion the fact that in RFC series all zspool looks would become preemptible). - New zs object mapping API (Yosry hated RFC API with passion). We know have obj_read_begin()/obj_read_end() and obj_write(). - obj_write() saves extra memcpy() calls for objects that span two physical pages. - Dropped zram deferred slot-free-notification handling (I hated it with passion) Sergey Senozhatsky (6): zsmalloc: factor out pool locking helpers zsmalloc: factor out size-class locking helpers zsmalloc: make zspage lock preemptible zsmalloc: introduce new object mapping API zram: switch to new zsmalloc object mapping API zram: add might_sleep to zcomp API drivers/block/zram/zcomp.c | 6 +- drivers/block/zram/zcomp.h | 2 + drivers/block/zram/zram_drv.c | 28 +-- include/linux/zsmalloc.h | 8 + mm/zsmalloc.c | 372 ++++++++++++++++++++++++++-------- 5 files changed, 311 insertions(+), 105 deletions(-)