From patchwork Mon Jan 27 07:59:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13951053 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3F58C0218F for ; Mon, 27 Jan 2025 08:03:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6DEA5280132; Mon, 27 Jan 2025 03:03:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 68E1C28012F; Mon, 27 Jan 2025 03:03:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 52E91280132; Mon, 27 Jan 2025 03:03:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 33DD028012F for ; Mon, 27 Jan 2025 03:03:14 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id DE434122070 for ; Mon, 27 Jan 2025 08:03:13 +0000 (UTC) X-FDA: 83052491466.09.D3786A9 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf01.hostedemail.com (Postfix) with ESMTP id 135BB40012 for ; Mon, 27 Jan 2025 08:03:11 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b="oY/PpW8Z"; spf=pass (imf01.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.177 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737964992; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9BKLEcG6n3DfmSf8Cjanopzri1qniom6oLk870hy4qs=; b=LBr4/ZtYe6Ul+7aURtI1rDhgl2T1hLOhLRa433sO7Zh4Pyscdki3H0aHXpuRSWnHsyv7xZ UMmphlwRGzx3ymyyXryyDeS4NVSxejfzBqf7OBPHQY1fcJDdCDu0jO7v3CNpyygY0IdfM5 zqKfdu9WeaRc1F47CtXx26WQXVxG8Ic= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b="oY/PpW8Z"; spf=pass (imf01.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.177 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737964992; a=rsa-sha256; cv=none; b=0xR8P5BaiLzSvR0QDqcIEfR1qSgdL396yq9wMvQd/uP0hO2VQJ12keWpmqbxlnLoy52jGC P4QEix6ui/nOaoP1bex8Fo0L41rqDOhSXmX1INsSq03BY00mmTS93eLWFC17AsmUOvDOEy rbQOe0H8E56oAiR52LjOC7BuAt+tg6Y= Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-21649a7bcdcso67392005ad.1 for ; Mon, 27 Jan 2025 00:03:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1737964991; x=1738569791; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9BKLEcG6n3DfmSf8Cjanopzri1qniom6oLk870hy4qs=; b=oY/PpW8Z+TNWX+VzsHwt+xem2kxGTDGKKR9vrzGzV5kVEVJraTwSZHoksi44+azxLO FALwGGqONOWJ1o85Aeb8L8LbLQbQ8d4vOMQ9Frabj6LdKBqB5150NC/lsVEB4VBF5BST ikCXL64Ye7R8mPmASGepwjN7BqWVAytWdbIZ8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737964991; x=1738569791; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9BKLEcG6n3DfmSf8Cjanopzri1qniom6oLk870hy4qs=; b=Cl5JmCWgi7N9zXYJhXUREIrWjgQKxaeDIJmI04hPi55MKQqfOXJLKxSPiv6P5QhyxR ykMUU1vumS0StPXtQX7iwlhRjYahh5DHCOA6+4WcrJj0tq7DvNdfhFm1rTlu6G3m375v wia01eSJ6RGTqmHaqke86uLSoyc788+YXZYuI4zDP9ECdD4kcu15OC5fxXxYwVas4Ss3 /6AvrFmRjVMa8WWaZzNln5Rt8JAr5vt8HCzf66rwPQvPhfkzSwe+dgT0oycUjNfXaDTu 6INf1SM4phWhqZaZUGZE6FangTC66uLFddbZ0uKFpJV1FW0Oqts10KgH65ReQvR7/EJv KMRQ== X-Gm-Message-State: AOJu0YynsAAw0Fl8YjrPrURz3v8+7XUWyx20BTOsKlLEUMhi49y5Zl2b XVMrBS6VZDy21EqT6fzlXYw4fL07lqgdFTIln7RV8tjTJuuC7UNJh2MNE1e6+g== X-Gm-Gg: ASbGncsgg9RH3uaxR3MQnTJysqnZTpaqr6FIDgXFfpOqUg8NEanEiPqxAFf/PJShh3o pg5i+BOoCVRkHEVms5qowqpN3wVD5xQMg7wogjSQQTkozD/1ZKf/DGzdfwF8gI/zdeO/hNDs0Aa M+ltbeVwP03fZ8DZVI34dZyLm4Tha/dGaU34X8l8iV/QKZms/S51N3rcpXKDmZ3PtteuuAp8iJ7 dmsuFP+h1NRWgGBhaUjY8oDyvAMB9kQR3DZr96iGX69Lqx9RP4G8tRwRqpuaBmyWwNnozLnx8IW gEPrCUA= X-Google-Smtp-Source: AGHT+IGkzFE/72UtLpMeB1N62Pwf7Zxax/xK4f/D9pilE4rZeetABBV2aUoAWwiZJw+ncza8QLiWeg== X-Received: by 2002:a17:903:124d:b0:21a:8dec:e57a with SMTP id d9443c01a7336-21c35619607mr540586485ad.48.1737964990824; Mon, 27 Jan 2025 00:03:10 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:566d:6152:c049:8d3a]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-21da3d9c9f2sm57195075ad.42.2025.01.27.00.03.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 27 Jan 2025 00:03:10 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton , Minchan Kim , Johannes Weiner , Yosry Ahmed , Nhat Pham Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [RFC PATCH 1/6] zram: deffer slot free notification Date: Mon, 27 Jan 2025 16:59:26 +0900 Message-ID: <20250127080254.1302026-2-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog In-Reply-To: <20250127080254.1302026-1-senozhatsky@chromium.org> References: <20250127080254.1302026-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 135BB40012 X-Stat-Signature: rt49j6twqpi1zya8x1utdd3o74c7fchw X-Rspamd-Server: rspam08 X-Rspam-User: X-HE-Tag: 1737964991-289767 X-HE-Meta: U2FsdGVkX1/wRRErf2cDb8M1ZnAv/jvtSVOpuYXM3d0TbEx27TcDQWWBJejcaLOaLxVtMwSI5UvvLoby17j9KE8BjCuEAqRT6OMtA3NVjA+ypGSpmFo9W/otisnv6VehWUrMVs2b1A7M+JkRc4nfofL/NKUnyelXORG+Mv0yKHDtXEeWn/HlKIK0WMGCv0d5R4Jd+td4dxcsfMKDzRsscby/EGI0bRgS6xQ0MHlCP9AnSfyGrcDKcXxdqPcx3sGzG7+7+sSQUrOQVrvQgDfh5cQKBpw9Zn4nEr8hM1RCh1/mYbqNKBpC3X9mGntSM22VyFqG/+MmQ6yAMehRsKnxMDqLeB91E6MNny5wCRN4Ap4NhmVh+aeF7OrmKioqoxwmuZ897OkzOu5ERNLEtj2+PEdRKNv98Sc7BG1tE4Abh38AptqMkJHRs6nm8anbjhwSAgkAuoQsmACdgDftQ84425TrT6hcSHKbwqsveCceoUlXD0AEzuCxdFJhOm3Y4djQQ/Vzt4gu9dfAoyDfEo9fXf8NKhwrT+Ikg9LPHb/5KVuKEb5QojFl4tO39d7B/PvU4he4a1ZNfmhWshR3pFPBuW0OBb7Z/KlrUDADqXebXdrZNyQ7z/6o9+JGNS3FbzsRtkdV4/BF6u8valoO+awBQsPzRmnUTzLdv4A05PRNpJrdCi/rkJNAiFdcWrHOaeLhYPYUMr18C78iMzhyH5h74jJDPiplK0glJ5g2lEMYfoVvDvwpgpitCvsEDf8mq27RYR2RrDUeHSPcmxFrIIpnHlbp5xQ8BLxfC6iI5fIJD0XLvr8r8osuYmgKXewjPMe3s/4Mu5BWfRaCRmUEWFg0/Gou2TzJMqKrLL1u9nULcVXJlUpIY+/Rykm0yfN/V0+pLN1tfemDiE4eB04K+RTivSbbnEmFFisOdZMzWgipuylyfiWubT+UXRdcu5QYYnXWobxTY8x3sHgqWbGBHwt G1M78oPR aDO6YAl+FLV8BJ5UJoU+tnrylmfpUSiKeZH48Olmy2ggJxCRtdSzJMC53z+RHwE+Fe6mlAeBH6TcZfgNuxQcD8rDL9U7F8uCeUzVNulIPWi+B1u7hokBQD3zT/Bmy36mmchcWj07NlFhCHZsYs3iSHEs3FokPscsWbje7ACFDckpC1USR5OrxNUIWFoPX5YGmNkIR6XsuDkYHC3apIafoijcPRJbIcRC1hVzLIb6DqUyu8y4Z3Jlf4m8JUHvufvmoAkeUVg1+N9tFKMSwS5M+g/hc+pbKrhJrdItC0BzkH9fxoZLcZjigoJwtew== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: As of this moment ->swap_slot_free_notify is called from atomic section (under spin-lock) which makes it impossible to make zsmalloc fully preemptible. Deffer slot-free to a non-atomic context. Signed-off-by: Sergey Senozhatsky --- drivers/block/zram/zram_drv.c | 66 +++++++++++++++++++++++++++++++++-- drivers/block/zram/zram_drv.h | 4 +++ 2 files changed, 68 insertions(+), 2 deletions(-) diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index ad3e8885b0d2..9c72beb86ab0 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -2315,9 +2315,52 @@ static void zram_submit_bio(struct bio *bio) } } +static void async_slot_free(struct work_struct *work) +{ + struct zram *zram = container_of(work, struct zram, slot_free_work); + + spin_lock(&zram->slot_free_lock); + while (!list_empty(&zram->slot_free_list)) { + struct zram_pp_slot *pps; + + pps = list_first_entry(&zram->slot_free_list, + struct zram_pp_slot, + entry); + list_del_init(&pps->entry); + spin_unlock(&zram->slot_free_lock); + + zram_slot_write_lock(zram, pps->index); + if (zram_test_flag(zram, pps->index, ZRAM_PP_SLOT)) + zram_free_page(zram, pps->index); + zram_slot_write_unlock(zram, pps->index); + + kfree(pps); + spin_lock(&zram->slot_free_lock); + } + spin_unlock(&zram->slot_free_lock); +}; + +static void zram_kick_slot_free(struct zram *zram) +{ + schedule_work(&zram->slot_free_work); +} + +static void zram_flush_slot_free(struct zram *zram) +{ + flush_work(&zram->slot_free_work); +} + +static void zram_init_slot_free(struct zram *zram) +{ + spin_lock_init(&zram->slot_free_lock); + INIT_LIST_HEAD(&zram->slot_free_list); + INIT_WORK(&zram->slot_free_work, async_slot_free); +} + static void zram_slot_free_notify(struct block_device *bdev, - unsigned long index) + unsigned long index) { + struct zram_pp_slot *pps; struct zram *zram; zram = bdev->bd_disk->private_data; @@ -2328,7 +2371,24 @@ static void zram_slot_free_notify(struct block_device *bdev, return; } - zram_free_page(zram, index); + if (zram_test_flag(zram, index, ZRAM_PP_SLOT)) + goto out; + + pps = kzalloc(sizeof(*pps), GFP_ATOMIC); + if (!pps) { + atomic64_inc(&zram->stats.miss_free); + goto out; + } + + INIT_LIST_HEAD(&pps->entry); + pps->index = index; + zram_set_flag(zram, index, ZRAM_PP_SLOT); + spin_lock(&zram->slot_free_lock); + list_add(&pps->entry, &zram->slot_free_list); + spin_unlock(&zram->slot_free_lock); + + zram_kick_slot_free(zram); +out: zram_slot_write_unlock(zram, index); } @@ -2473,6 +2533,7 @@ static ssize_t reset_store(struct device *dev, /* Make sure all the pending I/O are finished */ sync_blockdev(disk->part0); + zram_flush_slot_free(zram); zram_reset_device(zram); mutex_lock(&disk->open_mutex); @@ -2618,6 +2679,7 @@ static int zram_add(void) atomic_set(&zram->pp_in_progress, 0); zram_comp_params_reset(zram); comp_algorithm_set(zram, ZRAM_PRIMARY_COMP, default_compressor); + zram_init_slot_free(zram); /* Actual capacity set using sysfs (/sys/block/zram/disksize */ set_capacity(zram->disk, 0); diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index b7e250d6fa02..27ca269f4a4e 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -134,5 +134,9 @@ struct zram { struct dentry *debugfs_dir; #endif atomic_t pp_in_progress; + + spinlock_t slot_free_lock; + struct list_head slot_free_list; + struct work_struct slot_free_work; }; #endif