From patchwork Fri Feb 21 09:38:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergey Senozhatsky X-Patchwork-Id: 13985087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECB6FC021B3 for ; Fri, 21 Feb 2025 09:39:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 85E7A6B00B0; Fri, 21 Feb 2025 04:39:47 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 80A2C280008; Fri, 21 Feb 2025 04:39:47 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 686EC280004; Fri, 21 Feb 2025 04:39:47 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 48EE46B00B0 for ; Fri, 21 Feb 2025 04:39:47 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id F10E1C0BC6 for ; Fri, 21 Feb 2025 09:39:46 +0000 (UTC) X-FDA: 83143454772.04.A3C54D2 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by imf28.hostedemail.com (Postfix) with ESMTP id 0CAF7C0002 for ; Fri, 21 Feb 2025 09:39:44 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=jbV24U1I; spf=pass (imf28.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.182 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740130785; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=c43sUKc+c1340lwZZrmUjsY+PbdZGHYK9fwL3MZ7pbk=; b=13X4v9jV3LXjYCb+DuC2AHAk6XEJlYBht923dhYtjbrPbAZ1juXPjInbi1GHFLRnIuvqZN CHWsj8RBnW1TRvZCCzyriNSVNHThMznpSv5re7AaNLiFYWMApANn2GuKxRf8Q/QESSEHUs SOqi7ty1yhr1IrVLmLBhqfzB9rFZGPc= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=jbV24U1I; spf=pass (imf28.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.214.182 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org; dmarc=pass (policy=none) header.from=chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740130785; a=rsa-sha256; cv=none; b=b8F++FfxQ2V1Yr4Zgqej9GOZgxDQ5deAIICuq4NeXBUx3pbHD6yIRn4qBNPLoI71hs1lkg t++o73h5T7j/xQCYhawX/NIjv44NgePM58G0H9NqOdR7TETyJRmcTN1i/9XFn0tn8CsrkA m7zZqub+kkc+7PlZWkqtPA4H+3V9cVI= Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-220c665ef4cso30668425ad.3 for ; Fri, 21 Feb 2025 01:39:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1740130784; x=1740735584; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=c43sUKc+c1340lwZZrmUjsY+PbdZGHYK9fwL3MZ7pbk=; b=jbV24U1I/+QTU0v0GvihGFu8ekIFwefAXOlEu7bWXL1jTiQh8hQrq6vSlwU4cPCzPc 2rj3P30KAhVDKd5LbmhIOkh14ypqmuaEbBx5hieVblVK7nz1+4jgrBuzYZy/O8kCBdvq p4zEySu9aROv8GE29ZIpfRZjULlx6/G6ZBJ6c= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740130784; x=1740735584; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=c43sUKc+c1340lwZZrmUjsY+PbdZGHYK9fwL3MZ7pbk=; b=cYe4EcgxLUOYUfRE3aiHcVucD/axamEUB+2r0RQaY+TPwSgzOQjEzySubGLpvRfcvi A7oGl3UeRYLKbz8T6Of8IhNjRDCQAE6Jo8zq6PFjyXRO+udAUIs98Q0Ywa+Qr4syszrz 6Dm6sHuSVUaQPY2TuSMQtg7RP0SvNuyy8QebPVXz8U6ciNLqr5p+NJOaAR7LQRAVxRs3 rGI9zi/dcky6G4164svSdqxIck1R0bSiNCmILiesz/rRsNiIWa3Y0igqqq+er16I2bxP kgHEwpyTqSKLfTRv3DPpOj3Xl5WsJb5963lj0R2btDuufwckFs+x6F68FpW4Nuvjsj3k /+Jw== X-Forwarded-Encrypted: i=1; AJvYcCXz8Bn02BTN0uo4fXmWljPjcHTuiCG6O/IxkDeZP83f2dBWEIh3Kn/VizmiOTdTxMPKN21gfbypeQ==@kvack.org X-Gm-Message-State: AOJu0YxKpmIjZRcY2S4pY0BJLDL5DdBdagOYysErf+RvClWNtId7wX2e VtSdBE27VyCGWMctRvfX5USj1W2QDnztzOR8LCtomo49CLvDi1rZa1lGyH2VIA== X-Gm-Gg: ASbGncupGWAJw0g9xwiv9F6jSvXKeiztzrDt4kzT5Od+Wa0ZWigtPYktZa4Dls0rjwm mDn2aqqsRg0pICsc3HTKKq2hDe2hRWzkk1rK8gseBtjLP2zBXGcAlu+oMjdaht1c05Z5xjJzg4j MZ/OialGG/X2G510hNng4wuJUsLTJPf1n/uEHXCpIr6eLbiWnW2PIJhlfaNGEUSngqIYTAfQqK9 ADVKc5rwkA2XA0XNS9rJlONo2XBDBrwiF4gj5uTc3MyzH1FHb9rPGlZ7kHpwY7BqAbaf86Y1XrB jvkiyKr40tWQhZcW/8i1jp6Ui08= X-Google-Smtp-Source: AGHT+IGQLO+1U1vS6Lct91u4p8i4NavSONuQ6g0x/XphxsfnVjWvF85vBjeEIzZsrJ0hPSWktsRKtw== X-Received: by 2002:a17:902:e783:b0:219:e4b0:4286 with SMTP id d9443c01a7336-2219ffc00d7mr41532145ad.29.1740130783941; Fri, 21 Feb 2025 01:39:43 -0800 (PST) Received: from localhost ([2401:fa00:8f:203:f987:e1e:3dbb:2191]) by smtp.gmail.com with UTF8SMTPSA id d9443c01a7336-220d545d4bcsm132919005ad.144.2025.02.21.01.39.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 21 Feb 2025 01:39:43 -0800 (PST) From: Sergey Senozhatsky To: Andrew Morton Cc: Yosry Ahmed , Hillf Danton , Kairui Song , Sebastian Andrzej Siewior , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Sergey Senozhatsky Subject: [PATCH v7 11/17] zsmalloc: make zspage lock preemptible Date: Fri, 21 Feb 2025 18:38:04 +0900 Message-ID: <20250221093832.1949691-12-senozhatsky@chromium.org> X-Mailer: git-send-email 2.48.1.601.g30ceb7b040-goog In-Reply-To: <20250221093832.1949691-1-senozhatsky@chromium.org> References: <20250221093832.1949691-1-senozhatsky@chromium.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 0CAF7C0002 X-Stat-Signature: nbrcxt8zaynf34ghkpfno3a8ni7jrwur X-HE-Tag: 1740130784-729288 X-HE-Meta: U2FsdGVkX1/pJcemnVWSMUaPkqt06MeUULAJqQeMkcHU5u+jX9Siffh1jr2Pm6G2jQ/RqqHdnVsgWKnjue7sye606ankVZjO6U3zHNHNOPDgcAvuSyEvMFF3uHxRyYK+0afr6iWlJYiKUflV/f7Vm/fE8GjKv53QDJSy+mrEmUgsQH+6P0GmqQ+bXXQoKYe3Fn0wzeE/Z11tnlH+Fk/zkBXmP5XM1DZNwCxCQsS0Dbc8Qfjy4Cs5YeVFcUsfl0kdWTjfI0AkkIjUowsvhi7ow/D70lANyJsDl5VmVnDnrb54B5AZAV+5Jhk/cDz64nTrHjFEJo3vlROKtmXZL56JHtsNq5k52gRtjEkJ/Bn2OYlyL7IgB6ObNuQvIDnK7CJlCLUKRge48Sa6A/IuR/PCjNAgN0CJY52CTOXY6Mw1XbgaYo6pKP/z1fWNYu4EWd5bDr6IQ5BlKyhqo1Scq6pwdHfuJC8ofG4OnBAwVxA8cCpd9QW7m6NMU9l8VYvrMfga3PxBE2x6ZtHL3ua9gJb4cZDPNAHVptwGoEPoa/ymVVezgG/4kM/kMX3oTPWsgIKiWgzbpJ/f26udW6vOk8t9QNN1DebKTS/b5+hezxBD/NuLh2lLrtFFdZxPUoXMw5NkOCz6yn3xdvC7xbf+yIpxAqHK6vtojIDOQYAOhIZsimB8pzk830H1GVdF7d7fGIPAxWi4gdoZ0sQHRRywiXfxzOMfqL8LpaujsSYE7fojz61LRkHFxqbXDlSnn7WbQZSkodQL4gaUNooBAFcDZulDL/aXj6ATU1ZafejTRJJEGg9RZBHn6yr4ZzyB343/NOmLpnv78o8xX8gqR0jhGNCnlOn0Z0BHeeH/AbHsaTiVBZ7TCwvzEWOQ7heQD5ruBMmM/1Qjcrn6FZ1y3pzR0E6lHDWFWH313tBq/SWtEJFsO7Pyxe1O9os3tY2/GA93y/Dyb1KTZ1jrqQrGiJpyQRq hcLxaK/y 8z661OXAf5gWsuTc2In4M1jHLhsSlBH1zPYBOSEl8BTg1yAorhhfk4CeUJHhgqW9sfQVXF5bOGtW5jQyrFOeTAmGpvLuyDY8SHfvVEY0BvGANPGO1eKnxmI2nVtyNBz/dyEMj95v09k2u9w2TfgOmUu2ofi8OJV4cju15NQT51pmR3/m7cXUgH3QMY/iYN+QC7R5DkgcMSUm+3T8U1ILL6NKLck0OCkEm7hkQsPgQ8t3gqzzx670lXJ0qk9Ks95HFyTfGYVf+OzklAmfy7ibOWBoHEi7Azq3jVzP3jrrUgFh9zASpP4XZIFbVrFdfNArV3SPRtvtK12FrjqM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In order to implement preemptible object mapping we need a zspage lock that satisfies several preconditions: - it should be reader-write type of a lock - it should be possible to hold it from any context, but also being preemptible if the context allows it - we never sleep while acquiring but can sleep while holding in read mode An rwsemaphore doesn't suffice, due to atomicity requirements, rwlock doesn't satisfy due to reader-preemptability requirement. It's also worth to mention, that per-zspage rwsem is a little too memory heavy (we can easily have double digits megabytes used only on rwsemaphores). Switch over from rwlock_t to a atomic_t-based implementation of a reader-writer semaphore that satisfies all of the preconditions. The spin-lock based zspage_lock is suggested by Hillf Danton. Suggested-by: Hillf Danton Signed-off-by: Sergey Senozhatsky --- mm/zsmalloc.c | 189 +++++++++++++++++++++++++++++++++++--------------- 1 file changed, 135 insertions(+), 54 deletions(-) diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c index 1424ee73cbb5..250f1fddaf34 100644 --- a/mm/zsmalloc.c +++ b/mm/zsmalloc.c @@ -226,6 +226,9 @@ struct zs_pool { /* protect zspage migration/compaction */ rwlock_t lock; atomic_t compaction_in_progress; +#ifdef CONFIG_DEBUG_LOCK_ALLOC + struct lock_class_key lock_class; +#endif }; static inline void zpdesc_set_first(struct zpdesc *zpdesc) @@ -257,6 +260,18 @@ static inline void free_zpdesc(struct zpdesc *zpdesc) __free_page(page); } +#define ZS_PAGE_UNLOCKED 0 +#define ZS_PAGE_WRLOCKED -1 + +struct zspage_lock { + spinlock_t lock; + int cnt; + +#ifdef CONFIG_DEBUG_LOCK_ALLOC + struct lockdep_map dep_map; +#endif +}; + struct zspage { struct { unsigned int huge:HUGE_BITS; @@ -269,7 +284,7 @@ struct zspage { struct zpdesc *first_zpdesc; struct list_head list; /* fullness list */ struct zs_pool *pool; - rwlock_t lock; + struct zspage_lock zsl; }; struct mapping_area { @@ -279,6 +294,91 @@ struct mapping_area { enum zs_mapmode vm_mm; /* mapping mode */ }; +#ifdef CONFIG_DEBUG_LOCK_ALLOC +#define zsl_dep_map(zsl) (&(zsl)->dep_map) +#else +#define zsl_dep_map(zsl) NULL +#endif + +static void zspage_lock_init(struct zspage *zspage) +{ +#ifdef CONFIG_DEBUG_LOCK_ALLOC + lockdep_init_map(&zspage->zsl.dep_map, "zspage->lock", + &zspage->pool->lock_class, 0); +#endif + spin_lock_init(&zspage->zsl.lock); + zspage->zsl.cnt = ZS_PAGE_UNLOCKED; +} + +/* + * The zspage lock can be held from atomic contexts, but it needs to remain + * preemptible when held for reading because it remains held outside of those + * atomic contexts, otherwise we unnecessarily lose preemptibility. + * + * To achieve this, the following rules are enforced on readers and writers: + * + * - Writers are blocked by both writers and readers, while readers are only + * blocked by writers (i.e. normal rwlock semantics). + * + * - Writers are always atomic (to allow readers to spin waiting for them). + * + * - Writers always use trylock (as the lock may be held be sleeping readers). + * + * - Readers may spin on the lock (as they can only wait for atomic writers). + * + * - Readers may sleep while holding the lock (as writes only use trylock). + */ +static void zspage_read_lock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + rwsem_acquire_read(zsl_dep_map(zsl), 0, 0, _RET_IP_); + + spin_lock(&zsl->lock); + zsl->cnt++; + spin_unlock(&zsl->lock); + + lock_acquired(zsl_dep_map(zsl), _RET_IP_); +} + +static void zspage_read_unlock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + rwsem_release(zsl_dep_map(zsl), _RET_IP_); + + spin_lock(&zsl->lock); + zsl->cnt--; + spin_unlock(&zsl->lock); +} + +static __must_check bool zspage_write_trylock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + spin_lock(&zsl->lock); + if (zsl->cnt == ZS_PAGE_UNLOCKED) { + zsl->cnt = ZS_PAGE_WRLOCKED; + rwsem_acquire(zsl_dep_map(zsl), 0, 1, _RET_IP_); + lock_acquired(zsl_dep_map(zsl), _RET_IP_); + return true; + } + + lock_contended(zsl_dep_map(zsl), _RET_IP_); + spin_unlock(&zsl->lock); + return false; +} + +static void zspage_write_unlock(struct zspage *zspage) +{ + struct zspage_lock *zsl = &zspage->zsl; + + rwsem_release(zsl_dep_map(zsl), _RET_IP_); + + zsl->cnt = ZS_PAGE_UNLOCKED; + spin_unlock(&zsl->lock); +} + /* huge object: pages_per_zspage == 1 && maxobj_per_zspage == 1 */ static void SetZsHugePage(struct zspage *zspage) { @@ -290,12 +390,6 @@ static bool ZsHugePage(struct zspage *zspage) return zspage->huge; } -static void migrate_lock_init(struct zspage *zspage); -static void migrate_read_lock(struct zspage *zspage); -static void migrate_read_unlock(struct zspage *zspage); -static void migrate_write_lock(struct zspage *zspage); -static void migrate_write_unlock(struct zspage *zspage); - #ifdef CONFIG_COMPACTION static void kick_deferred_free(struct zs_pool *pool); static void init_deferred_free(struct zs_pool *pool); @@ -992,7 +1086,9 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, return NULL; zspage->magic = ZSPAGE_MAGIC; - migrate_lock_init(zspage); + zspage->pool = pool; + zspage->class = class->index; + zspage_lock_init(zspage); for (i = 0; i < class->pages_per_zspage; i++) { struct zpdesc *zpdesc; @@ -1015,8 +1111,6 @@ static struct zspage *alloc_zspage(struct zs_pool *pool, create_page_chain(class, zspage, zpdescs); init_zspage(class, zspage); - zspage->pool = pool; - zspage->class = class->index; return zspage; } @@ -1217,7 +1311,7 @@ void *zs_map_object(struct zs_pool *pool, unsigned long handle, * zs_unmap_object API so delegate the locking from class to zspage * which is smaller granularity. */ - migrate_read_lock(zspage); + zspage_read_lock(zspage); read_unlock(&pool->lock); class = zspage_class(pool, zspage); @@ -1277,7 +1371,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle) } local_unlock(&zs_map_area.lock); - migrate_read_unlock(zspage); + zspage_read_unlock(zspage); } EXPORT_SYMBOL_GPL(zs_unmap_object); @@ -1671,18 +1765,18 @@ static void lock_zspage(struct zspage *zspage) /* * Pages we haven't locked yet can be migrated off the list while we're * trying to lock them, so we need to be careful and only attempt to - * lock each page under migrate_read_lock(). Otherwise, the page we lock + * lock each page under zspage_read_lock(). Otherwise, the page we lock * may no longer belong to the zspage. This means that we may wait for * the wrong page to unlock, so we must take a reference to the page - * prior to waiting for it to unlock outside migrate_read_lock(). + * prior to waiting for it to unlock outside zspage_read_lock(). */ while (1) { - migrate_read_lock(zspage); + zspage_read_lock(zspage); zpdesc = get_first_zpdesc(zspage); if (zpdesc_trylock(zpdesc)) break; zpdesc_get(zpdesc); - migrate_read_unlock(zspage); + zspage_read_unlock(zspage); zpdesc_wait_locked(zpdesc); zpdesc_put(zpdesc); } @@ -1693,41 +1787,16 @@ static void lock_zspage(struct zspage *zspage) curr_zpdesc = zpdesc; } else { zpdesc_get(zpdesc); - migrate_read_unlock(zspage); + zspage_read_unlock(zspage); zpdesc_wait_locked(zpdesc); zpdesc_put(zpdesc); - migrate_read_lock(zspage); + zspage_read_lock(zspage); } } - migrate_read_unlock(zspage); + zspage_read_unlock(zspage); } #endif /* CONFIG_COMPACTION */ -static void migrate_lock_init(struct zspage *zspage) -{ - rwlock_init(&zspage->lock); -} - -static void migrate_read_lock(struct zspage *zspage) __acquires(&zspage->lock) -{ - read_lock(&zspage->lock); -} - -static void migrate_read_unlock(struct zspage *zspage) __releases(&zspage->lock) -{ - read_unlock(&zspage->lock); -} - -static void migrate_write_lock(struct zspage *zspage) -{ - write_lock(&zspage->lock); -} - -static void migrate_write_unlock(struct zspage *zspage) -{ - write_unlock(&zspage->lock); -} - #ifdef CONFIG_COMPACTION static const struct movable_operations zsmalloc_mops; @@ -1769,7 +1838,7 @@ static bool zs_page_isolate(struct page *page, isolate_mode_t mode) } static int zs_page_migrate(struct page *newpage, struct page *page, - enum migrate_mode mode) + enum migrate_mode mode) { struct zs_pool *pool; struct size_class *class; @@ -1785,9 +1854,6 @@ static int zs_page_migrate(struct page *newpage, struct page *page, VM_BUG_ON_PAGE(!zpdesc_is_isolated(zpdesc), zpdesc_page(zpdesc)); - /* We're committed, tell the world that this is a Zsmalloc page. */ - __zpdesc_set_zsmalloc(newzpdesc); - /* The page is locked, so this pointer must remain valid */ zspage = get_zspage(zpdesc); pool = zspage->pool; @@ -1803,8 +1869,15 @@ static int zs_page_migrate(struct page *newpage, struct page *page, * the class lock protects zpage alloc/free in the zspage. */ spin_lock(&class->lock); - /* the migrate_write_lock protects zpage access via zs_map_object */ - migrate_write_lock(zspage); + /* the zspage write_lock protects zpage access via zs_map_object */ + if (!zspage_write_trylock(zspage)) { + spin_unlock(&class->lock); + write_unlock(&pool->lock); + return -EINVAL; + } + + /* We're committed, tell the world that this is a Zsmalloc page. */ + __zpdesc_set_zsmalloc(newzpdesc); offset = get_first_obj_offset(zpdesc); s_addr = kmap_local_zpdesc(zpdesc); @@ -1835,7 +1908,7 @@ static int zs_page_migrate(struct page *newpage, struct page *page, */ write_unlock(&pool->lock); spin_unlock(&class->lock); - migrate_write_unlock(zspage); + zspage_write_unlock(zspage); zpdesc_get(newzpdesc); if (zpdesc_zone(newzpdesc) != zpdesc_zone(zpdesc)) { @@ -1971,9 +2044,11 @@ static unsigned long __zs_compact(struct zs_pool *pool, if (!src_zspage) break; - migrate_write_lock(src_zspage); + if (!zspage_write_trylock(src_zspage)) + break; + migrate_zspage(pool, src_zspage, dst_zspage); - migrate_write_unlock(src_zspage); + zspage_write_unlock(src_zspage); fg = putback_zspage(class, src_zspage); if (fg == ZS_INUSE_RATIO_0) { @@ -2233,7 +2308,9 @@ struct zs_pool *zs_create_pool(const char *name) * trigger compaction manually. Thus, ignore return code. */ zs_register_shrinker(pool); - +#ifdef CONFIG_DEBUG_LOCK_ALLOC + lockdep_register_key(&pool->lock_class); +#endif return pool; err: @@ -2270,6 +2347,10 @@ void zs_destroy_pool(struct zs_pool *pool) kfree(class); } +#ifdef CONFIG_DEBUG_LOCK_ALLOC + lockdep_unregister_key(&pool->lock_class); +#endif + destroy_cache(pool); kfree(pool->name); kfree(pool);