From patchwork Fri Oct 11 10:20:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "yuan.gao" X-Patchwork-Id: 13832339 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76C74CFD316 for ; Fri, 11 Oct 2024 10:21:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0150E6B00B2; Fri, 11 Oct 2024 06:21:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F06C66B00B3; Fri, 11 Oct 2024 06:21:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA77B6B00B4; Fri, 11 Oct 2024 06:21:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id BAD666B00B2 for ; Fri, 11 Oct 2024 06:21:11 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 76F3BA0F36 for ; Fri, 11 Oct 2024 10:21:03 +0000 (UTC) X-FDA: 82660928700.03.DB65B44 Received: from mail-m11887.qiye.163.com (mail-m11887.qiye.163.com [115.236.118.87]) by imf19.hostedemail.com (Postfix) with ESMTP id 20D6B1A0011 for ; Fri, 11 Oct 2024 10:21:05 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=ucloud.cn header.s=default header.b=YvxwWZoI; dmarc=pass (policy=quarantine) header.from=ucloud.cn; spf=pass (imf19.hostedemail.com: domain of yuan.gao@ucloud.cn designates 115.236.118.87 as permitted sender) smtp.mailfrom=yuan.gao@ucloud.cn ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728641957; a=rsa-sha256; cv=none; b=ap6puO0VmKvm4bzE7NKHjS1uibWKDoWaND5m/FJbZSmP7jX4ff+mnTx/RW0cj/VxxkVo/1 fCHOakDjjbzXkikz6UbLcqQtPLgw96KkfaYkkHAo0OeP9NDD+3G05+oPnmqgQViY6NxmxM RUPYlJrmuaL3LXFvmhWYWyyk2zF0FVw= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=ucloud.cn header.s=default header.b=YvxwWZoI; dmarc=pass (policy=quarantine) header.from=ucloud.cn; spf=pass (imf19.hostedemail.com: domain of yuan.gao@ucloud.cn designates 115.236.118.87 as permitted sender) smtp.mailfrom=yuan.gao@ucloud.cn ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728641957; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=z5v8GyfWYgxGOEi26tfofm7WMNl1vBshfrvXCTiKfrM=; b=gzOETFuvChC5x3qbCJ4qsYnOcnkMyO2sJLVen8FsJZAYZ9haaHFjOfiaKhOVCcP2/43JID lYlvkUPEYwJxa3LtjH0DsqI72Of+E5Q4sTesOZ3dxAz0kLNDUmqS6MfoMeX00VMGiy6bb/ M2kodo9ANS3gXIoF21z6DogTiRhPeLs= DKIM-Signature: a=rsa-sha256; b=YvxwWZoIFVKDU/fd3+E4O9Sb3XGm8riIZTECpdu7/WccHWmXsHIWLb3ZV5R7pfExhdQoC9uPDVpt9Wua6iywqa+XMng/F2N6s4aYLOo7h96DtIORb+6gkPIkV+5fbTHkVK+KleF3/GT0Cd2MApt30v8NTLeNgbvIuQgRa+QO2hM=; s=default; c=relaxed/relaxed; d=ucloud.cn; v=1; bh=z5v8GyfWYgxGOEi26tfofm7WMNl1vBshfrvXCTiKfrM=; h=date:mime-version:subject:message-id:from; Received: from localhost.localdomain (unknown [106.75.220.2]) by smtp.qiye.163.com (Hmail) with ESMTPA id 87B6C66041C; Fri, 11 Oct 2024 18:20:24 +0800 (CST) From: "yuan.gao" To: cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, akpm@linux-foundation.org, vbabka@suse.cz, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, yuan.gao@ucloud.cn Subject: [PATCH v3] mm/slub: Avoid list corruption when removing a slab from the full list Date: Fri, 11 Oct 2024 18:20:20 +0800 Message-Id: <20241011102020.58087-1-yuan.gao@ucloud.cn> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 X-HM-Spam-Status: e1kfGhgUHx5ZQUpXWQgPGg8OCBgUHx5ZQUlOS1dZFg8aDwILHllBWSg2Ly tZV1koWUFJQjdXWS1ZQUlXWQ8JGhUIEh9ZQVlCSUIfVkpMSR9DGExLTEpPQ1YVFAkWGhdVGRETFh oSFyQUDg9ZV1kYEgtZQVlKS01VTE5VSUlLVUlZV1kWGg8SFR0UWUFZT0tIVUpLSU9PT0hVSktLVU pCS0tZBg++ X-HM-Tid: 0a927b18064b023bkunm87b6c66041c X-HM-MType: 1 X-HM-Sender-Digest: e1kMHhlZQR0aFwgeV1kSHx4VD1lBWUc6PxA6Dyo4GjcaDT4cDyE5E00N FlEaCk9VSlVKTElDTU9JS0lOQ0NNVTMWGhIXVQIOGhVVHBoUOw4YFxQOH1UYFUVZV1kSC1lBWUpL TVVMTlVJSUtVSVlXWQgBWUFNTE5KNwY+ X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 20D6B1A0011 X-Stat-Signature: auhx4nwrt8sp9hofjecgp1enw94zaght X-Rspam-User: X-HE-Tag: 1728642065-521003 X-HE-Meta: U2FsdGVkX1+DqH1U4/Qx51kqM753B4FN7vpQLK19xWkEoTWXzilxg7hq0Omm2i5KSkgNBrAA+t8HYZjIdiQnEqgPILP6AQPsk9nhNzbrq74y68DUOnhUrUHdrOk6ANc03TAuxgX8dTJn7+dzKCy5fTAbwbqTRbujm1m6rT64TjfarMQdRxiv5uqlmk8/MmL1eK2fc+htSnLGuDKJj++2CP/Jc/vFT/LQm8WmH5RLVyVtpCVlLZDPLac0Hl+H3DdeGfkka1792VO56tSdnapjOniEVYfpMFJ21SFKGe0XoyXB2W6Uhv2DNFdYw/1+RsLu7ywbMoZ/PChKiu6YfQNcEB6wl9w+ApI7fgtbIM+pjS8ww4+MnKeQ/b9p+EFIIxkX4OnKBE+34qyh35Kj117WPTH4x9nn3RyOkTx5Veqv/upLHlQpw1hr4sT9nXUbdOEMFRXDAlnubRfGRdRFl8ckrioO9Q8aoKI9zSvWOTRK4Y+2q8XJDDygiQiFNGHzKTqC3T4gWDg9IWPrW9Vod7yhHFV3o3ZrTcvFW/gsbVPdMJcDiqplE3kdaNiyP6f2VMl6gfO7gdLZaSNu5PxmCb4jEanFMFKvNup1C06qcCZGyqqUXNOWpCO+U0uC6bF4JmwpkV63ssm0Q8kNO+moDwobxynDWtIuAwpMSEK1vhk3KPpoa+U5+UTnCDKNKgW/krmlLoweR5mYmSYVLnDRNqZ3t3uUyBOOLXSE+eIUvy9WRu8PUp0kOGc5BGSfdEti+KrpC65m5fVU0gleokOHFhX02/iJ4DthDCNTHieL3ryLhttYaqdyCrHfFgk0E9qhOavQhDpW4SlVA1gXMkBiKLjqZc2y01Z0eMLh/BtMy1KmUnVm564IOwQTJEvmFHphOq9OQgGOpRVOAlzW2aQnScjn1VcsKWX+dIgNrgcdDufX039BaYcCJ2cnSeIJpXQfbdJ/MFiSDu27eDpfEJewFJp d0W9N9EJ nX1Cn44Bf3pg7mtlMc60wudBq8y5OnG+bMDCggF30tdH9HHMDpY+qB91ih3kymlKn9xMpnWo77dZ8O51iTrJNGVtH3IScEAKffcLKkqH5vDn6FUuWtPkvI8lm+/paPPWVV8blj+dV1KfXp16UmZB3R8/qEVxgaa0btz5KP+N52x9onk1zG5rwx/5ZybOXrLcb8x8n5mpAMAnlPlgvTuFlZFzjBoXD0jjCkiobTfHfi199bkFoVfmVqnFRcRXruJnluQ72hAr7XbxF/KgM/dibDN2ystJttutQnYOktH7I5iSA87ps3dkpoJJJrCrR2d3puwFjDNO7z6gRSX9vgF0vBsOzIcQpyA2q+gUOd6gAMIVB8yGBWn0fHdWd8xcFSwXHZKEC1giBc9jvFc1YTwqDAzNMLt08NtmtmrQT X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Boot with slub_debug=UFPZ. If allocated object failed in alloc_consistency_checks, all objects of the slab will be marked as used, and then the slab will be removed from the partial list. When an object belonging to the slab got freed later, the remove_full() function is called. Because the slab is neither on the partial list nor on the full list, it eventually lead to a list corruption. So we need to mark and isolate the corrupted slab page, do not put it back in circulation. Because the debug caches avoid all the fastpaths, reusing the frozen bit seems to be fine. [ 4277.385669] list_del corruption, ffffea00044b3e50->next is LIST_POISON1 (dead000000000100) [ 4277.387023] ------------[ cut here ]------------ [ 4277.387880] kernel BUG at lib/list_debug.c:56! [ 4277.388680] invalid opcode: 0000 [#1] PREEMPT SMP PTI [ 4277.389562] CPU: 5 PID: 90 Comm: kworker/5:1 Kdump: loaded Tainted: G OE 6.6.1-1 #1 [ 4277.392113] Workqueue: xfs-inodegc/vda1 xfs_inodegc_worker [xfs] [ 4277.393551] RIP: 0010:__list_del_entry_valid_or_report+0x7b/0xc0 [ 4277.394518] Code: 48 91 82 e8 37 f9 9a ff 0f 0b 48 89 fe 48 c7 c7 28 49 91 82 e8 26 f9 9a ff 0f 0b 48 89 fe 48 c7 c7 58 49 91 [ 4277.397292] RSP: 0018:ffffc90000333b38 EFLAGS: 00010082 [ 4277.398202] RAX: 000000000000004e RBX: ffffea00044b3e50 RCX: 0000000000000000 [ 4277.399340] RDX: 0000000000000002 RSI: ffffffff828f8715 RDI: 00000000ffffffff [ 4277.400545] RBP: ffffea00044b3e40 R08: 0000000000000000 R09: ffffc900003339f0 [ 4277.401710] R10: 0000000000000003 R11: ffffffff82d44088 R12: ffff888112cf9910 [ 4277.402887] R13: 0000000000000001 R14: 0000000000000001 R15: ffff8881000424c0 [ 4277.404049] FS: 0000000000000000(0000) GS:ffff88842fd40000(0000) knlGS:0000000000000000 [ 4277.405357] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 4277.406389] CR2: 00007f2ad0b24000 CR3: 0000000102a3a006 CR4: 00000000007706e0 [ 4277.407589] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 4277.408780] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 4277.410000] PKRU: 55555554 [ 4277.410645] Call Trace: [ 4277.411234] [ 4277.411777] ? die+0x32/0x80 [ 4277.412439] ? do_trap+0xd6/0x100 [ 4277.413150] ? __list_del_entry_valid_or_report+0x7b/0xc0 [ 4277.414158] ? do_error_trap+0x6a/0x90 [ 4277.414948] ? __list_del_entry_valid_or_report+0x7b/0xc0 [ 4277.415915] ? exc_invalid_op+0x4c/0x60 [ 4277.416710] ? __list_del_entry_valid_or_report+0x7b/0xc0 [ 4277.417675] ? asm_exc_invalid_op+0x16/0x20 [ 4277.418482] ? __list_del_entry_valid_or_report+0x7b/0xc0 [ 4277.419466] ? __list_del_entry_valid_or_report+0x7b/0xc0 [ 4277.420410] free_to_partial_list+0x515/0x5e0 [ 4277.421242] ? xfs_iext_remove+0x41a/0xa10 [xfs] [ 4277.422298] xfs_iext_remove+0x41a/0xa10 [xfs] [ 4277.423316] ? xfs_inodegc_worker+0xb4/0x1a0 [xfs] [ 4277.424383] xfs_bmap_del_extent_delay+0x4fe/0x7d0 [xfs] [ 4277.425490] __xfs_bunmapi+0x50d/0x840 [xfs] [ 4277.426445] xfs_itruncate_extents_flags+0x13a/0x490 [xfs] [ 4277.427553] xfs_inactive_truncate+0xa3/0x120 [xfs] [ 4277.428567] xfs_inactive+0x22d/0x290 [xfs] [ 4277.429500] xfs_inodegc_worker+0xb4/0x1a0 [xfs] [ 4277.430479] process_one_work+0x171/0x340 [ 4277.431227] worker_thread+0x277/0x390 [ 4277.431962] ? __pfx_worker_thread+0x10/0x10 [ 4277.432752] kthread+0xf0/0x120 [ 4277.433382] ? __pfx_kthread+0x10/0x10 [ 4277.434134] ret_from_fork+0x2d/0x50 [ 4277.434837] ? __pfx_kthread+0x10/0x10 [ 4277.435566] ret_from_fork_asm+0x1b/0x30 [ 4277.436280] v3: - Reuse slab->fronzen bit as a corrupted marker. v2: - Call remove_partial() and add_full() only for slab folios. - https://lore.kernel.org/linux-mm/20241007091850.16959-1-yuan.gao@ucloud.cn/ v1: - https://lore.kernel.org/linux-mm/20241006044755.79634-1-yuan.gao@ucloud.cn/ Signed-off-by: yuan.gao Fixes: 643b113849d8 ("slub: enable tracking of full slabs") Acked-by: Christoph Lameter Suggested-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Suggested-by: Vlastimil Babka --- mm/slab.h | 4 ++++ mm/slub.c | 11 ++++++++++- 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/mm/slab.h b/mm/slab.h index 6c6fe6d630ce..7681e71d9a13 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -73,6 +73,10 @@ struct slab { struct { unsigned inuse:16; unsigned objects:15; + /* + * Reuse frozen bit for slab with debug enabled: + * frozen == 1 means it is corrupted + */ unsigned frozen:1; }; }; diff --git a/mm/slub.c b/mm/slub.c index 5b832512044e..b9265e9f11aa 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -1423,6 +1423,11 @@ static int check_slab(struct kmem_cache *s, struct slab *slab) slab->inuse, slab->objects); return 0; } + if (slab->frozen) { + slab_err(s, slab, "Corrupted slab"); + return 0; + } + /* Slab_pad_check fixes things up after itself */ slab_pad_check(s, slab); return 1; @@ -1603,6 +1608,7 @@ static noinline bool alloc_debug_processing(struct kmem_cache *s, slab_fix(s, "Marking all objects used"); slab->inuse = slab->objects; slab->freelist = NULL; + slab->frozen = 1; /* mark consistency-failed slab as frozen */ } return false; } @@ -2744,7 +2750,10 @@ static void *alloc_single_from_partial(struct kmem_cache *s, slab->inuse++; if (!alloc_debug_processing(s, slab, object, orig_size)) { - remove_partial(n, slab); + if (folio_test_slab(slab_folio(slab))) { + remove_partial(n, slab); + add_full(s, n, slab); + } return NULL; }