From patchwork Fri Apr 26 21:55:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leah Rumancik X-Patchwork-Id: 13645424 Received: from mail-pl1-f173.google.com (mail-pl1-f173.google.com [209.85.214.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6337639FF2 for ; Fri, 26 Apr 2024 21:56:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714168568; cv=none; b=eILDfMPOQdXir/nGCnWzq9+c0yEyeldpFCXBOKYG+xrZY1oCCSfZdF+GY8vJTTpKd++hh0VnngMZ33OX9oO+wJLVBq+2fKZEOXGwp8JJe5qmicuRCXZ0523JmcIPrURp6Jliihz3d/GwV10jZFbCyxJlbL7nwcIinBR8+JXE76o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714168568; c=relaxed/simple; bh=K0CYLz+X0WaXFgwOIZ9fp3/KM0f7gN+QqCDJ8lRzRwI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NUR/grq24w3t9zG0GeVWIjprYCLMB8uT1ES7w0HYDzMZ8Z8nUFigQ74nnnKX+jScCTxW6gFA53AmPO7Nz/MZ8oOQaxyzxXsaPQwsVSiBsTKbjlO22LIVCGTEClV//XlqKqyckUyK6hjCtLSSY2g9nJyiVMyNQIPRlDp4TGg8r7E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=An0n5bdu; arc=none smtp.client-ip=209.85.214.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="An0n5bdu" Received: by mail-pl1-f173.google.com with SMTP id d9443c01a7336-1e3ff14f249so20158225ad.1 for ; Fri, 26 Apr 2024 14:56:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714168566; x=1714773366; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Y6LUDHkv9eYKxn5TY/+wGN2T9MmB63zoVimXrOLrpvw=; b=An0n5bduarcPDMux+Wp/2rsStbGTEsnRp1ZJlN6MJpsjz4AbXPTBwP5Bud1LISfgLD QmMwNZxDpMb7H6vU5asRh30zUq4PKystBsIpu3YFZIV1czzl4eknxo2uWdqTVXGaY9wQ X0GMbQ7njhhZuY1dS8kJY0Jr4DHD9AEYBv4OV5CGtUTHL44l/gIq1tMpXeK1J6g3CaZc M2/EEZ88kr7xIxYumsM3izVDCp18czwrEsovYeES/XiEseEpNc+iiRcr+k6AMKMziZyF Vxd1BfFKxmBnK/jz8kdW5rzBv5DFvRJHsR/+qvhmkzIuBGeN353LIIbh7H81o+FraJZj cMLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714168566; x=1714773366; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Y6LUDHkv9eYKxn5TY/+wGN2T9MmB63zoVimXrOLrpvw=; b=ER5J7wzKxlRG02aXKSY2Xbgm9X7XAKK4dbfEHTymdIwa0K026Hzjs2Rw/tvLtNAEwP 0tKgmXVtR4Q4i8mKFY3JeZDeHH3vPm0FXfJAHN+8jSYtXwNfbem5u8WdYzqtuiFDMYNQ 9o0CF2zvqcBI3KOflpGcuXsiV0EKjJi8Flg/kBR5SE3nwS4yk72xAPo2K8d1ANNEyHMK LxkC4NCxA+SdiMvIzoeikWozk5nhMfWoWzlLqS71oiCYf4dw3n1zZaS9J5TxsV+QxpK8 amC5z5PcAy5PbKTQKuApudtagbTxyYysQWUoLzjqoD2LS77btFxrg0Dsj2U0qlk7u9mm qZUA== X-Gm-Message-State: AOJu0YyqPlWkZekDBTL/vBhazChjkmRrcasVJk9poVSf4KVLOuGUMjd+ ccFPfLe9WaypiYmvVTKIgO81wixHu+/HeayobW0YYCBiGSL1fyoG/ISh2Fa4 X-Google-Smtp-Source: AGHT+IF3mGN2BhcM7Mjv6dRXH7lrvSPsEGHlDzmfceQmklC2DrY+PM91hyiviN3/NqGgCxYUGaAG0w== X-Received: by 2002:a17:902:d50a:b0:1eb:4930:58d3 with SMTP id b10-20020a170902d50a00b001eb493058d3mr814463plg.4.1714168566572; Fri, 26 Apr 2024 14:56:06 -0700 (PDT) Received: from lrumancik.svl.corp.google.com ([2620:15c:2a3:200:2b3a:c37d:d273:a588]) by smtp.gmail.com with ESMTPSA id b18-20020a170903229200b001eb2e6b14e0sm855772plh.126.2024.04.26.14.56.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 26 Apr 2024 14:56:06 -0700 (PDT) From: Leah Rumancik To: linux-xfs@vger.kernel.org Cc: amir73il@gmail.com, chandan.babu@oracle.com, fred@cloudflare.com, mngyadam@amazon.com, Guo Xuenan , "Darrick J . Wong" , Leah Rumancik Subject: [PATCH 6.1 CANDIDATE 17/24] xfs: fix super block buf log item UAF during force shutdown Date: Fri, 26 Apr 2024 14:55:04 -0700 Message-ID: <20240426215512.2673806-18-leah.rumancik@gmail.com> X-Mailer: git-send-email 2.44.0.769.g3c40516874-goog In-Reply-To: <20240426215512.2673806-1-leah.rumancik@gmail.com> References: <20240426215512.2673806-1-leah.rumancik@gmail.com> Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Guo Xuenan [ Upstream commit 575689fc0ffa6c4bb4e72fd18e31a6525a6124e0 ] xfs log io error will trigger xlog shut down, and end_io worker call xlog_state_shutdown_callbacks to unpin and release the buf log item. The race condition is that when there are some thread doing transaction commit and happened not to be intercepted by xlog_is_shutdown, then, these log item will be insert into CIL, when unpin and release these buf log item, UAF will occur. BTW, add delay before `xlog_cil_commit` can increase recurrence probability. The following call graph actually encountered this bad situation. fsstress io end worker kworker/0:1H-216 xlog_ioend_work ->xlog_force_shutdown ->xlog_state_shutdown_callbacks ->xlog_cil_process_committed ->xlog_cil_committed ->xfs_trans_committed_bulk ->xfs_trans_apply_sb_deltas ->li_ops->iop_unpin(lip, 1); ->xfs_trans_getsb ->_xfs_trans_bjoin ->xfs_buf_item_init ->if (bip) { return 0;} //relog ->xlog_cil_commit ->xlog_cil_insert_items //insert into CIL ->xfs_buf_ioend_fail(bp); ->xfs_buf_ioend ->xfs_buf_item_done ->xfs_buf_item_relse ->xfs_buf_item_free when cil push worker gather percpu cil and insert super block buf log item into ctx->log_items then uaf occurs. ================================================================== BUG: KASAN: use-after-free in xlog_cil_push_work+0x1c8f/0x22f0 Write of size 8 at addr ffff88801800f3f0 by task kworker/u4:4/105 CPU: 0 PID: 105 Comm: kworker/u4:4 Tainted: G W 6.1.0-rc1-00001-g274115149b42 #136 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014 Workqueue: xfs-cil/sda xlog_cil_push_work Call Trace: dump_stack_lvl+0x4d/0x66 print_report+0x171/0x4a6 kasan_report+0xb3/0x130 xlog_cil_push_work+0x1c8f/0x22f0 process_one_work+0x6f9/0xf70 worker_thread+0x578/0xf30 kthread+0x28c/0x330 ret_from_fork+0x1f/0x30 Allocated by task 2145: kasan_save_stack+0x1e/0x40 kasan_set_track+0x21/0x30 __kasan_slab_alloc+0x54/0x60 kmem_cache_alloc+0x14a/0x510 xfs_buf_item_init+0x160/0x6d0 _xfs_trans_bjoin+0x7f/0x2e0 xfs_trans_getsb+0xb6/0x3f0 xfs_trans_apply_sb_deltas+0x1f/0x8c0 __xfs_trans_commit+0xa25/0xe10 xfs_symlink+0xe23/0x1660 xfs_vn_symlink+0x157/0x280 vfs_symlink+0x491/0x790 do_symlinkat+0x128/0x220 __x64_sys_symlink+0x7a/0x90 do_syscall_64+0x35/0x80 entry_SYSCALL_64_after_hwframe+0x63/0xcd Freed by task 216: kasan_save_stack+0x1e/0x40 kasan_set_track+0x21/0x30 kasan_save_free_info+0x2a/0x40 __kasan_slab_free+0x105/0x1a0 kmem_cache_free+0xb6/0x460 xfs_buf_ioend+0x1e9/0x11f0 xfs_buf_item_unpin+0x3d6/0x840 xfs_trans_committed_bulk+0x4c2/0x7c0 xlog_cil_committed+0xab6/0xfb0 xlog_cil_process_committed+0x117/0x1e0 xlog_state_shutdown_callbacks+0x208/0x440 xlog_force_shutdown+0x1b3/0x3a0 xlog_ioend_work+0xef/0x1d0 process_one_work+0x6f9/0xf70 worker_thread+0x578/0xf30 kthread+0x28c/0x330 ret_from_fork+0x1f/0x30 The buggy address belongs to the object at ffff88801800f388 which belongs to the cache xfs_buf_item of size 272 The buggy address is located 104 bytes inside of 272-byte region [ffff88801800f388, ffff88801800f498) The buggy address belongs to the physical page: page:ffffea0000600380 refcount:1 mapcount:0 mapping:0000000000000000 index:0xffff88801800f208 pfn:0x1800e head:ffffea0000600380 order:1 compound_mapcount:0 compound_pincount:0 flags: 0x1fffff80010200(slab|head|node=0|zone=1|lastcpupid=0x1fffff) raw: 001fffff80010200 ffffea0000699788 ffff88801319db50 ffff88800fb50640 raw: ffff88801800f208 000000000015000a 00000001ffffffff 0000000000000000 page dumped because: kasan: bad access detected Memory state around the buggy address: ffff88801800f280: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ffff88801800f300: fb fb fb fc fc fc fc fc fc fc fc fc fc fc fc fc >ffff88801800f380: fc fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb ^ ffff88801800f400: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb ffff88801800f480: fb fb fb fc fc fc fc fc fc fc fc fc fc fc fc fc ================================================================== Disabling lock debugging due to kernel taint Signed-off-by: Guo Xuenan Reviewed-by: Darrick J. Wong Signed-off-by: Darrick J. Wong Signed-off-by: Leah Rumancik --- fs/xfs/xfs_buf_item.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/xfs/xfs_buf_item.c b/fs/xfs/xfs_buf_item.c index 522d450a94b1..df7322ed73fa 100644 --- a/fs/xfs/xfs_buf_item.c +++ b/fs/xfs/xfs_buf_item.c @@ -1018,6 +1018,8 @@ xfs_buf_item_relse( trace_xfs_buf_item_relse(bp, _RET_IP_); ASSERT(!test_bit(XFS_LI_IN_AIL, &bip->bli_item.li_flags)); + if (atomic_read(&bip->bli_refcount)) + return; bp->b_log_item = NULL; xfs_buf_rele(bp); xfs_buf_item_free(bip);