From patchwork Fri Nov 8 17:32:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13868591 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 263DAD64070 for ; Fri, 8 Nov 2024 17:36:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A92906B009D; Fri, 8 Nov 2024 12:36:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A1ABF6B00AD; Fri, 8 Nov 2024 12:36:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 894206B00B6; Fri, 8 Nov 2024 12:36:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 67D1D6B009D for ; Fri, 8 Nov 2024 12:36:54 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1EAE51C3BEE for ; Fri, 8 Nov 2024 17:36:54 +0000 (UTC) X-FDA: 82763631972.10.C9333FA Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf11.hostedemail.com (Postfix) with ESMTP id 93CAE4000D for ; Fri, 8 Nov 2024 17:36:06 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=EnlVuOKH; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731087286; a=rsa-sha256; cv=none; b=H/khpfwXWy/k6jJ9hCknRIgRH37TQQJLq0Z8SHDSOG8mP8LgdoXswKPnpIZykvOil1SOL3 TnjaYisg9wHqvvvbTVr0cZOlTJYCenQuVclb5wDYAqItD35pVNm5vD751gg6xGnhJ8Tp8B 5LFbbifqpq5CTlhBHCz59wAo7XMpnio= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=EnlVuOKH; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731087286; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Jlfv2ApsSBgn//KdBsSlx8BkotK5+tCpg2808yZLllw=; b=Yii/1wDlhHuA0IJP8QyF5e1EoUry6XrasSiF4wwE+W02F7mcspvvULfML/7Mr2SiyWBfA9 3cGZrFnUgNV58M9ISQRXs7tGe9PCDzHfZ65KJ+ePVhtlMt23xf0DlGEqt/sVU7xB6Kc847 4si3YLj+b93ti3fafhgazxzwaRSTKgQ= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1731087411; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Jlfv2ApsSBgn//KdBsSlx8BkotK5+tCpg2808yZLllw=; b=EnlVuOKHtxRm5r/XUbwfEDUBfGg31uzPO+BhRxchcqfbRr6OVJnmSkMZd0jMy66pgqC8zh 2AnnDYGSll+xZ6ojZPTVsb7ehvZRQrSsSm2bzTFHRwyPrGf2ZS3fc+iey5vHpGiUcoTRPF 7N0oO5JsWY4du3oVMEUlc0lotqfO3QA= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-226-sjYpWV5SMt6Lmz3uofTwYw-1; Fri, 08 Nov 2024 12:36:48 -0500 X-MC-Unique: sjYpWV5SMt6Lmz3uofTwYw-1 X-Mimecast-MFC-AGG-ID: sjYpWV5SMt6Lmz3uofTwYw Received: from mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 935F41954128; Fri, 8 Nov 2024 17:36:43 +0000 (UTC) Received: from warthog.procyon.org.uk.com (unknown [10.42.28.231]) by mx-prod-int-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9CB78195E480; Fri, 8 Nov 2024 17:36:37 +0000 (UTC) From: David Howells To: Christian Brauner , Steve French , Matthew Wilcox Cc: David Howells , Jeff Layton , Gao Xiang , Dominique Martinet , Marc Dionne , Paulo Alcantara , Shyam Prasad N , Tom Talpey , Eric Van Hensbergen , Ilya Dryomov , netfs@lists.linux.dev, linux-afs@lists.infradead.org, linux-cifs@vger.kernel.org, linux-nfs@vger.kernel.org, ceph-devel@vger.kernel.org, v9fs@lists.linux.dev, linux-erofs@lists.ozlabs.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, syzbot+af5c06208fa71bf31b16@syzkaller.appspotmail.com, Chang Yu Subject: [PATCH v4 33/33] netfs: Report on NULL folioq in netfs_writeback_unlock_folios() Date: Fri, 8 Nov 2024 17:32:34 +0000 Message-ID: <20241108173236.1382366-34-dhowells@redhat.com> In-Reply-To: <20241108173236.1382366-1-dhowells@redhat.com> References: <20241108173236.1382366-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.12 X-Rspamd-Queue-Id: 93CAE4000D X-Stat-Signature: 8aapdqjujj89m8gus79fopcbjhkk61te X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1731087366-127412 X-HE-Meta: U2FsdGVkX19CyDxesOsI1StvWlAdIAiqfNn6za+gcGs0xBYFWv2unrSkKisyJgE6aOW8D9hNdw9HkK7IbiTyy6LS4MlMp81BOl3DWwdXipvHzMZTK7B0E4kCLlBW9mgDE81QPNBXJsJpnoXrwQ7C0iN3fJcP5FrZr3AfPh3JTFeo1feUQrTXpt+KlIarOIwVpGGjsyAJU9XgNT58DCvPZzcwa5OK87JP29b6Vp/4rP8bXT6v3O0fchA+KKilz38NxrWnUsGYE2sr88F+J9MS18+VNMFquP1iRkPs8Xp+ToZ0HHumPmorHT+6VF0Yb/K0rAFixphxSbMb0AaYAxtbmLXjroRH4udPbznEUCTrxjFTSzcScYA0+0bNoU+wwO6PwCiAYUuM577S1sitf5tEqUUuH4jOsnOVv1ZC8+kkQ6mS5VtJwNww89cWEsBd7uwRGl2QJ0544Z7nSGuOYBIHIppv7ubTynaOlOL3ZxIxaNxN2pioWUiislzrX813+F+1iUhlqRSn+4Q/3BKzqg1voFQ46zvFwHrgy8c/PXgHsCcVedLdnHX/6Pyc98nWsDSxwgLQx1nL6CvEgyY+LvrlY+65WswhrkpZQkth8nDXq2yYHZDCBSy1HNNyMB8npvmDtS08c5qWNBw/0Xi15cn7pwE6xf5TVvUmJKR9EN59bfqHH4eDN2JttM3KvpkX7qIamT4KWcFkX41cPJUOFRFjBKXpmV9OJ5G9qJPzzxUsKXaDXTDP4R8eKAW1kKBNeFdRm5GkS0SQ0co5gHv80WOx/m2DVls2sAFsXTaxSRjQpcBjYSX43nOyj+a+9wkmEfEdiZaCsX60oQrqcP3K4sQ5Ojo8JyJklsEQicSASl0oceltSf7Sgcd3+kX/kDDyEfbn6slIESr3mOQQbZb8llQy74qQLxj+Ihq+Ehf0B/UTQWa7yPk8rL+d500JoGt6U5RiztgPCMbcb6xF+6dtOaf jzNh474I TmPOBmk2voubP9yNgW4Da/1rGGbXWTDiJkDnjGEJz0TCeTb3+gmcv9nemquYGGn5gM06dlJNQHUy1xkwu+eTfmUU/Fd5QUZYThSa7zGfMDoOrRPMG6PknnKffZI6PGfNTcP3ttUS3fDvR0bz+VT7QGavPJaAZC7G5SJCaaEkjbEY5ve8K+PxKxSyYmxRBdd/xs7+FvexgEayva8UAwTZvKohYM7qUccoxbraXcaTampemfOVXUg2XDrE7LdtDeOuE8nDMif4bNQHkgvd58lHAcjwR4fGOGh2TJ2BVNBY+NgK+TdeAdEpF++Hu552iYIXa7Vjh3bhizXCQN33wD3iDkB0FPf2pkZCSSKcofECUeAgL8xtZfmLntGDoH0qTiPizWv1RBetiNmZ9qWeRzfppZILtJxkrB+N3acYHTAqn/tJaYllT7epXLRKNdCd8RqsvEy8ayWH3WLvtXn+2ZVHwKUGnvrM5fy1AWIVpesByCx0dp3QNKA9ogrtKLlvZDrhgczFqwBFb+HYLHoQ4EUjlR3v/RYqcotjrh6ZWcpq/u3ajr1BPMPlCdTuMn43CMh5nCwiJA225KbBOlN/wOdtbCDuz7i0lk0ZupkFN+k6aPoXnkg7etuDSiXNVbV4xK/EUawsqKV+pWdqDzZdfGtH4lWwCkd/mIS13lnaiLF8nnQlbad4P6ytGBqWY/xRfYiaf692fKuJPkR0INfE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: It seems that it's possible to get to netfs_writeback_unlock_folios() with an empty rolling buffer during buffered writes. This should not be possible as the rolling buffer is initialised as the write request is set up and thereafter maintains at least one folio_queue struct therein until it gets destroyed. This allows lockless addition and removal of folio_queue structs in the buffer because, unlike with a ring buffer, the producer and consumer each only need to look at and alter one pointer into the buffer. Now, the rolling buffer is only used for buffered I/O operations as netfs_collect_write_results() should only call netfs_writeback_unlock_folios() if the request is of origin type NETFS_WRITEBACK, NETFS_WRITETHROUGH or NETFS_PGPRIV2_COPY_TO_CACHE. So it would seem that one of the following occurred: (1) I/O started before the request was fully initialised, (2) the origin got switched mid-flow or (3) the request has already been freed and this is a UAF error. I think the last is the most likely. Make netfs_writeback_unlock_folios() report information about the request and subrequests if folioq is seen to be NULL to try and help debug this, throw a warning and return. Note that this does not try to fix the problem. Reported-by: syzbot+af5c06208fa71bf31b16@syzkaller.appspotmail.com Link: https://syzkaller.appspot.com/bug?extid=af5c06208fa71bf31b16 Signed-off-by: David Howells cc: Chang Yu Link: https://lore.kernel.org/r/ZxshMEW4U7MTgQYa@gmail.com/ cc: Jeff Layton cc: netfs@lists.linux.dev cc: linux-fsdevel@vger.kernel.org --- fs/netfs/write_collect.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/fs/netfs/write_collect.c b/fs/netfs/write_collect.c index 3d8b87c8e6a6..4a1499167770 100644 --- a/fs/netfs/write_collect.c +++ b/fs/netfs/write_collect.c @@ -21,6 +21,34 @@ #define NEED_RETRY 0x10 /* A front op requests retrying */ #define SAW_FAILURE 0x20 /* One stream or hit a permanent failure */ +static void netfs_dump_request(const struct netfs_io_request *rreq) +{ + pr_err("Request R=%08x r=%d fl=%lx or=%x e=%ld\n", + rreq->debug_id, refcount_read(&rreq->ref), rreq->flags, + rreq->origin, rreq->error); + pr_err(" st=%llx tsl=%zx/%llx/%llx\n", + rreq->start, rreq->transferred, rreq->submitted, rreq->len); + pr_err(" cci=%llx/%llx/%llx\n", + rreq->cleaned_to, rreq->collected_to, atomic64_read(&rreq->issued_to)); + pr_err(" iw=%pSR\n", rreq->netfs_ops->issue_write); + for (int i = 0; i < NR_IO_STREAMS; i++) { + const struct netfs_io_subrequest *sreq; + const struct netfs_io_stream *s = &rreq->io_streams[i]; + + pr_err(" str[%x] s=%x e=%d acnf=%u,%u,%u,%u\n", + s->stream_nr, s->source, s->error, + s->avail, s->active, s->need_retry, s->failed); + pr_err(" str[%x] ct=%llx t=%zx\n", + s->stream_nr, s->collected_to, s->transferred); + list_for_each_entry(sreq, &s->subrequests, rreq_link) { + pr_err(" sreq[%x:%x] sc=%u s=%llx t=%zx/%zx r=%d f=%lx\n", + sreq->stream_nr, sreq->debug_index, sreq->source, + sreq->start, sreq->transferred, sreq->len, + refcount_read(&sreq->ref), sreq->flags); + } + } +} + /* * Successful completion of write of a folio to the server and/or cache. Note * that we are not allowed to lock the folio here on pain of deadlocking with @@ -87,6 +115,12 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq, unsigned long long collected_to = wreq->collected_to; unsigned int slot = wreq->buffer.first_tail_slot; + if (WARN_ON_ONCE(!folioq)) { + pr_err("[!] Writeback unlock found empty rolling buffer!\n"); + netfs_dump_request(wreq); + return; + } + if (wreq->origin == NETFS_PGPRIV2_COPY_TO_CACHE) { if (netfs_pgpriv2_unlock_copied_folios(wreq)) *notes |= MADE_PROGRESS;