From patchwork Fri Jan 24 22:48:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13949999 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50EE5C02181 for ; Fri, 24 Jan 2025 22:48:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B7BD62800AA; Fri, 24 Jan 2025 17:48:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B2BA22800A8; Fri, 24 Jan 2025 17:48:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A19A82800AA; Fri, 24 Jan 2025 17:48:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 849AD2800A8 for ; Fri, 24 Jan 2025 17:48:50 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id EF6331A19AD for ; Fri, 24 Jan 2025 22:48:49 +0000 (UTC) X-FDA: 83043836778.27.5FD8CA4 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf07.hostedemail.com (Postfix) with ESMTP id D301F40011 for ; Fri, 24 Jan 2025 22:48:46 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=M77uMF7m; spf=none (imf07.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737758928; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=sL8io6Ogy3tJI6VLP/NMc9XdkDtY/hwHMQ1XfBAdOus=; b=aMtwkBcleRO/5ZISFFU9NAN3zvepufy9PsEnrYzWqR1UTunml5hFhPamhvfbPnhbVtZbTQ rkwcSov3qBcJ2x8YJPi8jg+enLJoMFhwLjvvnRMxVjnbexLLmZ94wShA7+wrZp94Dxaxrq TitEfPxgvzSt3QOY6S6Q07at/7pEoB4= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=M77uMF7m; spf=none (imf07.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737758928; a=rsa-sha256; cv=none; b=iQglYoTf/QOI2CcFhSbDCLuBdlKQKgvJXvr8w1qt5XPuF+T5y38GP+ikP5th6YOmGvinyr 1LMvMnJ3G10AvMxdX1FDD39GThXJIHkd+RwrVs1IAG1hE1sgmxeXj7oKSsPfW7pzdGNUz1 amLPWnz9J3yBsdrBo9GfgNyuwrPNnoU= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:In-Reply-To:References; bh=sL8io6Ogy3tJI6VLP/NMc9XdkDtY/hwHMQ1XfBAdOus=; b=M77uMF7m5thKuIcC1WXWpGX43A f6dgfrTHh316HxHRzFwpf8AgeQp9J1cUmyrIWXylh6ppQGoFYbqCdjBGAtUGAjwDbqNAbXgt067PD VnNm28te8WQgmSdGqWX8CEUpMq3aPVvSLFu7qyZAkIv6v+hnzhBmG4Z7QRBw1k50QR5zV2hu+LGXn YqDPRQooK3UdCMDm4tmdwuaEcRbutFjUDZSdhRlfR7v18wA3tDqSt4dvr0vvOe33vUBYBarkHZuIo 7GTh/z93RgAynMlfPFv9cPl556T6ejg/7hVPuxyTTgkX1pQOJos5l3lwYr1Tian1VtxM9stkd/yu4 +LhAkEAQ==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tbSTk-00000001M3d-1ych; Fri, 24 Jan 2025 22:48:44 +0000 From: "Matthew Wilcox (Oracle)" To: Jens Axboe Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-block@vger.kernel.org, linux-fsdevel@vger.kernel.org, Andres Freund Subject: [PATCH] block: Skip the folio lock if the folio is already dirty Date: Fri, 24 Jan 2025 22:48:29 +0000 Message-ID: <20250124224832.322771-1-willy@infradead.org> X-Mailer: git-send-email 2.48.1 MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: D301F40011 X-Stat-Signature: pxwumn96fagmtkx4wm8wietag6ac3bz4 X-Rspam-User: X-HE-Tag: 1737758926-103127 X-HE-Meta: U2FsdGVkX18LYZpRimcRG1RlyBz6ls5bhW2/rix/Z9djNgDfJ3OK6uSDMkOVGubq3pLouuXgOpiw6aRrV2y53po7tSK7US6Mvm2uZx1PqIy6qh20XhzAVEQn2geQ49xyhq9KTXrlCKcOhfE3sbs/JGZrnEqw0qpG7wpCLLzGMMzvlK2CtQGllfh2Vje9AHyqexq89xFLArsp3v3x+MUsllOA43COm3yOWXHxARL+nfH/RF6JZ1iQqVgZ8fuzFZ265YeBLENITBblO2g5H9QWy2cxGWlFKmWb8GEA4dLPjomGEB0GIqysMxO6Z3qTZiKFl1z4ziWol7l/bVZGmZgZ54enJiNmk6DrTu50PgtfOOi0ctF8fvea1wT/umpgPVIg2GU0Efk1rVAYIlFxRmDAaSiEXXeT8Z+rDo9cBENfmOhpYkLG9YyQ8agwXEou7FOdnytJT4xJhDzqKgiuTgQJibQpnecdKIOMZYCboJK6PnwHiD3YRQJlr40+1OrLrvtTmWuHAK43GzyZmaDLznIo6LHFHlLCSsMpOvvVf8vGj/tbGDH1SdeBQSo4jjQqCUt26qvVB8IShHBY+nj+OYzzj0MyrpL09xiU5xMQQHpSyST6s9FOMki11cLdj4oqZzgKuRBgodmpwBdOOHjlfiRPFhjQl/ZD4hBB//dR6601RO5e12WK1wCNECPx937+jdlyG2ecyUW8tFLZDiwznHUPE/i2aqMvqrp8N2ZJIsHP91r+fSA2s+hQ0lk7NGYbQguGuK6FYPCNCCUTrrs4JpH9Acofkudc4swUGDpXSiTYdOZqHfh/1rnpwB1W7EhnA9vxwTJv71GzFXGYHtiEufFzQz2UAjvRr5UW78/wt5xYG2k6J797ca8qV7wUoAXsbho54fKontWbyNxICROfL7w2RpWzNTCfr5FX1EM2lMtkRc0RLikML0/n40wSGSorHtqD5gKnx3hUSeLXZnlgL20 y54Vp/38 TeZ5wZSoNNtCA/idbz2jquNejyjMqzHIdpeR28QzW7Che6zrIbJs0CJLCRPatNX4uk3L3mikJRm1zfT0lf0kAmpygwjL22bw0bSRpv5lKfjS1uTT8QkbXF0Q/LnUnr1Ax9+dHrEQebbEejNKWuW2G8xuoB2/nSiKJre61zGFEztisZ/yppror4PsANVmNAQzwxdT/UTDlJlqCcBUZKq5LTjxC/tgcelhuSl1TG8SVdfZTfSVJemsaNSL51aoI+7hIcC6orvJr1T8V2W8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Postgres sees significant contention on the hashed folio waitqueue lock when performing direct I/O to 1GB hugetlb pages. This is because we mark the destination pages as dirty, and the locks end up 512x more contended with 1GB pages than with 2MB pages. We can skip the locking if the folio is already marked as dirty. The writeback path clears the dirty flag before commencing writeback, if we see the dirty flag set, the data written to the folio will be written back. In one test, throughput increased from 18GB/s to 20GB/s and moved the bottleneck elsewhere. Reported-by: Andres Freund Signed-off-by: Matthew Wilcox (Oracle) --- block/bio.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/block/bio.c b/block/bio.c index f0c416e5931d..e8d18a0fecb5 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1404,6 +1404,8 @@ void bio_set_pages_dirty(struct bio *bio) struct folio_iter fi; bio_for_each_folio_all(fi, bio) { + if (folio_test_dirty(folio)) + continue; folio_lock(fi.folio); folio_mark_dirty(fi.folio); folio_unlock(fi.folio);