From patchwork Wed Oct 4 16:53:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13409076 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7BEDE7C4E6 for ; Wed, 4 Oct 2023 16:53:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7B4B66B028A; Wed, 4 Oct 2023 12:53:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 765E76B028C; Wed, 4 Oct 2023 12:53:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 604E46B028D; Wed, 4 Oct 2023 12:53:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 46A786B028A for ; Wed, 4 Oct 2023 12:53:35 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 1D60E1A03A4 for ; Wed, 4 Oct 2023 16:53:35 +0000 (UTC) X-FDA: 81308375190.16.CA6A26B Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf25.hostedemail.com (Postfix) with ESMTP id 74AB4A001A for ; Wed, 4 Oct 2023 16:53:33 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=XKrwcSOW; dmarc=none; spf=none (imf25.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696438413; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KV8LjVnp8SrGIYzPBzTmbRSPvfzlFJYoh9EV/UdX1VU=; b=AEAcQl80KmccFdw/Ox264+cRH/j7I+j1bIy3uooZBu2hT9dSoXQOm2zXf+GY8LuoWIVyH4 muYDjdoLcC483aNYxa2suxVnQjS+pjjvP3yW7MPn68J7NMVg6P4+0prHz1kKxiibyngvxc fLuygOil0xIfJXasJLFUxvS1j0EaCG4= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=XKrwcSOW; dmarc=none; spf=none (imf25.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696438413; a=rsa-sha256; cv=none; b=NDqXQgq/T+aXlbYxvW4ngOCxRFRpeU7vGX/p11I5FUGs3iuVRIHcD1EXeAeUDT+Bh/ot8o /ox22C3wPPP5baPAiMTZQ5MK4KS+7kHXwg4ekVG0QsDezsQflHFKPGRvfmYRpTL1/ztQJ6 4oXQLTRrPfAWvSHGGcof/OclxsfwbpM= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=KV8LjVnp8SrGIYzPBzTmbRSPvfzlFJYoh9EV/UdX1VU=; b=XKrwcSOWaQs6f+lvNnjqWsQNeA tgWGSVcwUcSUK+ppxTN6wCxuM6QU4zpdWpMiV6KVrIp0r98Vydm6CI9x3+M231XCLbvBHeUsTq1Ls +eZkvGGQwfurJSHA3nMDjyfvsg3VWB1w7Wx3V3DziFeKyn/pTJoHxUMnw/Y0g1ZbGCod0NlutrX1u figIDtFZopRavwvsxs9+1eC5eeKlB+YwkNGYdG+6yHXeWhuaeNGxlKiTUBIQrV/JCqNxwxPBkUPJ4 ygOewt9UVPVSMnaPp9qvSI+NvRcESnTW6dPLxrxNb6Il9rZ36q3vUjsCZiCcnR1NwTo+eprsm+LCY ab+q/3Xg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qo57f-004SEt-6o; Wed, 04 Oct 2023 16:53:19 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, npiggin@gmail.com Subject: [PATCH v2 01/17] iomap: Hold state_lock over call to ifs_set_range_uptodate() Date: Wed, 4 Oct 2023 17:53:01 +0100 Message-Id: <20231004165317.1061855-2-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231004165317.1061855-1-willy@infradead.org> References: <20231004165317.1061855-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 74AB4A001A X-Stat-Signature: 7eymgbaeqkztj5hoyjz1krgtcqudi6gk X-HE-Tag: 1696438413-805769 X-HE-Meta: U2FsdGVkX180lA1diPHrkyzibudTVi4wj7jbtZUhTNIqGqxdDVxd7TC+GtVA/g2CLgNShGPFH29aWQQ09Pi9Y4TsvtnKOL0CU4C3AqAOsSbfAXLYOtSp9gdlFV7ePOg4S83FriDM9RcsMgiZXJNErOxt2HhDEZP3+rJzxHltwtcZV6p4Uy6W5bXKSJE4OvMV1/ZbyXkhqAUKZYkAKwGtv3MlWAq6JY+4L7xuZjweehp7GqGi+4U4AfAhmiiIsUX98YXGrCEMuyNF+WpSqSAPuOwqcgFTEotDwKSVqo0sVOAVsv2rzWg3zd6RaihzMeosPII58mQ2MGZeKXElfd70KEs5D/CjZ2QmOsaQ8mcmGVOJOzSSmvyriW/ZeIDhl7E16uoaitaTFyDxBHJdKPIUu1X54RS1LLERK0LQncwezh5S7p6uH5GNFXjAGKPnqY8foZFNLAbAfNmoJeB53FC03fplz057Edp6On06RnW2ExP0K0DxWaSZqijvL8eCD+LvTQiDJH1Ewhkp9Id0E8tgNVKhsv/w3ipk7cdraLQuUhTqfxGMFQTt6zCxJKhw5fmV19m+8415UM9yPBFYbSwbnMaVTaDb+yZ77JHkjvVPyB+zn40k+ClP+0XPHOraLKMVZwvijbgf5rOMpkbjB4rvHJeDCRdudRER9/oqljCY1kF8dGxtmrC4pGaQlRRWyDAe/JDVe4Dmq0I72JLR4wulP2f/fzXoFMMGwjS2EfZui7ABkqjmIKNrCC83GVT8Jsba6Ag0ATyJFiQ0PJ9YsiVITrOknJWMWy7tyD4OwDK2dgI7b2yRmVYGQYnbMoELMrS7VtUTm8ETGniIsW1qI17BE4EMuSGGB32AeISQPKv2OHawxJ9hLFwzwzyhD1WJ8ZhrmDLaVfJHk0aBCSMhK8vtJUXksc28QGpZDFgK/ECAC/skzneeygYUFBOCfmVYErMSsW8D1O+5//5QYYKXZoh xgrVYLlD lJPl6e4ajxkDgt/R7KNmV2xzTs6mTAEVteJiygIxXzZPza1zMyBs8UXhlnBUiyJD8kZtJcnEOmonk2lYIOy4nhRSIK9RwcznLwUAxyGwpXedEV/8HGVMEOMC9ilMwxG5GWB2EGvcsS4WhewCyFoxXP/kdOD+zwYGwD2ZMxXbgclBjS9ORwh3IVW6baDqkoB5MPnmRO5jBEcInzb0Ul+mbMVPjfA7cH6Bzxk7L X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is really preparation for the next patch, but it lets us call folio_mark_uptodate() in just one place instead of two. Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/buffered-io.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 5db54ca29a35..6e780ca64ce3 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -57,30 +57,32 @@ static inline bool ifs_block_is_uptodate(struct iomap_folio_state *ifs, return test_bit(block, ifs->state); } -static void ifs_set_range_uptodate(struct folio *folio, +static bool ifs_set_range_uptodate(struct folio *folio, struct iomap_folio_state *ifs, size_t off, size_t len) { struct inode *inode = folio->mapping->host; unsigned int first_blk = off >> inode->i_blkbits; unsigned int last_blk = (off + len - 1) >> inode->i_blkbits; unsigned int nr_blks = last_blk - first_blk + 1; - unsigned long flags; - spin_lock_irqsave(&ifs->state_lock, flags); bitmap_set(ifs->state, first_blk, nr_blks); - if (ifs_is_fully_uptodate(folio, ifs)) - folio_mark_uptodate(folio); - spin_unlock_irqrestore(&ifs->state_lock, flags); + return ifs_is_fully_uptodate(folio, ifs); } static void iomap_set_range_uptodate(struct folio *folio, size_t off, size_t len) { struct iomap_folio_state *ifs = folio->private; + unsigned long flags; + bool uptodate = true; - if (ifs) - ifs_set_range_uptodate(folio, ifs, off, len); - else + if (ifs) { + spin_lock_irqsave(&ifs->state_lock, flags); + uptodate = ifs_set_range_uptodate(folio, ifs, off, len); + spin_unlock_irqrestore(&ifs->state_lock, flags); + } + + if (uptodate) folio_mark_uptodate(folio); } From patchwork Wed Oct 4 16:53:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13409077 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B682EE7C4E2 for ; Wed, 4 Oct 2023 16:53:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3C08E6B028C; Wed, 4 Oct 2023 12:53:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2D11E6B028E; Wed, 4 Oct 2023 12:53:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 197B56B028F; Wed, 4 Oct 2023 12:53:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 0B5596B028C for ; Wed, 4 Oct 2023 12:53:38 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id D96DA120395 for ; Wed, 4 Oct 2023 16:53:37 +0000 (UTC) X-FDA: 81308375274.20.353B13B Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf09.hostedemail.com (Postfix) with ESMTP id 4BBF6140004 for ; Wed, 4 Oct 2023 16:53:36 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=P9PqackI; spf=none (imf09.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696438416; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=D/8R0cQXGxymytv8qeJZP60gdDV9xdbcF13gSUAJ8s0=; b=nqQNHMVGTEIz6yNz13+XAtFMUMGO5+KRPPlO5tp+qflmswiJUfSoFPu8cMVtt01fjBZZOE V3GHJggmo/3JRLGzCmPLWwaiAif9MkgZwvLhZWPzvh6M2ZzsCZtghUplQ65JIPdJTfD47D zwsfYHr7GMiFEQfbW4z0jFw2P2G1fQw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696438416; a=rsa-sha256; cv=none; b=h5WxJNXgWOqAWhQcnH2C7iht3k887TRmq7ckksSUplABt3KlzZSiWQFRNC18zpIA1cTkBS tXsgScu2GRxxC6/By/2NdtrW1tH75uZCt2sN/ihQdf4LgvM8xWDmLzJV5SWUutqu9OITFS BOyn0xFNxWxbIk6OzT71+RD+zOUk6g4= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=P9PqackI; spf=none (imf09.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=D/8R0cQXGxymytv8qeJZP60gdDV9xdbcF13gSUAJ8s0=; b=P9PqackIT0b7fZvk2PUiwq2eEL ogL4pXi4p90LlQsy6dqVmscrou91ZCSdQPbrsXf44oAxDyO/BxwBm5Z2vIwKUlQlUDpjFbnFPG9fM BrJ63AckCUHqeJPCN9sjwAn+cG1XEsPXgyCNIWA2qvjPyp6VYgJA83cYUMVSJesCNkv4zWzv7yoTQ gP2Z1xQ6jEcNIyFm3dsMavIcKirjm580itF3FOEGmJRHUDqy4pF9sGf2LiGyEorzm8grd31hr/VVK KTTbbG5C8guCMxuwevbECmyH8mc5IYYAvCK+Lb8Nkenf0880jROkyzHaXctQk/+6ULTCzBapwqBs3 erU9zo2g==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qo57f-004SEv-92; Wed, 04 Oct 2023 16:53:19 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, npiggin@gmail.com Subject: [PATCH v2 02/17] iomap: Protect read_bytes_pending with the state_lock Date: Wed, 4 Oct 2023 17:53:02 +0100 Message-Id: <20231004165317.1061855-3-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231004165317.1061855-1-willy@infradead.org> References: <20231004165317.1061855-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4BBF6140004 X-Rspam-User: X-Stat-Signature: adqjfwomuodr97oq8e1pgmke61tz4fgo X-Rspamd-Server: rspam03 X-HE-Tag: 1696438416-486160 X-HE-Meta: U2FsdGVkX1/09d84drkvUY8SGUAnNnl7KmEBpm0da7d2IEUJq9toV3pwE3LiDGP4wLrv27xpXrvc6QIzOw+EMV8ZgnqHlKqxmJLSjgvtIEjLkphJ3qayAGJ2CeZv0Q+0RnDc9TXHDUrEu9boxUDaBmAk+TSXcRryhATu4MltC4kB1XKsudYl102unPMbuojvPGWqdTZNL1K1Hht079B5XjiVVl0KK70y3yo0mlKz2ECENtyVN1QjvGvg+eP41+jO5oUyFOZzde5nlrc7zyWPsu8kcqjQZRfJ7DmnAIK6IbJZ+C1s393/dUnHEQ78qhR/7mEPPWm9ouB8a6jfoRDdRkfleQARjSNfLvW+kJqYhaYNTVRQuxx8u4jA4/nuJZDkTutshbuLyBnl3KNYZTFyePMFbXaekBRDGlI78+wMALA/9EDJw8EO2TRTJ6qoh9g/cG6zG4hkTYHDadMIEa1WoKHY3M7sUa0VI+xSVDbzH4HGfWVrsO7zBBSdtcuhI1rav+LVgDoKVAJHq5UvcI3CaSb2E6/Qi5DGtMVEhSN52zuwhd7/pluBwl+rrqy2C/JzPcQVEhFQ7il6bSHeSLZv7dS2m/CIYXcuqGvOUYjcP/YcA519FjQT9OIbAaOO93yPInjoohWjJvdk8ZQdIx2h+52+GwZkX36hniZfGd583jmwlhD8/cetDLMV538UEIgIi0Y2YEPVN5YMAkGRjQ8pUj9qJvzXKym5GcSgp7Kj5OK6AC+wXF8viMM9m13gN9O9dakqCcGzwgL96YIUuhAcZ8KokezUg8CGqHeljfkgv47X1HT3jzPzBUtZw9tClGsI7V6bboX6NqpxxSNyqn6lzgN2EmPqO7YUJBC4bZEpWvUew3gyLt0Fq7tvuH8GMrSE6QuIY6tv/rdLty9fppGmRn+TXt2jdeY0fVLOmeDozLvtWg7SJBsYtV2tp8F+D569RRawFt5nv+kjjPX4tKC 4y8MHaYJ 9uEHtCBZVvP+g/fHPkFS4oSDEGaEHzrMxU2pq/wven0BFwR0spyEFsCOPUberzSe4+9H1ZJOkxeNHczOuao0S/90hEqGF7Lv57d7TU1Jee/RW0JoukaPQAtLfX8ueEcWAftxqzFt9d6/vI0U3q6yUwEcHgEA3q9nvv9VVV3Dopu/CrQwLUjZk8LP3S63YYGjEhZKAEZDm87TiZs9zE1XIrZY7IZnM6cunQ+HR X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Perform one atomic operation (acquiring the spinlock) instead of two (spinlock & atomic_sub) per read completion. Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/buffered-io.c | 37 +++++++++++++++++++++++++------------ 1 file changed, 25 insertions(+), 12 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 6e780ca64ce3..4a996c5327ef 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -29,9 +29,9 @@ typedef int (*iomap_punch_t)(struct inode *inode, loff_t offset, loff_t length); * and I/O completions. */ struct iomap_folio_state { - atomic_t read_bytes_pending; - atomic_t write_bytes_pending; spinlock_t state_lock; + unsigned int read_bytes_pending; + atomic_t write_bytes_pending; /* * Each block has two bits in this bitmap: @@ -183,7 +183,7 @@ static void ifs_free(struct folio *folio) if (!ifs) return; - WARN_ON_ONCE(atomic_read(&ifs->read_bytes_pending)); + WARN_ON_ONCE(ifs->read_bytes_pending != 0); WARN_ON_ONCE(atomic_read(&ifs->write_bytes_pending)); WARN_ON_ONCE(ifs_is_fully_uptodate(folio, ifs) != folio_test_uptodate(folio)); @@ -250,19 +250,29 @@ static void iomap_adjust_read_range(struct inode *inode, struct folio *folio, *lenp = plen; } -static void iomap_finish_folio_read(struct folio *folio, size_t offset, +static void iomap_finish_folio_read(struct folio *folio, size_t off, size_t len, int error) { struct iomap_folio_state *ifs = folio->private; + bool uptodate = !error; + bool finished = true; - if (unlikely(error)) { - folio_clear_uptodate(folio); - folio_set_error(folio); - } else { - iomap_set_range_uptodate(folio, offset, len); + if (ifs) { + unsigned long flags; + + spin_lock_irqsave(&ifs->state_lock, flags); + if (!error) + uptodate = ifs_set_range_uptodate(folio, ifs, off, len); + ifs->read_bytes_pending -= len; + finished = !ifs->read_bytes_pending; + spin_unlock_irqrestore(&ifs->state_lock, flags); } - if (!ifs || atomic_sub_and_test(len, &ifs->read_bytes_pending)) + if (error) + folio_set_error(folio); + if (uptodate) + folio_mark_uptodate(folio); + if (finished) folio_unlock(folio); } @@ -360,8 +370,11 @@ static loff_t iomap_readpage_iter(const struct iomap_iter *iter, } ctx->cur_folio_in_bio = true; - if (ifs) - atomic_add(plen, &ifs->read_bytes_pending); + if (ifs) { + spin_lock_irq(&ifs->state_lock); + ifs->read_bytes_pending += plen; + spin_unlock_irq(&ifs->state_lock); + } sector = iomap_sector(iomap, pos); if (!ctx->bio || From patchwork Wed Oct 4 16:53:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13409063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DCB7EE7C4E5 for ; Wed, 4 Oct 2023 16:53:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6A1EC6B0269; Wed, 4 Oct 2023 12:53:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6798D6B0266; Wed, 4 Oct 2023 12:53:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4CB0E6B026C; Wed, 4 Oct 2023 12:53:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 3B3466B0266 for ; Wed, 4 Oct 2023 12:53:24 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 13B59140371 for ; Wed, 4 Oct 2023 16:53:24 +0000 (UTC) X-FDA: 81308374728.24.54FA9D5 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf24.hostedemail.com (Postfix) with ESMTP id C17C1180014 for ; Wed, 4 Oct 2023 16:53:21 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=YtNGtSMz; spf=none (imf24.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696438402; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+0PawpxhJKqaj77T374P+o4JeUu6sT89JK6ue2yK9T0=; b=TcBMP9LDlocRO/oAF69a0vMHoiTDTSLG+Tln79vtYjRv4ORFVI8tc2xiGyHrlrcEp75X2B bN6TpnN5Qx9IIVBl23qiljdJoM7A/jcLGIB2NwA7oPnOMXKNQJjdgopD37oFDFjJpzzD3Z 01NXq1S+71dyugdH9qRdTr3uy6tbFwc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696438402; a=rsa-sha256; cv=none; b=LPvnyGy7uSx1l1FQPCtxmaPlRw7ZGkspzkklDW6dykpx6PhHNtvL1Pl/hOR+IQICYF6S3D KqLUVpIVTwq1x+q3xqiJ6JnadB8TsKg2lQfMndKyP1ODObaPY7ay4kZR2fO48MvHXJfhQE M5gkFfd1O0/uMIGAz0H/cQsN61/vo10= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=YtNGtSMz; spf=none (imf24.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=+0PawpxhJKqaj77T374P+o4JeUu6sT89JK6ue2yK9T0=; b=YtNGtSMzne6N+0JeD40RVE95Pv 2LaBF0plT+dXJTT9umXL9Q5qH94HYx6QSxE5QGemVyAB+9qEYNOT01LEgHewvUIBwN+9CI5PB7rq6 V5RF+KubKhJsLsz/OCz1jCq5k9YDLQX0jJeOWqvB/xZqbrmDwN65iDoS2M/N04QW9IbCzVHaA79v3 TY7Q+Bi0Jfx2ve9d3/ryDr2L5Jde8QbzA20cm4EyfXY0A6IGsnqwIYyU1oPZHj5S9eibEbOV5BOCw L5FZQALvQ8eYHOf3cdx1O/M0HmQ7xHKgkP5uWOZlwFPTYoBIjWnmMMjt1EGy3f1zmuTJJHFMFzLvj oqp7VTFg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qo57f-004SEx-BB; Wed, 04 Oct 2023 16:53:19 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, npiggin@gmail.com Subject: [PATCH v2 03/17] mm: Add folio_end_read() Date: Wed, 4 Oct 2023 17:53:03 +0100 Message-Id: <20231004165317.1061855-4-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231004165317.1061855-1-willy@infradead.org> References: <20231004165317.1061855-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: C17C1180014 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: ju9ya1ubedtqhgw8cxpcxpjqcigkiy4w X-HE-Tag: 1696438401-915664 X-HE-Meta: U2FsdGVkX1/pp42HK/Ja/+0thvJqaH2oP3ARVAcMs8raziwchc5bhBBrjTgfOFXCMCqkvhgRiJbi8phHWz1d57Us5qwOzgrIl9LbbcTiKaRyyLbizMOx6hRkzE6dIc8K/il5oBk1xSbQfMeA47NtMhtMXFzeSLYxzyLfqmDCRCqDrGIbVFq7B3f4LR6eXBZnSLIkFnn/4usj8vyWhPTmQ7nGeEArqbsmgxBgpkWt7yU9Me56MsJsd2bKxBiW0bYNef5SQkxpZSRVQLTavDsKInvA0iOb+SIDSofSYnWhUH/R8x8p8erR+tkrTNTQK6NkUgMZ79WzFozFESpBXaiM+DPdl7k1eIB9RonEtyT/1/KWsrV7/VNc/j3y0y6Wg+7yl1gBzz6fgV1oJjyas/ZbUJVwLhcySRAH/M1jjnSIz3NJ/2EsAPhJ35EhaTt0GENH26ALX0FKG5EeLV42RUfTE4vVh3B+MjIXmj80P8Q0Qu3T1Cvas3rXgZvvE76ECXuF+AsgcylY/E/52fHY54M+mLlWABBjB+V86QW2ub6sWbi1TwysenzjlKFgmao1Z+t60fiOzVR//CvyvsH6Yi2cPlEL8V8hf9EDZ69EyMVnVvC4yBNLOlMuJ50oVTJ6U/cdXUD/6b8uChvk1MG3D3u4mZqPhTHYsrR/Q8i9gfdRoCWEC8L7XkxxkB+YlJqxM0aR1ddR/5gpoVBm50aL3LXaHw16S6aKtE/G6YGzEzwScpAYrGDsdLpYOCeOeniAL3jOLAHYVS53sVDqD74XxU2tcwm6oyarcdmpvbzM0XDd/VPH83yQ5lU/Jylwhr37eVB2lBpRc8KyDQME57110DotbWhY9KqDJ24nnW27t88EVRuHkAkYAaVp2cGTZvzHExaaibrU+41A50T/Zlb8Hf5sjQW+BRkVP2BxB76qI7frB70CO6DH1IEV3++tDRnockGoFXDFAI06uCU8kjWBeWy kcUF6Vtl zLdDPvkPHQ9yHxII2v/u/sdR2rDQeyGfyTIZtfAU3X+44J/OiFdhUx5eRR2Z/kpgU0eJdAmalYt95b3jaXiAVUzWsJGzimc0knwFP4SNHP8pLAAcvyX40SyaFyadjUI+yeFAPwTB0QNxztr7CeBdTYasJSljhDcjVUJHz0YE/dXYLrVfaczgxCZDSP0LYc23600KWcrSAtFshkgFhNKStk8v+R2JwgxjlbEgQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Provide a function for filesystems to call when they have finished reading an entire folio. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/pagemap.h | 1 + mm/filemap.c | 22 ++++++++++++++++++++++ 2 files changed, 23 insertions(+) diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 351c3b7f93a1..5bb2f5f802bc 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -1129,6 +1129,7 @@ static inline void wait_on_page_locked(struct page *page) folio_wait_locked(page_folio(page)); } +void folio_end_read(struct folio *folio, bool success); void wait_on_page_writeback(struct page *page); void folio_wait_writeback(struct folio *folio); int folio_wait_writeback_killable(struct folio *folio); diff --git a/mm/filemap.c b/mm/filemap.c index f0a15ce1bd1b..ea317cdf9532 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1527,6 +1527,28 @@ void folio_unlock(struct folio *folio) } EXPORT_SYMBOL(folio_unlock); +/** + * folio_end_read - End read on a folio. + * @folio: The folio. + * @success: True if all reads completed successfully. + * + * When all reads against a folio have completed, filesystems should + * call this function to let the pagecache know that no more reads + * are outstanding. This will unlock the folio and wake up any thread + * sleeping on the lock. The folio will also be marked uptodate if all + * reads succeeded. + * + * Context: May be called from interrupt or process context. May not be + * called from NMI context. + */ +void folio_end_read(struct folio *folio, bool success) +{ + if (likely(success)) + folio_mark_uptodate(folio); + folio_unlock(folio); +} +EXPORT_SYMBOL(folio_end_read); + /** * folio_end_private_2 - Clear PG_private_2 and wake any waiters. * @folio: The folio. From patchwork Wed Oct 4 16:53:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13409075 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F73FE7C4E5 for ; Wed, 4 Oct 2023 16:53:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 78F2E6B0289; Wed, 4 Oct 2023 12:53:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7647A6B028A; Wed, 4 Oct 2023 12:53:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5DDE36B028B; Wed, 4 Oct 2023 12:53:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 4A4A86B0289 for ; Wed, 4 Oct 2023 12:53:34 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 2494440396 for ; Wed, 4 Oct 2023 16:53:34 +0000 (UTC) X-FDA: 81308375148.25.2C8EA56 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf29.hostedemail.com (Postfix) with ESMTP id 6CB0112001A for ; Wed, 4 Oct 2023 16:53:32 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Aw3Cl2LA; dmarc=none; spf=none (imf29.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696438412; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZWM/c0Bk7RP+LkIKV16/78R+lD44jh4KeYNE9FiQZ2g=; b=sApBCg6eiWau/l78NZrYcvpCtcJIK7ef+IJS/kakImFmo6e5hc6jR1WJiBz1mlzonhrsee Uh0vC3Mjf0PyZu6xW/GCvyIUgrZ8NBL6+k+rf0L4hKxaqOPdSo3hMeo7zGkC443OLqO5R6 cGtrq8lbXUb54JQxeiByA1G1MFQUHK4= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Aw3Cl2LA; dmarc=none; spf=none (imf29.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696438412; a=rsa-sha256; cv=none; b=XVX67Rh2Rivm9xQM+9WhBs1v6H/B2dPdDc5p4enQytLj6KMjfDK93R5nFE1GzKDQVNc2hq dO1gl8eeFfZdx9OxGGRj6t6vKdUF9bf84jT+XH5NST0n0d0e6z8RYLlVHqrvDMrLu+RIaw oQ3KC3Sp+5NW79HnVzKy4os0+0cMHoU= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=ZWM/c0Bk7RP+LkIKV16/78R+lD44jh4KeYNE9FiQZ2g=; b=Aw3Cl2LAeVVsVTuLeNEwUq/VbA l81BMeA2S58v2J8f3iduduSKLq5Ymwo3HIDSHj2xkuXTosEziXIyE7ACXOdvHQWhHUd3P5J71NIMO xiGvXkUZP2C5I4rHffcnPBy9wVyXH5wXIZdFqV71NZtCtdtjIuNk9thk4UFfUpvE6etPONmOfwN65 zUosIjQElESczPZGIu2xynSiayxES/+8eLUaYJB0EYwYG+1frJmEMX9TeHHTFefunADL5TbmnR0R7 z5BWvZCFdQrRwZ+tua2GqBnj6E6lKbFUQ4tb4uEvGCnNpj3U9Ak8FHSTiKJeKeCNZlSrGLW0rM3HE 66+mylQw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qo57f-004SF0-Da; Wed, 04 Oct 2023 16:53:19 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, npiggin@gmail.com Subject: [PATCH v2 04/17] ext4: Use folio_end_read() Date: Wed, 4 Oct 2023 17:53:04 +0100 Message-Id: <20231004165317.1061855-5-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231004165317.1061855-1-willy@infradead.org> References: <20231004165317.1061855-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6CB0112001A X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 7qp3xaniqmbdpn9dpoi3apaac1t9x4dg X-HE-Tag: 1696438412-86795 X-HE-Meta: U2FsdGVkX18CaNlh4MU0bLMNtyA3K0/Ohfl8LxExuEtVAt+ag+w5tqkfOIFelZRsgpa3flhsYIttbBctUMtTmJ7/JAYsTfuxtJIWUYBc2T2rEM5KheVxD0K1h/aEtaLZIXffioOu2d8wvJYzjWMeQ48RGt7zPAPnGsxdiB2xfsg9cpnF/XPZ8u1Ajd3zGv17mDlmBn0dAcPXDTU4wDvwlnsbaDol9q9n9n8zUEJAZAay/yzaG/a+18H4wCXiMjwF/P/she4qYDuJDrF22w0DVABD6A/fUCflrvKni5G8qUNyF5MEIpBAtioSuv/p/TSVqIvkVM53wCoSdfKxDEvMKj/IH7WGceca4xlPMbEv2+LegCt4rrPcLTyDTgJf1/0QsD4B4lAnfj0IJODnsCVejFouDiqkJh5tzLuO5ME3zwrumctTr/2nRxi0sMSa5C9T88gKY+zHWD3qyz2nBPtM41raNxpTwEj/dIkvVS7nswg2u2PjlzbsKXmUIMztKauYyoxbNkBMRaP5q+mbGo4BO2mVzCrR5qA2CySAjAbN6chpiviqMSY9pbBqQhVrXPkVIz1/KDPn9f45s7BzPMqWbl8bn9qxidzHfb7Wr69H72G/ReCSUZlcGCNGq3U6ZqUt6IK1d300PvFb2ws1O0qqNso2Lmb/qyA2FUGuvintoqPMHf0Uy+lIlb07Czn9xYCpK5Pls+9Y3H/0eWI3O1u3dskc+6o7vKpoPsmls1WWO2lGTYWpSRyQJMmYN4VDEbIFEY+UjcF1LZltebFKRnTINWVBUUWQQIzU4UqzPezjpse/324y3qlyyM9RNxgQdzCCtbWZ8zBYQLDUNKMMSMYtadh7L1v2WeFYhxLEdPfZyHePgPsRPCjg1iBwKD/bdg3w4dNJ/rq8V8KHF/ZY+4q+V90CjfROpZ5EMbIyohGhLyq9Tll2i5CySMSTwpPdr8JCGzjHj6Untu8pxzkHzRP bPBOt/0Y Mz9+YM2lc3E2GYywWKh7DIfQHMdTTsFS2qUpBlAWp/5gdS1To9GR50cOVmWtz33Gz2Qi0Utk2NaXGhiZlQDBGxiJk9cfGBeVF/5lD5Ax74Ja9+SUcGYMIrdPKcPSA5H/Or/Djm6v10rS1u0gWUwpjTkdauPYWp+Si7t+g+aQbXVB7XyxCoQ2SkPTzO7+pBtebNzz43TC5EYax3KZkNF3BwW60om5REjsYICoQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: folio_end_read() is the perfect fit for ext4. Signed-off-by: Matthew Wilcox (Oracle) --- fs/ext4/readpage.c | 14 +++----------- 1 file changed, 3 insertions(+), 11 deletions(-) diff --git a/fs/ext4/readpage.c b/fs/ext4/readpage.c index 3e7d160f543f..21e8f0aebb3c 100644 --- a/fs/ext4/readpage.c +++ b/fs/ext4/readpage.c @@ -70,15 +70,8 @@ static void __read_end_io(struct bio *bio) { struct folio_iter fi; - bio_for_each_folio_all(fi, bio) { - struct folio *folio = fi.folio; - - if (bio->bi_status) - folio_clear_uptodate(folio); - else - folio_mark_uptodate(folio); - folio_unlock(folio); - } + bio_for_each_folio_all(fi, bio) + folio_end_read(fi.folio, bio->bi_status == 0); if (bio->bi_private) mempool_free(bio->bi_private, bio_post_read_ctx_pool); bio_put(bio); @@ -336,8 +329,7 @@ int ext4_mpage_readpages(struct inode *inode, if (ext4_need_verity(inode, folio->index) && !fsverity_verify_folio(folio)) goto set_error_page; - folio_mark_uptodate(folio); - folio_unlock(folio); + folio_end_read(folio, true); continue; } } else if (fully_mapped) { From patchwork Wed Oct 4 16:53:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13409078 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1484E7C4E5 for ; Wed, 4 Oct 2023 16:53:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 480BB6B028E; Wed, 4 Oct 2023 12:53:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4308B6B0290; Wed, 4 Oct 2023 12:53:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2A94A6B0291; Wed, 4 Oct 2023 12:53:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 193A36B028E for ; Wed, 4 Oct 2023 12:53:41 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id EDACCC03B9 for ; Wed, 4 Oct 2023 16:53:40 +0000 (UTC) X-FDA: 81308375400.18.59EE0E6 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf26.hostedemail.com (Postfix) with ESMTP id 3994E140013 for ; Wed, 4 Oct 2023 16:53:38 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="RYa3D/dV"; spf=none (imf26.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696438419; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5DKdxAyadRWDMiaslNEJr4yDpo8DKXMNV0hB6QcBBkM=; b=Jx9ZDLCXDvR67NABBO+24h7Yr767ZCM3AAKGbR2xejp9i4nKzDVj8DbQPnDplNoi/jBZYZ I4tW5hBV5MaiumUI3R4C1hlfrYLWmf7PONyZrCcNZ6wSErv+R9TrwBvrkxNZ6aXKajsdFo vheMwVgOoaVQGyxrcHtQGlu/xc6CdV8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696438419; a=rsa-sha256; cv=none; b=eN/jzovANlQCd7xlAvaxSTSggQFQHbNQ5LB195+0Z4mh9rNu0JnqmDjdmSQsd1lt8X2/1j MMG5L3aqeDh/cJRJ/d+jS9E3QlOhlq6nXJ38UPAVkbmfc3Zat50KXYgiZX6iPm8+wQUfgf mt1mTyyJ/w3Q7trew5GwFxPM2/IXWbk= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="RYa3D/dV"; spf=none (imf26.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=5DKdxAyadRWDMiaslNEJr4yDpo8DKXMNV0hB6QcBBkM=; b=RYa3D/dVz/0wGHfevaGhpqecso oT/Olzw7bs2m/lGKXPnZgBQatCf8z7bbcJjMtVfb0rXmyId6jA7S52FGGUqnby6PARrrczH/tB+vq We8hxsnFlycmu3R+t9Hg+VA4GM8ODiAUa7NU9DLjpuEbw/0AZ/iezCzuzB9+HkzTs3GrLbqbefqdz FjnB4rMgOWGRfIMh3v+2asasqkmqjO6UKVDTIjqqQ0D5VUKk1n+87VHmoGUUb67cVpiXOCZdSiMUp mQAZqJAfG8Mo6maQNsXErIyaMjGeLDbbjBrqxR0eyXp59QLdFngcX8oNVmFosewRlKa6F7HDj/h9a DAi5gaWw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qo57f-004SF3-GH; Wed, 04 Oct 2023 16:53:19 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, npiggin@gmail.com Subject: [PATCH v2 05/17] buffer: Use folio_end_read() Date: Wed, 4 Oct 2023 17:53:05 +0100 Message-Id: <20231004165317.1061855-6-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231004165317.1061855-1-willy@infradead.org> References: <20231004165317.1061855-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 3994E140013 X-Rspam-User: X-Stat-Signature: 1zs13dqw5fg43jzzzq4ir9tirseprq75 X-Rspamd-Server: rspam03 X-HE-Tag: 1696438418-837261 X-HE-Meta: U2FsdGVkX1/Cwh3ICutPmUbK8az+dAhnb+oRzD6i4sSgqNe6DTIPvEtEvEFqIB27Xt+3lhoDDU4XMo6cDMWvvUeLoDR5R0dpTtuGOKp7JWgsFoobGI3WoDXuKkABPGr3GOCAipEV5vph7birBkr9T4XqPGueIAbMNhTaTq+3R2vfByFmGBsanO0+AthaACHvJX2P7OFJIl46krdOcduE+tWLHfKSzojgjTHx47aH893bzQoPumJGy90ruEZncXal1i+RKOo1gYemBaSzzrhBnvzFjVMfJEtCbeHJCx5D8jBkFTA1rO2IShxtqbC/aGeFK/N9Pr6casIzeTvD7dCRL2c8n3TpW1FNmNFWm/mvkOiDBpQtG/YSAKB3M/tTNryP6QvwQ9ROWwl9V4l5t5l0jJ7BO88kd+WCs4FC8QnQfyorIB1nvi+tcCsgAByNuoSHX+KWjRnQ8uOR7G+BDZp8/FogHdhNLOhdWG22nbfa/rQmCJTXU+V/C6XJBJN4zfHZYNJoy72bukS3uHou84wIcEBrzS3/Phxy+QiqGEg1a66FWkh0guOOfSsSZKcAMbCdW4Eu0A1tNy9RUXrhqeRLOQY87IpGvNC7k+8UKpm98VWaAjlS6YNaptRGrFOur1j0ecAz91f1dXqOMHu5VyYHtK8fY8lm8tLQBkE2yQHyUuIz7BKxa6YPzPpM296A1iZKXlf673LyyHdHAOLfSrR++z+sf+OocE+gUUh69KOFss/q5+h6k1XV8PDO+qxJWrtVqN2yPl+mWqZoXuwOq5xhfXoDtfx+HSl+l0evlKaPkC99hkNSl9xmyyr0QAV9KpLG/xpyMMUDvtJ7zmrckPPfap71ReVHuOzlehjiBQPzdDgeoi/YwQRkpzYSYOxdnW9v3qSlK2/P27FQ03svlphB7T73CZbA2l4TBeLTdfm5DNAU/0cjkzti/q09j+ws8CbMNS31aZqMXWpkd4n8UgR KOJMVWbo Q1Ieiw29o918xUd8O8uuY5GiZL6mHWIsIKCxZl2D5q5azF0HAdnLlyPOSY5w/MrnY3mIoaouDq4QksTm5XvxautQ05wemgwAZHgcP1YGmPvWtKaIe+1TPQy/524ZLG67biipZVr335XhTqMD9quMOex7dw9hot9D6oQLD+wrMpeJegkJBFChE3h1xbWCeLiRIAR5kWrvUlu7Q9zMeajpatVckQoRrRX/h17yP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There are two places that we can use this new helper. Signed-off-by: Matthew Wilcox (Oracle) --- fs/buffer.c | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index 12e9a71c693d..b7bafba87273 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -282,13 +282,7 @@ static void end_buffer_async_read(struct buffer_head *bh, int uptodate) } while (tmp != bh); spin_unlock_irqrestore(&first->b_uptodate_lock, flags); - /* - * If all of the buffers are uptodate then we can set the page - * uptodate. - */ - if (folio_uptodate) - folio_mark_uptodate(folio); - folio_unlock(folio); + folio_end_read(folio, folio_uptodate); return; still_busy: @@ -2425,12 +2419,10 @@ int block_read_full_folio(struct folio *folio, get_block_t *get_block) if (!nr) { /* - * All buffers are uptodate - we can set the folio uptodate - * as well. But not if get_block() returned an error. + * All buffers are uptodate or get_block() returned an + * error when trying to map them - we can finish the read. */ - if (!page_error) - folio_mark_uptodate(folio); - folio_unlock(folio); + folio_end_read(folio, !page_error); return 0; } From patchwork Wed Oct 4 16:53:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13409071 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 561D4E7C4F4 for ; Wed, 4 Oct 2023 16:53:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7665E6B0272; Wed, 4 Oct 2023 12:53:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 717A06B0273; Wed, 4 Oct 2023 12:53:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 62D826B0274; Wed, 4 Oct 2023 12:53:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 508AE6B0272 for ; Wed, 4 Oct 2023 12:53:26 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 15735160242 for ; Wed, 4 Oct 2023 16:53:26 +0000 (UTC) X-FDA: 81308374812.09.417F99B Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf07.hostedemail.com (Postfix) with ESMTP id 5CC1B4001E for ; Wed, 4 Oct 2023 16:53:24 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=hqTAtSri; dmarc=none; spf=none (imf07.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696438404; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=U/Vyp0aKIbZ3gNGHQHjQi68gNh+m4QRK2qf3WFlHu+A=; b=ex34bWmnPwqrqPEUrdtY3oSIC12z3y0E1nI5/bTfsPUoDd6CBZPB8AoJ5rz5A8iqKu9KJ0 IyISq2zptiNRINJdJRkfdlWSxspRktLfvq+LDffMbRQv14fceRFfMWiWhvp/j/epvvQX86 CaaZToWuSMJA7QDh8xnphELWH1MOsu0= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=hqTAtSri; dmarc=none; spf=none (imf07.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696438404; a=rsa-sha256; cv=none; b=3LEjdyIQWynjjqo8TNDeItpGxkfA0ZHJi9iYezi2Wy5ArjB/6yeGfadxJR+ZT/T9odmQnd zTl9hFz/+tWEMMzb0lKqvkdvZrHhjG+jorDRkHuCqOSAJ7xNIlZS2ZD2/5OLO9LcR2uP8L jJKWgceBoFPceGZ3w1cQdvynrCPlEcM= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=U/Vyp0aKIbZ3gNGHQHjQi68gNh+m4QRK2qf3WFlHu+A=; b=hqTAtSridyfli6pM0HAtD5mGUD x3UeVRaQnugrAHI9tVmxaGGWCylda42PfRvDvDTCfZosLAJOrtxXWncHhy7ugK5RC+8zwpsSMQiHx zjILixsx8KeS/GggIiiCKJzhv2UGKpZCzBgV6BeAQu5ok/eK9hSUAyQrJ1O3ENZIUjeaQ7kp/7Nwe L0Ys+GJYDUnAgy3jIgg1/c1F8Haysgu81TvbZFxBtDXotFrU5uUqVVjYfr3jI4tBbrlSM/3o9pUXv 9MHl2Ue6kGqBxr1af3neVOGUntd1/rGg9l1B+TgD81IChLPjBy4PBM5P7HDY2256LbtVQFNhiR8m3 aDTshtSg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qo57f-004SF5-Iy; Wed, 04 Oct 2023 16:53:19 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, npiggin@gmail.com Subject: [PATCH v2 06/17] iomap: Use folio_end_read() Date: Wed, 4 Oct 2023 17:53:06 +0100 Message-Id: <20231004165317.1061855-7-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231004165317.1061855-1-willy@infradead.org> References: <20231004165317.1061855-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5CC1B4001E X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: hs1k4sfidcm6iysxuwyud5uzmix411of X-HE-Tag: 1696438404-584975 X-HE-Meta: U2FsdGVkX18zwsbD3fxs3sV9Xb21Ej+yaEP2NE4xBtar6x5VpJAPNil7Ksy1v3sCYY5545YU/Q+Xks+hRCBDlWYsejPxZZUbLuHQRvSBXLOVZz+AZCMDcfTEVnpntsSYkNCrMCWQZ4j4xVW4YL1nmjzasex/IVfJheFwdJobFl3ytQ4hf8tKYwMxl77jZueR84Gfm+rrKmf2UK9uSjWAQEWePeIAFR/JCEDtX7PUjiksaVP48ZatLi+tLvLg9b8lYJWJBBb42syYugDPaWv8bHSWpQpv0knPsbOluHYBaeQLYR9fMnVdTUnkQcugnDUxq+KY1Iky/lVBcMfFkZRRh/wRJShs+Kopor7KN7006XX8/eMB2t/WdlBheFTIJpceVJSlo0j+uk2p3DU+K+uqrREeNrE5ULs4cvSaEw0ODm8vgyzJHymtumyjSURSsXX8rWZL2qcw0mPsSXdMaPy9BSy91pvTUof2uYLP2669/m1DveI4UM719v3IM2wfcdIv0lwpbOP/mU/mBemzQt0zBbz+0ItqeD/kgRM0pGN0wKIArQGoWxaEjvMdwDSPkmmRZLP5VY7wkjCniIzwbhE9DpRqTed0i5U53c+FL/Aof8Teqy+iZMIDv8IxIG40lwfcGZ7GKjErGmN9JdiPdBOB+q9CzM65FxmQdQ8TgDnJ+TgUyGQZ1MZRGoXZZyg6ivtgVxITQDNunWKTyPHhIwKr0mtXp77sQykJb+PtFjnBtpWwO0N+Ohlmza3JVKzxxvYIBSQg3YnQrWbaoq9Mnc5ORLg3rr4f5EzncUmbiuneowdYn+cauBfJ+1if8RKf0hfrTYfmplkySPhIxJ7f+vCTPcmBwBJkQ17eZa11LVSfIwlO6+e52H4tp2wFdUGe122Nz1RcNRN/Apt77wWJEiFLL0KC8ngJ7DM+Wirb90AhEBn9F7lNdcuNVkppi72MlBTtKi97GN9X9lM0V1SLjh6 RbJLeDDK IH2Mu6NxMatLLPFhg52sLvxQ7tTAlfj3zBJxe6GcGXwvoFPCyIMl+Z7a/ePOkoGaLRhc+I9Ch/o+IhO37loYMsf+Kiit76UeS8bBCyybSNir0hxEoNC5w78ZQaQ3rb1zuYZVbeP3G+GQa9ofDkT/Bw0Crnx6knrh/II4AKqlZKFludgwnAXA0siadikeKua3S9sqlhsJmQGV51uUJGmBaS1fZMPQlx0Vdi1CZ8hqs3zwADrUsHDREO7pnWdEJ1kiIvuO1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Combine the setting of the uptodate flag with the clearing of the locked flag. Signed-off-by: Matthew Wilcox (Oracle) --- fs/iomap/buffered-io.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 4a996c5327ef..5d19a2b47b6a 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -270,10 +270,8 @@ static void iomap_finish_folio_read(struct folio *folio, size_t off, if (error) folio_set_error(folio); - if (uptodate) - folio_mark_uptodate(folio); if (finished) - folio_unlock(folio); + folio_end_read(folio, uptodate); } static void iomap_read_end_io(struct bio *bio) From patchwork Wed Oct 4 16:53:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13409079 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E350CE7C4E2 for ; Wed, 4 Oct 2023 16:53:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4F5106B0290; Wed, 4 Oct 2023 12:53:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4A50C6B0292; Wed, 4 Oct 2023 12:53:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 31F3F6B0294; Wed, 4 Oct 2023 12:53:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 23A856B0290 for ; Wed, 4 Oct 2023 12:53:44 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id DD1881A03AF for ; Wed, 4 Oct 2023 16:53:43 +0000 (UTC) X-FDA: 81308375526.26.889771D Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf13.hostedemail.com (Postfix) with ESMTP id 383D620008 for ; Wed, 4 Oct 2023 16:53:41 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=UqleOktT; spf=none (imf13.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696438422; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OIlQfxJf3xAAiVPkn5SRGC4SYIlWa7s9rUyJZz1UNfs=; b=AAaaGnMwhAR6OmvvfefROYXvuasVppucA4uLTmOXkY4c/kwyQfxFhkDQUplWpfuHpqAQ6/ QZa1xigFXbepwcIhPWqVoOZNuWcogIwyhC4EqP18DKFDbvqWPZPUWNEPSzRHrOxIoklwFP FhvcPxLFxHe6P85VnbSMRyb3zDZP7Jo= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=UqleOktT; spf=none (imf13.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696438422; a=rsa-sha256; cv=none; b=SxsRmSfl6p6Y2Syto/sCyKiKdVz3TRMaKONwSASVLxLa+S32HXZPdQhysGzlWQZp3BkLZD Lhz3FPMT6+7RPh00RD4tA7KNkB75soFHq3Tw42xrFZ7rdr2WHAHkfEOk8bMmSq4XTAVCB6 vQ/TIiQUvT04Iu8boImqpezPZttFdIA= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=OIlQfxJf3xAAiVPkn5SRGC4SYIlWa7s9rUyJZz1UNfs=; b=UqleOktTdw71dOuqvUTkD/lTJC /1rNyzDWQux+76a/MyNZ7b97z52LjXlT+ak8UrubShO37NVtXE7k3Ph8/mT57SkTDQpAHSUKSTuu+ lZqaCxa7E81SY6oE4eEl1ohjTq6wwoE+ETJHlDi14DrN4NtXUJL1S8AYaArG2RntaEi1z1S66ufX+ Mla38SKlbyTzqxALsm+yu60psa1+hSMwo8NDSoRKlVfcWn5ElJqp3R/QObk1kXt0Yh7Ug4IhOY1zT cVDoxMJkxwI2p0Mf4fu1N1VjR7bmROWaxwNBJO4NFBFdxiVWxWWp4jYgraRylFw9dv/tKj+OrBI/9 BBp2jzDA==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qo57f-004SF7-LD; Wed, 04 Oct 2023 16:53:19 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, npiggin@gmail.com Subject: [PATCH v2 07/17] bitops: Add xor_unlock_is_negative_byte() Date: Wed, 4 Oct 2023 17:53:07 +0100 Message-Id: <20231004165317.1061855-8-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231004165317.1061855-1-willy@infradead.org> References: <20231004165317.1061855-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 383D620008 X-Rspam-User: X-Stat-Signature: mpciet3ipbcodqr831aa4mp89hqh9maz X-Rspamd-Server: rspam01 X-HE-Tag: 1696438421-734316 X-HE-Meta: U2FsdGVkX1/Sk49x9+cPQJ7R2tyYgNsnVqRWSWBjGDQ7XOz+QdrMChMIYvn1y5HyKkVbO4sAESVnGH0h5QebKGIjIpDFnwthK0whMHe3TVdzSm3RyAcUsIwG64ZQDe+kJR0xLMBm9ubl0ph/rvU7H4fcjrMRWg/C08TzULWh/QkCRv2mT9p0BZIQ+gxczz8jlq+nA2Sv99soQS8esNi6Uy15YvwGJtnBOcyf+SX6YCIzxrszdxTlzt8M8jHWfnbbJ6eEQ3oe3Got+h9wIKoSDX5v1qm3+TN5ASF9qDomMlaBIJu6x2KdZ0voXYVT7ziJ/McPpfepKIk7YU/hVnBk7THlJF5Q09XBvuTspmwiU9thFiN2ViZJqFJ7ko18iu/AMXJV04HLiJFqkidkF+XUJDxJ8tUXhHP3HMSyePVnfCecwvE6ZcjdbruSPOc6n7E39W9iSc6kD3EbVXvNP98kA2oLQiZyvxtpk3i+UqYArsANl13F1A1JoNzEYb65yKyEkhBXgXNTtU2eJQNMar8XPOLn22JCK7JYi+qUZmYM1KJVy5r9FZeN0AQ1OTj2AjTI0JaadNHqb3/sLRJw1Yps9SbJBqWKAhLQyxUY5bQjqI/tq0/HU1SlzfpLM/wvaK1rGp14t3ENzg+PqdM17GurouPdgE2eeK8Rq7ZOCMT5zYTJOr9cI1/JG0d3xVKu9UGgyLMt0IQ//fc+Ekw/7JfwdyjmRI4Fi9WDVpOAl4A9dTtxF1/LNKMz54cvLuKDmzEhOpBDGpv7Hzr64ztQUfj9mp0somGoOBV8uXCqXCy176QLDeiYP/zLACpWUGHhbyQXUchW5LT0PjBTecCgyBgdmB6ELFcELCE6aTitW9x9MnufyFQ9G+4Ol7Zc+rpN74I/sPF6rp1BA4G4TpYap5i5coHrVzX18bbs2f6SjapnHQnPUSDTLfdtcYT1tNTU1WW9QEhaHBevLuznnwQq0FG RrpKN5Z0 qhTc6RM34g9A5/QgQ47+6vnZy+CJ333+a8wraBspda+glK/ZVY6RKeQbwqTY0SE3e1gG/LbJagl1+IexMeGa3BKDFFYHBm909Cegm31/2Nz/s28av98hmOCc4APDavT8coIjQPkasw+QDKO/vBxxWTfNfpS7xqfaXaOGtHdGAf1tZO2AoWbl7cRhAix7kZWUCQTxk3o+sHYyUoBmoRDeU9tq7WoTrZYy+eA6k X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Replace clear_bit_and_unlock_is_negative_byte() with xor_unlock_is_negative_byte(). We have a few places that like to lock a folio, set a flag and unlock it again. Allow for the possibility of combining the latter two operations for efficiency. We are guaranteed that the caller holds the lock, so it is safe to unlock it with the xor. The caller must guarantee that nobody else will set the flag without holding the lock; it is not safe to do this with the PG_dirty flag, for example. Signed-off-by: Matthew Wilcox (Oracle) --- arch/powerpc/include/asm/bitops.h | 17 ++++-------- arch/x86/include/asm/bitops.h | 11 ++++---- .../asm-generic/bitops/instrumented-lock.h | 27 ++++++++++--------- include/asm-generic/bitops/lock.h | 21 ++++----------- kernel/kcsan/kcsan_test.c | 8 +++--- kernel/kcsan/selftest.c | 8 +++--- mm/filemap.c | 5 ++++ mm/kasan/kasan_test.c | 7 ++--- 8 files changed, 47 insertions(+), 57 deletions(-) diff --git a/arch/powerpc/include/asm/bitops.h b/arch/powerpc/include/asm/bitops.h index 7e0f0322912b..40cc3ded60cb 100644 --- a/arch/powerpc/include/asm/bitops.h +++ b/arch/powerpc/include/asm/bitops.h @@ -234,32 +234,25 @@ static inline int arch_test_and_change_bit(unsigned long nr, } #ifdef CONFIG_PPC64 -static inline unsigned long -clear_bit_unlock_return_word(int nr, volatile unsigned long *addr) +static inline bool arch_xor_unlock_is_negative_byte(unsigned long mask, + volatile unsigned long *p) { unsigned long old, t; - unsigned long *p = (unsigned long *)addr + BIT_WORD(nr); - unsigned long mask = BIT_MASK(nr); __asm__ __volatile__ ( PPC_RELEASE_BARRIER "1:" PPC_LLARX "%0,0,%3,0\n" - "andc %1,%0,%2\n" + "xor %1,%0,%2\n" PPC_STLCX "%1,0,%3\n" "bne- 1b\n" : "=&r" (old), "=&r" (t) : "r" (mask), "r" (p) : "cc", "memory"); - return old; + return (old & BIT_MASK(7)) != 0; } -/* - * This is a special function for mm/filemap.c - * Bit 7 corresponds to PG_waiters. - */ -#define arch_clear_bit_unlock_is_negative_byte(nr, addr) \ - (clear_bit_unlock_return_word(nr, addr) & BIT_MASK(7)) +#define arch_xor_unlock_is_negative_byte arch_xor_unlock_is_negative_byte #endif /* CONFIG_PPC64 */ diff --git a/arch/x86/include/asm/bitops.h b/arch/x86/include/asm/bitops.h index 2edf68475fec..f03c0a50ec3a 100644 --- a/arch/x86/include/asm/bitops.h +++ b/arch/x86/include/asm/bitops.h @@ -94,18 +94,17 @@ arch___clear_bit(unsigned long nr, volatile unsigned long *addr) asm volatile(__ASM_SIZE(btr) " %1,%0" : : ADDR, "Ir" (nr) : "memory"); } -static __always_inline bool -arch_clear_bit_unlock_is_negative_byte(long nr, volatile unsigned long *addr) +static __always_inline bool arch_xor_unlock_is_negative_byte(unsigned long mask, + volatile unsigned long *addr) { bool negative; - asm volatile(LOCK_PREFIX "andb %2,%1" + asm volatile(LOCK_PREFIX "xorb %2,%1" CC_SET(s) : CC_OUT(s) (negative), WBYTE_ADDR(addr) - : "ir" ((char) ~(1 << nr)) : "memory"); + : "iq" ((char)mask) : "memory"); return negative; } -#define arch_clear_bit_unlock_is_negative_byte \ - arch_clear_bit_unlock_is_negative_byte +#define arch_xor_unlock_is_negative_byte arch_xor_unlock_is_negative_byte static __always_inline void arch___clear_bit_unlock(long nr, volatile unsigned long *addr) diff --git a/include/asm-generic/bitops/instrumented-lock.h b/include/asm-generic/bitops/instrumented-lock.h index eb64bd4f11f3..e8ea3aeda9a9 100644 --- a/include/asm-generic/bitops/instrumented-lock.h +++ b/include/asm-generic/bitops/instrumented-lock.h @@ -58,27 +58,30 @@ static inline bool test_and_set_bit_lock(long nr, volatile unsigned long *addr) return arch_test_and_set_bit_lock(nr, addr); } -#if defined(arch_clear_bit_unlock_is_negative_byte) +#if defined(arch_xor_unlock_is_negative_byte) /** - * clear_bit_unlock_is_negative_byte - Clear a bit in memory and test if bottom - * byte is negative, for unlock. - * @nr: the bit to clear - * @addr: the address to start counting from + * xor_unlock_is_negative_byte - XOR a single byte in memory and test if + * it is negative, for unlock. + * @mask: Change the bits which are set in this mask. + * @addr: The address of the word containing the byte to change. * + * Changes some of bits 0-6 in the word pointed to by @addr. * This operation is atomic and provides release barrier semantics. + * Used to optimise some folio operations which are commonly paired + * with an unlock or end of writeback. Bit 7 is used as PG_waiters to + * indicate whether anybody is waiting for the unlock. * - * This is a bit of a one-trick-pony for the filemap code, which clears - * PG_locked and tests PG_waiters, + * Return: Whether the top bit of the byte is set. */ -static inline bool -clear_bit_unlock_is_negative_byte(long nr, volatile unsigned long *addr) +static inline bool xor_unlock_is_negative_byte(unsigned long mask, + volatile unsigned long *addr) { kcsan_release(); - instrument_atomic_write(addr + BIT_WORD(nr), sizeof(long)); - return arch_clear_bit_unlock_is_negative_byte(nr, addr); + instrument_atomic_write(addr, sizeof(long)); + return arch_xor_unlock_is_negative_byte(mask, addr); } /* Let everybody know we have it. */ -#define clear_bit_unlock_is_negative_byte clear_bit_unlock_is_negative_byte +#define xor_unlock_is_negative_byte xor_unlock_is_negative_byte #endif #endif /* _ASM_GENERIC_BITOPS_INSTRUMENTED_LOCK_H */ diff --git a/include/asm-generic/bitops/lock.h b/include/asm-generic/bitops/lock.h index 40913516e654..6a638e89d130 100644 --- a/include/asm-generic/bitops/lock.h +++ b/include/asm-generic/bitops/lock.h @@ -66,27 +66,16 @@ arch___clear_bit_unlock(unsigned int nr, volatile unsigned long *p) raw_atomic_long_set_release((atomic_long_t *)p, old); } -/** - * arch_clear_bit_unlock_is_negative_byte - Clear a bit in memory and test if bottom - * byte is negative, for unlock. - * @nr: the bit to clear - * @addr: the address to start counting from - * - * This is a bit of a one-trick-pony for the filemap code, which clears - * PG_locked and tests PG_waiters, - */ -#ifndef arch_clear_bit_unlock_is_negative_byte -static inline bool arch_clear_bit_unlock_is_negative_byte(unsigned int nr, - volatile unsigned long *p) +#ifndef arch_xor_unlock_is_negative_byte +static inline bool arch_xor_unlock_is_negative_byte(unsigned long mask, + volatile unsigned long *p) { long old; - unsigned long mask = BIT_MASK(nr); - p += BIT_WORD(nr); - old = raw_atomic_long_fetch_andnot_release(mask, (atomic_long_t *)p); + old = raw_atomic_long_fetch_xor_release(mask, (atomic_long_t *)p); return !!(old & BIT(7)); } -#define arch_clear_bit_unlock_is_negative_byte arch_clear_bit_unlock_is_negative_byte +#define arch_xor_unlock_is_negative_byte arch_xor_unlock_is_negative_byte #endif #include diff --git a/kernel/kcsan/kcsan_test.c b/kernel/kcsan/kcsan_test.c index 0ddbdab5903d..1333d23ac4ef 100644 --- a/kernel/kcsan/kcsan_test.c +++ b/kernel/kcsan/kcsan_test.c @@ -700,10 +700,10 @@ static void test_barrier_nothreads(struct kunit *test) KCSAN_EXPECT_RW_BARRIER(mutex_lock(&test_mutex), false); KCSAN_EXPECT_RW_BARRIER(mutex_unlock(&test_mutex), true); -#ifdef clear_bit_unlock_is_negative_byte - KCSAN_EXPECT_READ_BARRIER(clear_bit_unlock_is_negative_byte(0, &test_var), true); - KCSAN_EXPECT_WRITE_BARRIER(clear_bit_unlock_is_negative_byte(0, &test_var), true); - KCSAN_EXPECT_RW_BARRIER(clear_bit_unlock_is_negative_byte(0, &test_var), true); +#ifdef xor_unlock_is_negative_byte + KCSAN_EXPECT_READ_BARRIER(xor_unlock_is_negative_byte(1, &test_var), true); + KCSAN_EXPECT_WRITE_BARRIER(xor_unlock_is_negative_byte(1, &test_var), true); + KCSAN_EXPECT_RW_BARRIER(xor_unlock_is_negative_byte(1, &test_var), true); #endif kcsan_nestable_atomic_end(); } diff --git a/kernel/kcsan/selftest.c b/kernel/kcsan/selftest.c index 8679322450f2..619be7417420 100644 --- a/kernel/kcsan/selftest.c +++ b/kernel/kcsan/selftest.c @@ -228,10 +228,10 @@ static bool __init test_barrier(void) spin_lock(&test_spinlock); KCSAN_CHECK_RW_BARRIER(spin_unlock(&test_spinlock)); -#ifdef clear_bit_unlock_is_negative_byte - KCSAN_CHECK_RW_BARRIER(clear_bit_unlock_is_negative_byte(0, &test_var)); - KCSAN_CHECK_READ_BARRIER(clear_bit_unlock_is_negative_byte(0, &test_var)); - KCSAN_CHECK_WRITE_BARRIER(clear_bit_unlock_is_negative_byte(0, &test_var)); +#ifdef xor_unlock_is_negative_byte + KCSAN_CHECK_RW_BARRIER(xor_unlock_is_negative_byte(1, &test_var)); + KCSAN_CHECK_READ_BARRIER(xor_unlock_is_negative_byte(1, &test_var)); + KCSAN_CHECK_WRITE_BARRIER(xor_unlock_is_negative_byte(1, &test_var)); #endif kcsan_nestable_atomic_end(); diff --git a/mm/filemap.c b/mm/filemap.c index ea317cdf9532..7da465616317 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1484,6 +1484,11 @@ void folio_add_wait_queue(struct folio *folio, wait_queue_entry_t *waiter) } EXPORT_SYMBOL_GPL(folio_add_wait_queue); +#ifdef xor_unlock_is_negative_byte +#define clear_bit_unlock_is_negative_byte(nr, p) \ + xor_unlock_is_negative_byte(1 << nr, p) +#endif + #ifndef clear_bit_unlock_is_negative_byte /* diff --git a/mm/kasan/kasan_test.c b/mm/kasan/kasan_test.c index b61cc6a42541..c1de091dfc92 100644 --- a/mm/kasan/kasan_test.c +++ b/mm/kasan/kasan_test.c @@ -1098,9 +1098,10 @@ static void kasan_bitops_test_and_modify(struct kunit *test, int nr, void *addr) KUNIT_EXPECT_KASAN_FAIL(test, __test_and_change_bit(nr, addr)); KUNIT_EXPECT_KASAN_FAIL(test, kasan_int_result = test_bit(nr, addr)); -#if defined(clear_bit_unlock_is_negative_byte) - KUNIT_EXPECT_KASAN_FAIL(test, kasan_int_result = - clear_bit_unlock_is_negative_byte(nr, addr)); +#if defined(xor_unlock_is_negative_byte) + if (nr < 7) + KUNIT_EXPECT_KASAN_FAIL(test, kasan_int_result = + xor_unlock_is_negative_byte(1 << nr, addr)); #endif } From patchwork Wed Oct 4 16:53:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13409080 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A57A3E7C4E3 for ; Wed, 4 Oct 2023 16:53:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3481F6B0292; Wed, 4 Oct 2023 12:53:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2F7956B0296; Wed, 4 Oct 2023 12:53:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1711A6B0297; Wed, 4 Oct 2023 12:53:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id EF0F06B0292 for ; Wed, 4 Oct 2023 12:53:46 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id BFFCE1A031B for ; Wed, 4 Oct 2023 16:53:46 +0000 (UTC) X-FDA: 81308375652.25.9EB6229 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf29.hostedemail.com (Postfix) with ESMTP id 37ECF12001A for ; Wed, 4 Oct 2023 16:53:44 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=RPsCKy9R; dmarc=none; spf=none (imf29.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696438425; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GcBru3fPdpYKCu6uB4MY5f4K8PaG3ucCqPSqDGXRe9s=; b=fWZSrUwj0VcJMiKHHTeMWe6NrmCnxmBNMnAcWoyo4mSOtytItlqFJwS0crpOdog+dZ7fbb cFTF1ky23Vh5XhyzzTJ6rTZVQhdfRgI1B6GZMLAJku+AiiiE7KbxrwYqMTFzt4Tr830etT DCblQV6VIyZsXvYBEalAcAEOAaoPxCo= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=RPsCKy9R; dmarc=none; spf=none (imf29.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696438425; a=rsa-sha256; cv=none; b=70ypLtKB0UVcC/bBrISB6pALG2StnoSTOLocdsqOydTdC5Ci7v3la2RzwLsCbTrku49CGH Z3+NIDuye8oJfS9p7WgXp1MaT7YH6ybrr90cdfKzY3kw0JMJX72bCw4lBktmZLZFre6RIc zTe0LyUKw2v27mrhW46txwGjGKI5Pr4= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=GcBru3fPdpYKCu6uB4MY5f4K8PaG3ucCqPSqDGXRe9s=; b=RPsCKy9Rm26TskJl3Eur107F7S 57OHLJxeUuwkSmlr5qVXZQNjmq1fs6H47cGDYcmjzJxyEwDsj8u0jKLys/oRopqDf28XCm639dRkF 9yBzlYSdR7vgpEP7jHoxcf6LR3/zR9PCM+Ttcw7pDWHC2Fb1C3BcOcK5DulZT8hp7ezhCo/yCky9S pkxMgSich07C8uTk28HlWqZ3lcIgGVe23a/iYvjX/OrOn/BuvAeYNMag8CYx0+Xxssf72G8/uGqXV AY5Kndoit3mw7LA3XgJfj3g5Ben6dO64o9z+tqXsl9Vk8Z6UXSNiqQBLfYajlHYtcokZxJlu28Tdu owaSdX3Q==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qo57f-004SFA-Pp; Wed, 04 Oct 2023 16:53:19 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, npiggin@gmail.com Subject: [PATCH v2 08/17] alpha: Implement xor_unlock_is_negative_byte Date: Wed, 4 Oct 2023 17:53:08 +0100 Message-Id: <20231004165317.1061855-9-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231004165317.1061855-1-willy@infradead.org> References: <20231004165317.1061855-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 37ECF12001A X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: hy5jcq78z7ca6p88yciuxhn17ub8im7m X-HE-Tag: 1696438424-276292 X-HE-Meta: U2FsdGVkX18zHAzAJccHwIcSSrAYa02wIlFvLN6NY59Q32xf0lBT+ZYTQKgENBHGTaGs/C2jpH3RnyKboJn/IuQWLzo14u+c1UGjoNw33X+/62PLqnZa1UphrexjYz1mfj4aFU7hdzHyTDVUUHyjoUwSKRnupzuzwvEaow+cFl1GM2iw+DGpzH9sFHQBUxGuDcM9tBQfylyrcDNPwcmd1erZFjHT94FQaiUFRIkDaQOGZnUkmM1nzupLx2mV0W253PzPYtB2Og6AVM5VEoERlitIp4fxW3eAjz0xSc9SyWe1OewynZl0ggWPalSS5omrD5LHPtQaB55hRPKQ2fJ8RGrtwXRgDrT3uhqvft6NHSq/yYzjuOSppvEpFI4pgaIwCEMXxowyzPo5OTLNChKV85xR6iOID6gDDOsLM2sBIakv4skMVy+DX71jYkkmrwagFLY7AxGX9bO2P4PHdFGsYjx0oKop2sVV6Yo2ieseaKaBaTuGve9WgKSf/FE+IXY44v30lg90EkOVPHyaxsbSLy8oLGf0WQpwruivmj2OfNQnRGMjs7t+rkOjIfD1EUW+koMWzEVchIcdR4ka9LLDaifRJCwRg8CRPtbVXNrcBWPDgsx6o848XB2x4Z7KkQdRadIsqgSI86LvoIUlslPAztx9Gsgm7rGauWEWSj9/ttqfPCn3iAseqlqBVe1r48eRzq3jHUSZBNreyXPHhVE5mHPVkITqgmvTcRx8l9w8+nM4Wo6xtNibVE36M7ndeoNSSLCHXxLKKYSdD3TjL0sTBPkRUugeVRV7J1PV4FS6G/h/F8Df2IjKYZj/w1Tbc2l8jj1oJQZbQBkrogpZx1bYoWn/7cWHdVj26oRF8ogKorasLxxlRVQv3Y5RQtwsltmgmcGXX8j6+mZ837svW/A40l+p6JIAN6nW3NUnRL4633zMZv2GCO0bxSMNct7GeVrilqmzclFaS4QTFiApjOe N/UCwrsM iEH4C5dnyg7Cg1itZ7uRyBoT3vOy/YoTJ0GSNpJwYX9ROznWW7A7eBwXBg/4F8eYxBqHLOz0IySJkulZonVDjpIO3WFG4PMbHNN0Zq3E5GjcvEdYmZbs2bkWh6QU33Mza9MIDOBWL7cagCaUMXPkkm7w0N9Y8Tic5xKjFaBbflrjNxDzXlvSS1mUAoJbHXJcebrJ6fAK/bLJKzenn03+Qs4fszrS9+sOBiAIR X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Inspired by the alpha clear_bit() and arch_atomic_add_return(), this will surely be more efficient than the generic one defined in filemap.c. Signed-off-by: Matthew Wilcox (Oracle) --- arch/alpha/include/asm/bitops.h | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/arch/alpha/include/asm/bitops.h b/arch/alpha/include/asm/bitops.h index bafb1c1f0fdc..b50ad6b83e85 100644 --- a/arch/alpha/include/asm/bitops.h +++ b/arch/alpha/include/asm/bitops.h @@ -286,6 +286,27 @@ arch___test_and_change_bit(unsigned long nr, volatile unsigned long *addr) #define arch_test_bit generic_test_bit #define arch_test_bit_acquire generic_test_bit_acquire +static inline bool xor_unlock_is_negative_byte(unsigned long mask, + volatile unsigned long *p) +{ + unsigned long temp, old; + + __asm__ __volatile__( + "1: ldl_l %0,%4\n" + " mov %0,%2\n" + " xor %0,%3,%0\n" + " stl_c %0,%1\n" + " beq %0,2f\n" + ".subsection 2\n" + "2: br 1b\n" + ".previous" + :"=&r" (temp), "=m" (*p), "=&r" (old) + :"Ir" (mask), "m" (*p)); + + return (old & BIT(7)) != 0; +} +#define xor_unlock_is_negative_byte xor_unlock_is_negative_byte + /* * ffz = Find First Zero in word. Undefined if no zero exists, * so code should check against ~0UL first.. From patchwork Wed Oct 4 16:53:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13409064 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24530E7C4E8 for ; Wed, 4 Oct 2023 16:53:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 98AA56B0266; Wed, 4 Oct 2023 12:53:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 878D26B026E; Wed, 4 Oct 2023 12:53:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 62AAD6B026F; Wed, 4 Oct 2023 12:53:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 504A96B0266 for ; Wed, 4 Oct 2023 12:53:24 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 13DA61403AF for ; Wed, 4 Oct 2023 16:53:24 +0000 (UTC) X-FDA: 81308374728.21.6702432 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf06.hostedemail.com (Postfix) with ESMTP id 040F4180019 for ; Wed, 4 Oct 2023 16:53:21 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=NWC6d6ZF; spf=none (imf06.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696438402; a=rsa-sha256; cv=none; b=ysnceRZ9X+OJzSSN8lY34A6l5ZCKbaTIjbkB4rM2kOzyGxC5wRpYTbaTdhTwCqvmY2v90l xCkOwaCNenuxDOB6IrtFNpwrt7FpfXzZwYKCo85srR7bKYFkW/hLluZEEyv3XsLchah+KN J0B9Gtpk1IITRomhSTkGLbB8bxwigW4= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=NWC6d6ZF; spf=none (imf06.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696438402; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bfs2p6og8MVmu/PmsnwWj8Grt12H9inYLMhPvOjgtD4=; b=n+pD70rhOh90ylLJjRBzOdG9mwQQXv3WlZSJ0Z9RodDBX3Jk2j7CWSNbgxgU0NUXRq+759 coopkp6AvhDPjmt9iZiAmGNDuVLDZPrI1PVaa+IHmcHFxo/iDXiyXyJ13RSVb/wYYg3C5y zvjV9WeMaVrIfGPBDFkfV4qcAY43SPo= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=bfs2p6og8MVmu/PmsnwWj8Grt12H9inYLMhPvOjgtD4=; b=NWC6d6ZFQxlxn/oWJ2tJnU0gwB ZelMR2zlgkLzaavxoF29vyK9rUKk3wZoVjQYF4K0JVoHtYF3lYdM3bogFOO8lFAEDNgqsWBU/VXZr YHo3HmIrljWo2RkuVx1+vSIVD0DPaQJalBikdceY0434SmUBzRY1jyCXk/mJwjly9XRiLKBwBwU0m 2HpWLNh91z6H8q4wUtSN7qp3N6J7evJn15lOl5L3vi8PtH4ySsZtI3e7CYoaeQaMTanTOJmyLMy2C 6wLxIkOasNlqbdxCUl5F6HQe+SYdShEeFYIl91MG5vji2Y56KV2H2hSVVyLjqcZCmQMVLsqOrDh6m rKkojOcg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qo57f-004SFN-Sx; Wed, 04 Oct 2023 16:53:19 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, npiggin@gmail.com Subject: [PATCH v2 09/17] m68k: Implement xor_unlock_is_negative_byte Date: Wed, 4 Oct 2023 17:53:09 +0100 Message-Id: <20231004165317.1061855-10-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231004165317.1061855-1-willy@infradead.org> References: <20231004165317.1061855-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 040F4180019 X-Stat-Signature: sdwz7t9iaz5hridn89wgqa9j8edx33fu X-Rspam-User: X-HE-Tag: 1696438401-626384 X-HE-Meta: U2FsdGVkX18zrYirk6WYdwM0tyrHDyi51d8Cb3c3mErxOXpKfHUaiRSaJ1t4NrjmbbrEK1NTiIAduf8ZEhxP6w4vIm5I/E9CxdAryhlF5sNfkFT4HQrUpUhF7zBEvjGZtxjDY1yQSctMnpgGEgnJdVj5PtIzxO/BB8Ge/UZoxm53Bc9C4WXbCxZZVk5sHOubunfP8l9bDXNfafIn4wL4NvXiPUe5Ujml+dkYxxLYoc5P/tDMTZNSbyUe+lCXQNUBdIKfGpRjg2Y89lXtVsvVL1RDVjopjy5AmPe5PavjGnR8bhOVWSh4313Ek55D09Xh+ACpEaYxcbl0dnviyEkXLEVKAI2/wr0/NecpcCA5neEVvRauDuSAum6iBOOxris5gGyzQ8Ws8Npif0ww5DYgpQkCRfXBUbv6CoQXLCpBHNSkMu20iKv+GQZV0QRvIIw92IJ8gQnwZY5XbsKNzia8AIGCZx67sjH2Gm9dLWLhBjdOnqWVjPwoF0LzKMm5rkvpILm0eAaYQB0aBLlKcnZcnDiwTyUNLQNi3juBeNCS02B4yIvoINTgm4Qic/mk5ZqJiJhaKrKAXTmNYJOWyad2uUXPBL5JI3KXIsHCOiBalJOUgD0BEekrYVlyJ5GlynKbZxTMTR/HZn9OPpCyt+xckhtgsVozqWRjtczTkTuncW/eKp9yyAbEZS8jLiaz11AJaHWHf//WSqWT3+a8lrcxOZ7vnGeyNLCKa8xrmtcxKJGTP9X2bIHjpxiDUhFOGRaObnHqMSscGEVkHWEH0RT+GTj/Lo4y8zqE/JcIe/Osx7s7014g7vNiuFtIlAycc8ohB2+Eqg4A3v+sXjruI2D13M/0sfyl+qmQ4B+FubkHLZwrZvDJMeR+GdMrIwMCAwjgIu62HhFGGOg7n+5wJhMuvsg/77nLjAQHnn24BBRUe6aaEXBYoSMM4r87sDramhO83hgYi3cD9MFrd3Qrj6N SRYIbM5r oJd3ApruOxW/nGsDaFLHzaOzVb37dAUKShbfJOfZ1mbNt3vE5xI+TFtm5h5EQKNTWRUMd6ZIJqNRO3CCDjzTW9MMEfXl+OKBfD/2laX0dwvsrMT1pocZwncG6YFyp1dyoOWzoijvpUdTV8JfZDYMiiZDywcykGEvD81yuvhjc/CipKQmk38pJsESI5M6gTMKK94zYpw6nYGD51YGMptuyF0SOnLWTYahlEC5N X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Using EOR to clear the guaranteed-to-be-set lock bit will test the negative flag just like the x86 implementation. This should be more efficient than the generic implementation in filemap.c. It would be better if m68k had __GCC_ASM_FLAG_OUTPUTS__. Coldfire doesn't have a byte-sized EOR, so we test bit 7 after the EOR, which is a second memory access, but it's slightly better than the current C code. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Greg Ungerer Acked-by: Geert Uytterhoeven --- arch/m68k/include/asm/bitops.h | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/arch/m68k/include/asm/bitops.h b/arch/m68k/include/asm/bitops.h index e984af71df6b..80ee36095905 100644 --- a/arch/m68k/include/asm/bitops.h +++ b/arch/m68k/include/asm/bitops.h @@ -319,6 +319,28 @@ arch___test_and_change_bit(unsigned long nr, volatile unsigned long *addr) return test_and_change_bit(nr, addr); } +static inline bool xor_unlock_is_negative_byte(unsigned long mask, + volatile unsigned long *p) +{ +#ifdef CONFIG_COLDFIRE + __asm__ __volatile__ ("eorl %1, %0" + : "+m" (*p) + : "d" (mask) + : "memory"); + return *p & (1 << 7); +#else + char result; + char *cp = (char *)p + 3; /* m68k is big-endian */ + + __asm__ __volatile__ ("eor.b %1, %2; smi %0" + : "=d" (result) + : "di" (mask), "o" (*cp) + : "memory"); + return result; +#endif +} +#define xor_unlock_is_negative_byte xor_unlock_is_negative_byte + /* * The true 68020 and more advanced processors support the "bfffo" * instruction for finding bits. ColdFire and simple 68000 parts From patchwork Wed Oct 4 16:53:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13409068 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C72DE7C4E2 for ; Wed, 4 Oct 2023 16:53:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 989FB6B026F; Wed, 4 Oct 2023 12:53:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8A2176B0273; Wed, 4 Oct 2023 12:53:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5DD1F6B0274; Wed, 4 Oct 2023 12:53:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 1D0D86B0272 for ; Wed, 4 Oct 2023 12:53:25 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id EBE11A0391 for ; Wed, 4 Oct 2023 16:53:24 +0000 (UTC) X-FDA: 81308374728.17.E6EA004 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf21.hostedemail.com (Postfix) with ESMTP id 48C9A1C001F for ; Wed, 4 Oct 2023 16:53:23 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=LRSyEDwt; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696438403; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vWwsy+BgXOEcfKuqkIEkXjDw+riS2szWYiR2hDnP9b4=; b=2go1D62bftfp256tuXzsR8Jr+c4WJedjpiHLKr/Sx4hJjJ6N8VvPyXxLNtvj8gPvhsaR6Q z8VDE2jMhjfUpl8tcAe1rjbsPaaumr7JRFy2UIZ/8t9Z5tpkL0Y+Ea+qvOwChESrS09VQt 4Wa5RuoSOkDNRjfpOh3KF7hPW9ITHlc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696438403; a=rsa-sha256; cv=none; b=tR+d4GxUuQ0Gi2ZhwT0whEiHcbFMLjm/5QCtZ57sWH6LfEvmOfW9lubRlIUXUVxBcN6swL sEy5iOyv1L7OK1riF+m0kTW7YG7s3Wfb8bXaPGk80W5uGJUUjO/pc9coaOXszAZS7KZad2 4HSYovXdAnZrss3h0rFsp1lGKZ8hHNA= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=LRSyEDwt; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=vWwsy+BgXOEcfKuqkIEkXjDw+riS2szWYiR2hDnP9b4=; b=LRSyEDwtmAXo0SM67p0C/noFmS iQRqwWT0ozrzzklRbep0Y8uPDstZQyqeEmgSEyRLFrVaMAAHGAi6KuiXuL7rxIpp9Wy70RMINXlAc zMlCgxVoBipZVAADPqwuURev5kaILMLePvWQRxlX0JOeMyGqz51FeD55M+cMEekRpNkbU5OhH5Pj9 W3/ibRKkViqP8KeDxEoR6lj+8CrmlJY/CGvO4/2DwLzgk33nshJZZ5xFeCpgyIFC7cP6whN+rIPCb 32UOsNltjlEo+FKhHJ3Ip0qXuHp6b+Nf9dJ3ITrnJDIiNYspgFlpByG8RcOATrn60yzanbbajMwd9 9shECiMw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qo57g-004SFV-1h; Wed, 04 Oct 2023 16:53:20 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, npiggin@gmail.com Subject: [PATCH v2 10/17] mips: Implement xor_unlock_is_negative_byte Date: Wed, 4 Oct 2023 17:53:10 +0100 Message-Id: <20231004165317.1061855-11-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231004165317.1061855-1-willy@infradead.org> References: <20231004165317.1061855-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 48C9A1C001F X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: z4rw5ow189rd6d9ap1631pk47yjhjsxf X-HE-Tag: 1696438403-814850 X-HE-Meta: U2FsdGVkX1++2MgNFpc7m8dzG7jQuJVMZR35JXZnvyaO4HBLtBfk9BkYBAlYs6ukS7gUNeOFnPIUCupczhZB+IqqUk5bswmY57ig1CKBcG5dcpU8bmYC2xWWD7PEwufAX7tcrBcx1We16PlN0+9evX82LlMzmN/6bEuAu8knk3jy+alppSerkQ7TN5qG4jN2SbDFqTvPUtxP4wSRH97z1llElZseN6fq988z/IOXysZobI3q1Hs5Pv+f/tnmAnBvtx+3UJcq/0uKVweZFFLPdzu9jp5l7tbclaqfCZnMA02Rz+WaOyKDibLoD4u/lDzEyZi/THxy3lpi1v97kGF6i9DLtLxQFN+oJmLifSEbH7uzBFeu/IecnU/cpLtuYClN2h7EP6cOSE/dfoAvzSavLuxhp04yXoHVB5d1comuIgO9GK94SMeRykTlJ0cv4jDRzqjbutxOr++JaLO80yUpLPz1t/+wpJhLmiAK4p4Dc130k/nYfqW+oh3IAWiuURzgWDin2K0Y2Vu61HyIeVir1cxxpMoQ7pRg0s2rT79KFlpavSjPMFjCgzNxwS/GmOG4R2qaastlSrP2VZwSHXxhIUPUx8smK7zRsfkaHl/Bjd++KUejpJPFJ9Ok4wsccK3MUWoZYpvftDDlmRboy1AkUa9Dt91eAVsUB7iitvMDii8c0zfuQOugAgPrNAz2JEs7JLqx+KX6HJDitZJuZfypkZATh1m/LqMbyiJCXpM0EVQ18RczdG5xRlbXgkVpmLNwiTb9aQvJ9oxc0DccV0nFHAsdSyXeiIlaEw7vZDHu3x0pGQjpq8rlRsloxj4ntnrkxWzPftS2bGmrkDU5Xh4nte40l2lOh2XwitJcyCmG+I01dcj2QU/WPWRNwBg2f1MpVBmYu8GjHVcOQ0bl2+G21arGw1TYq03O2t/WC8FxzQExBed2KgvODOX+XEIh0BKIjI9zeqG9d1YzaH11c9S YAMW+CsB X3TwSIF3NpKXm97axjBy7asN1dRKA46Bauc5CQO/DTdCpinsIhrEeXcRVyc6cH0WRohH1U79DmgaVGuygSZcYe30ZVtOH83FFfikllybe88WW1DecKuHfZheQypKOuptzcxU2aGvcwmbjB+kwxZ2w6CnL31dgVyfeTtJfHRnaEfCh3nfaNGRuRTmGhwotxRcDY/elEmPP8IvuYHuWKWldn6Ty6fTpHm+RGmVs X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Inspired by the mips test_and_change_bit(), this will surely be more efficient than the generic one defined in filemap.c Signed-off-by: Matthew Wilcox (Oracle) --- arch/mips/include/asm/bitops.h | 26 +++++++++++++++++++++++++- arch/mips/lib/bitops.c | 14 ++++++++++++++ 2 files changed, 39 insertions(+), 1 deletion(-) diff --git a/arch/mips/include/asm/bitops.h b/arch/mips/include/asm/bitops.h index b4bf754f7db3..d98a05c478f4 100644 --- a/arch/mips/include/asm/bitops.h +++ b/arch/mips/include/asm/bitops.h @@ -73,7 +73,8 @@ int __mips_test_and_clear_bit(unsigned long nr, volatile unsigned long *addr); int __mips_test_and_change_bit(unsigned long nr, volatile unsigned long *addr); - +bool __mips_xor_is_negative_byte(unsigned long mask, + volatile unsigned long *addr); /* * set_bit - Atomically set a bit in memory @@ -279,6 +280,29 @@ static inline int test_and_change_bit(unsigned long nr, return res; } +static inline bool xor_unlock_is_negative_byte(unsigned long mask, + volatile unsigned long *p) +{ + unsigned long orig; + bool res; + + smp_mb__before_atomic(); + + if (!kernel_uses_llsc) { + res = __mips_xor_is_negative_byte(mask, p); + } else { + orig = __test_bit_op(*p, "%0", + "xor\t%1, %0, %3", + "ir"(mask)); + res = (orig & BIT(7)) != 0; + } + + smp_llsc_mb(); + + return res; +} +#define xor_unlock_is_negative_byte xor_unlock_is_negative_byte + #undef __bit_op #undef __test_bit_op diff --git a/arch/mips/lib/bitops.c b/arch/mips/lib/bitops.c index 116d0bd8b2ae..00aee98e9d54 100644 --- a/arch/mips/lib/bitops.c +++ b/arch/mips/lib/bitops.c @@ -146,3 +146,17 @@ int __mips_test_and_change_bit(unsigned long nr, volatile unsigned long *addr) return res; } EXPORT_SYMBOL(__mips_test_and_change_bit); + +bool __mips_xor_is_negative_byte(unsigned long mask, + volatile unsigned long *addr) +{ + unsigned long flags; + unsigned long data; + + raw_local_irq_save(flags); + data = *addr; + *addr = data ^ mask; + raw_local_irq_restore(flags); + + return (data & BIT(7)) != 0; +} From patchwork Wed Oct 4 16:53:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13409069 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9F98E7C4E9 for ; Wed, 4 Oct 2023 16:53:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5B9676B0274; Wed, 4 Oct 2023 12:53:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 575226B0276; Wed, 4 Oct 2023 12:53:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4096D6B0279; Wed, 4 Oct 2023 12:53:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 2E1026B0274 for ; Wed, 4 Oct 2023 12:53:28 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id EEE4B40395 for ; Wed, 4 Oct 2023 16:53:27 +0000 (UTC) X-FDA: 81308374854.20.BBB887A Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf07.hostedemail.com (Postfix) with ESMTP id 3BEB84002E for ; Wed, 4 Oct 2023 16:53:26 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=SF1xzTiw; dmarc=none; spf=none (imf07.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696438406; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1WZX9lC/NFReBBEGgdt7ENtP8Fsb2zBQZdLGumoTT54=; b=PWNwT5eW2FgR1s7Rb9isaZUvhIC+6AWU0q7yOa2gpADQh7xYdbDye/A2iCiSWHxpcVhrXl Raz2D77U/gj+3+EdmKJv0hpCHHbN3hA1Z0IVLn92QifJ8YQ5IZvE0ifwqwDx2tx42sGVPx AL0UZm28glAsH43ve7CP2TmflkPUpio= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=SF1xzTiw; dmarc=none; spf=none (imf07.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696438406; a=rsa-sha256; cv=none; b=OPxa3Tptc9g2PhNT50Aq5c256ekyGLRyU640U/g2hw1occ6MlWy/JkIa7VDXRbGtP9geXN CIcCPe2D6yqRQaxYX/nfy0PgiPJ17b3SgPoWQaGGIleu8rGeBuhySLumQdUD4a4uqOQ03c StuGP+5ZqmtxzYtlsxqVcDuI/Xd8SRo= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=1WZX9lC/NFReBBEGgdt7ENtP8Fsb2zBQZdLGumoTT54=; b=SF1xzTiw/J25Aeb62yc9uOpB4+ CSgdaLxXlaa78DG9BEdY53INaBkJUqAr+2EtrSVjWQpYPaKNYd5iaGUJ4ztSk+WCuoWbTjGvaPy/p N7dbKRJGnb44SYp3gdAumx6FN//w89YofZHrNMytiUcrDvEmkfGGnt2lNFsQ5ouYJN/uhULOLJY5+ twshe/tfAlP+OBacdpWeOaATD0uQsnN80U9hR2Gf+p9mUBOlYQ6irsl+Rc8oDL9XpiW65OQs1h7Ag I6tYGX10O0v74mMxVQlkiamubgo3ld8LB2Pcr09LrBiLQTUQZqYJqYfcak4+b/sd7iGmKELvQGaY+ OERNzE5Q==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qo57g-004SFb-7X; Wed, 04 Oct 2023 16:53:20 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, npiggin@gmail.com Subject: [PATCH v2 11/17] powerpc: Implement arch_xor_unlock_is_negative_byte on 32-bit Date: Wed, 4 Oct 2023 17:53:11 +0100 Message-Id: <20231004165317.1061855-12-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231004165317.1061855-1-willy@infradead.org> References: <20231004165317.1061855-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 3BEB84002E X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 7khjpxtzwir3hyzw7unajjq6d5wo1eyt X-HE-Tag: 1696438406-834356 X-HE-Meta: U2FsdGVkX19q5eNLvKCYImuJU7ukR/2jSFXjpukAqfMctMr7zaG3lz+/pN2ERgnowCuvR2dh7pwS9yNpfBB6JHr+E2svd6u58ygDWeoCVLWtYJIY6kg1EFlofQ9DrMdrEIQW0IxS5OvBjq04BixH3qF9lEp5adlWGG6zdcXmJDo04kl2q/AM/foinlMRyOG47rav6zMb4tXTFgZWZplA0vDM4I8lsjTVfb5m+zCrJk58asRs6SWPe49gCCSyDT1j2jq8PtrYVOYwDn33IzgcZmBsMrmRMyY1CmNuk1kbegz6mEXkb2TK7gxzMDawNJTa1r6rEZC2xqk32ItoPAL0FIYQgz/VUIppVFwcAre9XsMvAoURSYwl9DEwRA4s8iyWyYxpo03xEb9RWdbU5Ep337e6u1B7xDfN3w8W5wbDCXeQaFKEC3oD/gLiIBQk/mBLv5MTA96HU6redux/IcCgUMnhyu03iXtv0ajemMGAGh0UKtyBRqQgQmAFHKXYgbKTN8k7ZY8xiFToeYA8fsBWahBT5s7/8MecHqCZACdAu5+BQ4bCE+P6gLt66cVeKUOyjabriVZ2uFKWVp3sy1D+177N53kKJIw1hDgQO9UEnq0hnDrhWDwpGKXs2BX5YS6OBiUINlJIIo+6g6c8s3WYdBbEpBnPVpgxJTkMtt7Qk/3aMsayqtun5ulOcFQzXWEr5evxLrSCcmFPUjKeyidCXuJPqLDJhwjSNIAJD0tvr1/WUbDBmArxEi9T4tvR3DPqd51r+klRDOy9gMnht9KtZCPFPBEvjTh6sEblzc6yIcLmjYfqxcly4tel0nga4HfFJLzr3RV+AMi4wJ3om6LuHkL+gR9eZLnWuTGj1yxWlazaHzvBXcGYJs46nZpzzeFv10YoBiQVGRQzyCRjnRVd0r2nRLeoj9FMakmNrcJx3bo1gDyyx/X/vf9sfRGJE9SQ3BMIBIIFwwauWYOrr25 KPwKrbFQ DaHF1BldrXjHiR2iW03lDyzBNdesf1IgSh330gZabjW3SWrE1mLIueVT1dghuikVNeMcXPQkxYpaSKfrcW78aUPBlPLy30OK7L5KDkxAxi5zqh8dfb70wCQiK3/rYcwES9KL2e6H5IUj+UrbZUqrY3JJa9CYS/KKLn2H6nyadv1slyKENchPIQAGCVo3kwI9MGmimKuIYza7psU792wAjuCvmOXx/Q3uMv+Tu X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Simply remove the ifdef. The assembly is identical to that in the non-optimised case of test_and_clear_bits() on PPC32, and it's not clear to me how the PPC32 optimisation works, nor whether it would work for arch_xor_unlock_is_negative_byte(). If that optimisation would work, someone can implement it later, but this is more efficient than the implementation in filemap.c. Signed-off-by: Matthew Wilcox (Oracle) --- arch/powerpc/include/asm/bitops.h | 4 ---- 1 file changed, 4 deletions(-) diff --git a/arch/powerpc/include/asm/bitops.h b/arch/powerpc/include/asm/bitops.h index 40cc3ded60cb..671ecc6711e3 100644 --- a/arch/powerpc/include/asm/bitops.h +++ b/arch/powerpc/include/asm/bitops.h @@ -233,7 +233,6 @@ static inline int arch_test_and_change_bit(unsigned long nr, return test_and_change_bits(BIT_MASK(nr), addr + BIT_WORD(nr)) != 0; } -#ifdef CONFIG_PPC64 static inline bool arch_xor_unlock_is_negative_byte(unsigned long mask, volatile unsigned long *p) { @@ -251,11 +250,8 @@ static inline bool arch_xor_unlock_is_negative_byte(unsigned long mask, return (old & BIT_MASK(7)) != 0; } - #define arch_xor_unlock_is_negative_byte arch_xor_unlock_is_negative_byte -#endif /* CONFIG_PPC64 */ - #include static inline void arch___clear_bit_unlock(int nr, volatile unsigned long *addr) From patchwork Wed Oct 4 16:53:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13409070 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23E1FE7C4F8 for ; Wed, 4 Oct 2023 16:53:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 347EF6B0273; Wed, 4 Oct 2023 12:53:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2F7DC6B0274; Wed, 4 Oct 2023 12:53:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1980D6B0275; Wed, 4 Oct 2023 12:53:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 0ABCC6B0273 for ; Wed, 4 Oct 2023 12:53:27 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id DCAC11403BC for ; Wed, 4 Oct 2023 16:53:26 +0000 (UTC) X-FDA: 81308374812.28.D244C21 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf21.hostedemail.com (Postfix) with ESMTP id 1D3731C0013 for ; Wed, 4 Oct 2023 16:53:24 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=VOk2evtN; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696438405; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aiOK8Ht5TYsSZN2/F06YkBbUJBHTCETngx448aYGRJo=; b=xcUqXTAF4mZnkiJSqTS9gidP3o6kKfL93En2nIxOvW043KWcZNUzN+bXApj45ViBzbUkFd /vtdNdWlwcVa3Xn/Qme5htufOcUa4r68AaIr7xfBCD6MPKLOPrqGE2amYfCRkLTp50stBB 1chzRXgJnX86KqY2Pn88fUVuwA65Xb0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696438405; a=rsa-sha256; cv=none; b=ZSDtccV57oO2yyPQM/6dmfzPwO8PYHaKxj2qgbC1VoHH6nPjt7qX1VPxrl2rGDoA5B4tu0 vbl9gX2UgdoSH6yk1/POzpfyLcyZ2R03u1NROBFJ9+x2rMts18BJ7Egd4RnYOU9xgMY0I+ mrw0A/k77+gW1nx8umpRqXYCGRW5ynQ= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=VOk2evtN; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=aiOK8Ht5TYsSZN2/F06YkBbUJBHTCETngx448aYGRJo=; b=VOk2evtNHuqg7ndh/VFc13jshQ 7Aq6JXqhdYoVvDjQrEneZ41ZwtOlizK9UoWsy42xy8NW3bUCNb3FM1Lnu/ipD7yoFi3JwlBWmBL9x QzfQgpXzrCBdr//gmxpCSLCzvgrpfK096UuAuYhHfitjA2uMljoQgeqWetGTlvfEYKoWf6dBMtmqN FNDJJdV9A88mGDAmIf14M2txfGBLtO4K6EU7at4YJB+4ni1EBNFCm+6C2/joXW2lKFFXN3w7Sbn/L 1o6SwpwpoUxRpJUIeguhYRCDOyQgh9vvfjUqn/3PeR8zhwrl4RRlWzioFMBcJWIp8FWcO/PedaqLM NGcjEprg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qo57g-004SFl-Cg; Wed, 04 Oct 2023 16:53:20 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, npiggin@gmail.com Subject: [PATCH v2 12/17] riscv: Implement xor_unlock_is_negative_byte Date: Wed, 4 Oct 2023 17:53:12 +0100 Message-Id: <20231004165317.1061855-13-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231004165317.1061855-1-willy@infradead.org> References: <20231004165317.1061855-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 1D3731C0013 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: qfai3awehk7grfck8anqbhrriowemsyb X-HE-Tag: 1696438404-183070 X-HE-Meta: U2FsdGVkX1/DJwaIWU1ZSjBR/hqu9k7lJuvbCciDNZWFlmxWnYB/lVOjTjw2luivb7A7zp3WDJVQ1RyW9I+XMRplg2iKmALCUP5DPSgMplPJdV6vNypF2BmKc3vOToxvlG7oY71C6DsDB6DmJAG6vCDSi4DJ/YgUP/XGAngbOLMY+i9Hb+eudrS7jgmmYZ8VAB7wzgBNCA8Hk/6OeX2Cj13VlQcWI8UB+XnuhTHVLrMs+nKuN/9Hr99TdV9Py44ZRoduMfXW9ZNJ0AQyjv5/TpVsYxVfyl37zbmgRcgz3DnBusXOpVU7ujaVqxwhe8vtThe+efRViZz7seeStCTPMOLBHkdyi2I1juaT0zUEpMROnOLP9i/U7OttcqvqhVc8SKcfLGpErCxoOna+vFMMMnH8pbkR7AbCnfnnfxmDOQY3KB8VZGMFmNeZwyhC/L0KYoq6oXbCphDgV8TcWd/ZWkG+Uf9L68JnR3XedBG+oKW4HheLZiwnuaYLR+JaL58muifuAfoWy24wepeJxlE8TXhjRavdAKp3K0PAHpM2Y/MIn2T1BNEsgVToNzBWpRFdMqPRysfuHbYVy/K5Ey3tSkN2zU5nzRGXq3+edz9mzZv0ojptZ5TuZaMhgJh6QGoYd0QYMpL4Am0I11b/4K2pBIpXp/M4TJ7m9jQ0ovKLKo7s+I502YiNQ/Su3WjbwWIC3fMAtk++B73Z3hY2/U4PE+8UjGzGQ/KT62rfMym/507JGu6tXn1KtaxRmUfFFQk9IYR1fDWmSSVM2vnRZ0Ity2HxrbAz87fdvzb/D6kO+dvWQYbLjm7PHGiEbyFdgYPM5lh/pK9SKUwmCe5AdTwaGpH9BXh/cAYRBClmM9fQtN1LWGh/GyE9ivZoqin3IVnCknvrEReuUNBgj8LtX6xnX10mW1M+5MHbg0pEv7lD2UgFiuucp6rp1xZwjNQ2I7m6ahxkTijJkD/PASQPUN8 9+JVIVmZ Pkucl9W9RZ0uCn9FF472GrT/lTV90ECMZR6+4+uUTu0rXv/fJkIkNM3hH7xipLLHuiEuOi5yiGrwLIAcHqj/PwYRXK8/QrwfflYHiA6WnDd7Py2Hq5NogbyXNlFwxL5DCNnWDAzSM+y8LWhxslHzTJZh9mVR+Km1gXcF9tdlWEzjDUKsfyHs5/oQOhhtWe42Cu5SjGlqkW6udOMi/21LGY6w2FiX/Ua1TmUHc X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Inspired by the riscv clear_bit_unlock(), this will surely be more efficient than the generic one defined in filemap.c. Signed-off-by: Matthew Wilcox (Oracle) --- arch/riscv/include/asm/bitops.h | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/arch/riscv/include/asm/bitops.h b/arch/riscv/include/asm/bitops.h index 3540b690944b..15e3044298a2 100644 --- a/arch/riscv/include/asm/bitops.h +++ b/arch/riscv/include/asm/bitops.h @@ -191,6 +191,19 @@ static inline void __clear_bit_unlock( clear_bit_unlock(nr, addr); } +static inline bool xor_unlock_is_negative_byte(unsigned long mask, + volatile unsigned long *addr) +{ + unsigned long res; + __asm__ __volatile__ ( + __AMO(xor) ".rl %0, %2, %1" + : "=r" (res), "+A" (*addr) + : "r" (__NOP(mask)) + : "memory"); + return (res & BIT(7)) != 0; +} +#define xor_unlock_is_negative_byte xor_unlock_is_negative_byte + #undef __test_and_op_bit #undef __op_bit #undef __NOP From patchwork Wed Oct 4 16:53:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13409073 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65C29E7C4E3 for ; Wed, 4 Oct 2023 16:53:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 45A6E6B027A; Wed, 4 Oct 2023 12:53:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 36CE86B027D; Wed, 4 Oct 2023 12:53:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1BFD86B027F; Wed, 4 Oct 2023 12:53:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 05A266B027A for ; Wed, 4 Oct 2023 12:53:30 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id CF1D3C0384 for ; Wed, 4 Oct 2023 16:53:29 +0000 (UTC) X-FDA: 81308374938.02.5B78204 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf07.hostedemail.com (Postfix) with ESMTP id 22BCA4001C for ; Wed, 4 Oct 2023 16:53:27 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=m5I9BXtx; dmarc=none; spf=none (imf07.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696438408; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NoYAPE+UO7xNrTZ3UFo3zibiwdFupzttBoXxylMj4Qw=; b=ra/gCUB2VKgqBqcdjise4DjJcM+vXm1TtNZ+QN8o0zvrokFZjnz/ccv/73udmo+lCJm2tA B5DCkiKAk6gcTsK7+yDzE7cklO6ogEieh1EneCRsZR2fPr6yBVbTJ7wopFq+YXumym9rwm kBddFvnHu0sJCX8y5yIJkOFIizLuzSg= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=m5I9BXtx; dmarc=none; spf=none (imf07.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696438408; a=rsa-sha256; cv=none; b=SJDtB0P1BLa9BY7+4Hi7bIWg3ndq5dyocy7iAkRjrJ8aZiQIo7lMGZwstCKo2oeAyRvZ0j ONTXQCCfaBCy8xDdGrUO6wHR+vBoCEB03GkW5AV+OTdvbd2PsR1LdHLA8bxNAyooPkmoK9 1do//EwaNnURbUVLTAYLTXvzsHKYGf0= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=NoYAPE+UO7xNrTZ3UFo3zibiwdFupzttBoXxylMj4Qw=; b=m5I9BXtxa7U3y+DGVFJFvAMKxg hBD+0YDc+5glcTSqGwZdiU6CHF8GGs8Fwr1IB1Q2wLElzB2gxr6IE3VibFZ3rtjCAnHgyMSEkvmjU COtxOWuztoH0qghXux20Z9rq0nyzLVraO5yLavoMMcks/CORStaKbNRgugrAJSXdkEYWPPt4JRFyR ghB03Xdi8DyFDINsJpQsA0Bku5Pz6tB5JxfpIW8ML04CpMn1Q2Dtbi+zN2rpVwnpXrGBU4/x9CWYK 8hBRdb6Pu+yl6r3m/UR7Sw1fVNiY+4PzLfApV1j+H0pPF0Rmvec00gSHlMcMfNnmhTfeRbn7642xQ NaBY1K5A==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qo57g-004SFr-Gi; Wed, 04 Oct 2023 16:53:20 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, npiggin@gmail.com Subject: [PATCH v2 13/17] s390: Implement arch_xor_unlock_is_negative_byte Date: Wed, 4 Oct 2023 17:53:13 +0100 Message-Id: <20231004165317.1061855-14-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231004165317.1061855-1-willy@infradead.org> References: <20231004165317.1061855-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 22BCA4001C X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 947ai8i5wwrma14s3qfzdi7weyose4uu X-HE-Tag: 1696438407-754494 X-HE-Meta: U2FsdGVkX1+dh7lg+NPQe3kQQpVefj0Jvm0YmzE/ZbV4gLuduQpEIHy2cm7rWeucP4VFfa3hYrOCP9DX41hv1IncSaL4bJLS/Vm6fyvT49Ye+UROvO22w2P4z/wxR9Gueqpj2PhQhbTR/A4QVYu9CLjzXcYZrna144IDdeOqmnQml0zEsac0Ucav58ayY4Z94YW0rJa80SNUdX9nH1ErrZaEM0zbr93WFy9itsbS/wOB+ErSx8mBf0mXOtZFdrHIILpqAbf92pcOSoIwwWgkQS5POwAJAMV8XzZUcbSijsqfZlkcDHQMs4KeAtNrvE0KqreQKpr6eQVFJipJ56JOS5xCMjqL5yb4LMplfKM0RmbBSfGCTMsq5yEDngR/IrE1OaOL1qqojcopXoH6FpzDT06KoSnuz4bDZDH3KCfR4JL69NjzpEHishKum0cYRfPX1HUfkPOITBPCqbuUmOAeEiFrzaPaA/qu8um1sNUOHx/LWcKYQ6VNtzA1Bs1Bz8powT3CV4dreUSWr2a/wCQgCJoxvLGGLh3LBUTKCmt4+IPTcKK+KbKzUMFLbf/7ukKM6BC4zpuOjEVsKwdTgHM3wzs9fygi0mq82f+ful/RzqN0nzTj+nyLnMjEysB2zrFmunqyr8ExCCrsKFCR8+nOzzODPNUjycbqlOEOvosL/QNXYk23lh+B9FZ6mYbh//DzUAHFzPd2xJAzZRNa4gUyxq2N+irw/AXLe26gdNB+ta1sLQ8y1ObWDrHf6f1ReFdoI0fvNOcJ7HXA9MTS7ZDYPAvG6SZatOAMH8L2Qae3p/qFLi309i03+PIliYsnnIiOFYI3F1DyvEoBZOFf28vA3h0aHGtVZm2fQX+kN2+B/h8nlrAChJLjZsMMpxBJjStB/KEZa50IokJMC48MbHdjIwproS/2BkNDmEqRy9nNKKVDcIgTz3dp30OOMsDmBeHhwVkxlW17YBb2eg5htAE ELgECH/y A2aXaS7jZMjFUsn1e1B3tmFt3tHRekyGOzeG/FC/+TTdAToMGMNaYFK87vYtIVTIGDCT32dibINTmdgU7/FQKhJYedptiC/J6JP1R5aGHNICMKhga9Z/zHI1ZmWdHjyXjEB1q3ynBNQf5KB26SzD8F8btvKmzoIqgly6D0G9LNnHTEdwL65gYUeAElTLLIyFa1ot7QCU6+WtNJj78D0E1oNDOrw1co0OtxufzSUR2mQGfRc4xLQo0O2KG5adMQSGwcz8RDAtGWxjDM4w= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Inspired by the s390 arch_test_and_clear_bit(), this will surely be more efficient than the generic one defined in filemap.c. Signed-off-by: Matthew Wilcox (Oracle) --- arch/s390/include/asm/bitops.h | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/arch/s390/include/asm/bitops.h b/arch/s390/include/asm/bitops.h index 2de74fcd0578..c467dffa8c12 100644 --- a/arch/s390/include/asm/bitops.h +++ b/arch/s390/include/asm/bitops.h @@ -201,6 +201,16 @@ static inline void arch___clear_bit_unlock(unsigned long nr, arch___clear_bit(nr, ptr); } +static inline bool arch_xor_unlock_is_negative_byte(unsigned long mask, + volatile unsigned long *ptr) +{ + unsigned long old; + + old = __atomic64_xor_barrier(mask, (long *)ptr); + return old & BIT(7); +} +#define arch_xor_unlock_is_negative_byte arch_xor_unlock_is_negative_byte + #include #include #include From patchwork Wed Oct 4 16:53:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13409072 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FEACE7C4E5 for ; Wed, 4 Oct 2023 16:53:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 322A76B0276; Wed, 4 Oct 2023 12:53:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 283506B0279; Wed, 4 Oct 2023 12:53:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0D5106B027A; Wed, 4 Oct 2023 12:53:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id F1A2E6B0276 for ; Wed, 4 Oct 2023 12:53:28 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id C44161CA3C9 for ; Wed, 4 Oct 2023 16:53:28 +0000 (UTC) X-FDA: 81308374896.09.2232C9B Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf21.hostedemail.com (Postfix) with ESMTP id 0D2FB1C0011 for ; Wed, 4 Oct 2023 16:53:26 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=jIVsI8tP; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696438407; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=i/THylbIgEiK5ujFChvbIlJFdxL/ilDxv/jfyb9enPw=; b=O/AABAxuJe3Am1nPybaiVdEbN2NPOamxBa8txZnV86ZqDFw/8BEdc+V14ntA1WdsPFCtKT nVU/p28nC4xiIIS6N+1Mct5/87NI6q56gV/17hJVkyXeXmTomfjgN9+pUq3M8rCGT7dwjt BS241S3Y+955l24gEiXeI4D3+kL25dE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696438407; a=rsa-sha256; cv=none; b=HZaf3yRJY74JacGgEbVdyU9tXtLpH2LalPFI/Wc/ySlmUumNyrcE7KYzAwBeP3kuwXS8oP vgnPvkotTCI+icY5fgI7XL5HdrqPu60EFCXD749oZHGrKls9znqLuI4T3sFYRuVy8+cNmX UWbVPmQHmpnFrnqA7Ok+mYaFCUB7+wM= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=jIVsI8tP; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=i/THylbIgEiK5ujFChvbIlJFdxL/ilDxv/jfyb9enPw=; b=jIVsI8tPAzH0SuNZpA70UkD5li 2ws2ktmfhTETzp44zx30Rj9cF7rmj9iSam52cmLtD0fu7LWyO6J9MfhW/ItU/0mYyQpkG/GaFTOiS kB32jO5aGAKCM3OKMlg5DQTcWcRHrCko9YH2S0INbvfuU4UuxB4qhmD3ltPfXcRUZqoMw38oHnZl2 2wDQWLhiLgfIjTK50jjihtxELCyWi0fWlirRWojikdn1mVv4Qt9HX9LaSE5beZZ4O68691U7CLC4l d3rZmrFJKUFhTUi/qSlmzAYNR4Hwxiwqzn8aAmQjfq8wm1VaHjq1rWwaIMLyICz0IcHYiZvN3jNze 9SjzPzXg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qo57g-004SFx-KW; Wed, 04 Oct 2023 16:53:20 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, npiggin@gmail.com Subject: [PATCH v2 14/17] mm: Delete checks for xor_unlock_is_negative_byte() Date: Wed, 4 Oct 2023 17:53:14 +0100 Message-Id: <20231004165317.1061855-15-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231004165317.1061855-1-willy@infradead.org> References: <20231004165317.1061855-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 0D2FB1C0011 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: q9ik98c549zkdtinkzobrqwg61ummne1 X-HE-Tag: 1696438406-8321 X-HE-Meta: U2FsdGVkX1/Q2P5w7K1TOHdpyMTQ9uKWi3MQhIrxc+8nseD6vagEtCCxJZKv39tw54Qr1ldrMo6sXQQMSu+YUEhGDCv8yEiC8B01EiojG6cqJ4mGGWTb97wuzfQxXA8GyIm/WmmVbJsgrZ7p55NcIV1rCp8BQ6jl3VhXZULHRsw1+F34y4ieVL3IWIYDJVWTYVNBRvsAdI+v9uq3BPq5Zq6EBEBkTYYs+vpo2y7YShNOhbe8BMKuR7KBoGrxGmmxzfvRDlEbH7PJHupLmijJh16UZUXoJzOvm6kPgRjf5RgMJN+p8zhSi1KUA0ZFybNj52Aclzk6dlHycdnVLePmhvPmyKGl1JSKZlYt1kS2DqYiUsZPhTKIqcCRl/wjstLAt0RZa3TDt9IyeSZvYIetujYlEL+QmgpbSX+6+SyMk0x4lnTuY0Wfu9dhteBiVzRaEhLfTgQyaUeq+n7xzcqNl6y7Ge3H3amk8Epy/0L8IIdMAAUq83L05lArw+Di3vrBIwfVQA9+SMwQeGMs6OIp6/pdlHLeLYTWheJpmWRMcI/OFbLiGQUiNkCjh77YSUcwIfczPoRZOnby+s5iIWkpomlhP5pUUW4XoGYs0w9GZaOiRrgZCHS4eaD9Ubz8FkVM0iXy7EZ4eUdZ8zRcYoKBVBHOSa81kPff3gIWY6g+JkAO6zbGFiWYpkhfcqcnQx/uHnIQbgHhIVzeL7bTgoa7TKlLE4G0qd+E8tvjBTmYHtq8C74X03mtPRkfWodhFjewDQgtSJqfC/I8aAO9zAw83Pq/Xi3m0c1PhUMTFenZWT08cipDDyzioiBFtbZmQahS8fH+qSeyLdEjKCbjtdPGxDoz3TDX5KFiswhcO9NtysfWWV8GXNeQkA0Lb/jFJZP775sXUEXAx3Rhu+Lx822yJOX2kBoj0ZTw8mTqBmYLeC050K0FnvKkFuMWB6xRWQanJt0BUTCiFC+TN5nmjQq seagG4zH EJFUKHnYiWMyqPehhdtc5Zv4L/ecu9hKzJ12zlv6PFXNXI9QqxSyFj/Uppt2z6yW+yochtW2naAxh6VeXHUailQuApPqxC8EwOhlNx2C2OAwLFtgvIuilDIylue/xKb/+N6TmF0uH9Q0Ca5iOyeL/X9fAG80j6FJTYGu7i87TLflKIVnW1fXM7RcVPYr37jmzJyZ5iPXXJkdMFTPuXKlf5dtIwEJ6n5LJUwjo X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Architectures which don't define their own use the one in asm-generic/bitops/lock.h. Get rid of all the ifdefs around "maybe we don't have it". Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Geert Uytterhoeven --- arch/alpha/include/asm/bitops.h | 1 - arch/m68k/include/asm/bitops.h | 1 - arch/mips/include/asm/bitops.h | 1 - arch/riscv/include/asm/bitops.h | 1 - .../asm-generic/bitops/instrumented-lock.h | 5 ---- include/asm-generic/bitops/lock.h | 1 - kernel/kcsan/kcsan_test.c | 3 -- kernel/kcsan/selftest.c | 3 -- mm/filemap.c | 30 +------------------ mm/kasan/kasan_test.c | 3 -- 10 files changed, 1 insertion(+), 48 deletions(-) diff --git a/arch/alpha/include/asm/bitops.h b/arch/alpha/include/asm/bitops.h index b50ad6b83e85..3e33621922c3 100644 --- a/arch/alpha/include/asm/bitops.h +++ b/arch/alpha/include/asm/bitops.h @@ -305,7 +305,6 @@ static inline bool xor_unlock_is_negative_byte(unsigned long mask, return (old & BIT(7)) != 0; } -#define xor_unlock_is_negative_byte xor_unlock_is_negative_byte /* * ffz = Find First Zero in word. Undefined if no zero exists, diff --git a/arch/m68k/include/asm/bitops.h b/arch/m68k/include/asm/bitops.h index 80ee36095905..14c64a6f1217 100644 --- a/arch/m68k/include/asm/bitops.h +++ b/arch/m68k/include/asm/bitops.h @@ -339,7 +339,6 @@ static inline bool xor_unlock_is_negative_byte(unsigned long mask, return result; #endif } -#define xor_unlock_is_negative_byte xor_unlock_is_negative_byte /* * The true 68020 and more advanced processors support the "bfffo" diff --git a/arch/mips/include/asm/bitops.h b/arch/mips/include/asm/bitops.h index d98a05c478f4..89f73d1a4ea4 100644 --- a/arch/mips/include/asm/bitops.h +++ b/arch/mips/include/asm/bitops.h @@ -301,7 +301,6 @@ static inline bool xor_unlock_is_negative_byte(unsigned long mask, return res; } -#define xor_unlock_is_negative_byte xor_unlock_is_negative_byte #undef __bit_op #undef __test_bit_op diff --git a/arch/riscv/include/asm/bitops.h b/arch/riscv/include/asm/bitops.h index 15e3044298a2..65f6eee4ab8d 100644 --- a/arch/riscv/include/asm/bitops.h +++ b/arch/riscv/include/asm/bitops.h @@ -202,7 +202,6 @@ static inline bool xor_unlock_is_negative_byte(unsigned long mask, : "memory"); return (res & BIT(7)) != 0; } -#define xor_unlock_is_negative_byte xor_unlock_is_negative_byte #undef __test_and_op_bit #undef __op_bit diff --git a/include/asm-generic/bitops/instrumented-lock.h b/include/asm-generic/bitops/instrumented-lock.h index e8ea3aeda9a9..542d3727ee4e 100644 --- a/include/asm-generic/bitops/instrumented-lock.h +++ b/include/asm-generic/bitops/instrumented-lock.h @@ -58,7 +58,6 @@ static inline bool test_and_set_bit_lock(long nr, volatile unsigned long *addr) return arch_test_and_set_bit_lock(nr, addr); } -#if defined(arch_xor_unlock_is_negative_byte) /** * xor_unlock_is_negative_byte - XOR a single byte in memory and test if * it is negative, for unlock. @@ -80,8 +79,4 @@ static inline bool xor_unlock_is_negative_byte(unsigned long mask, instrument_atomic_write(addr, sizeof(long)); return arch_xor_unlock_is_negative_byte(mask, addr); } -/* Let everybody know we have it. */ -#define xor_unlock_is_negative_byte xor_unlock_is_negative_byte -#endif - #endif /* _ASM_GENERIC_BITOPS_INSTRUMENTED_LOCK_H */ diff --git a/include/asm-generic/bitops/lock.h b/include/asm-generic/bitops/lock.h index 6a638e89d130..14d4ec8c5152 100644 --- a/include/asm-generic/bitops/lock.h +++ b/include/asm-generic/bitops/lock.h @@ -75,7 +75,6 @@ static inline bool arch_xor_unlock_is_negative_byte(unsigned long mask, old = raw_atomic_long_fetch_xor_release(mask, (atomic_long_t *)p); return !!(old & BIT(7)); } -#define arch_xor_unlock_is_negative_byte arch_xor_unlock_is_negative_byte #endif #include diff --git a/kernel/kcsan/kcsan_test.c b/kernel/kcsan/kcsan_test.c index 1333d23ac4ef..015586217875 100644 --- a/kernel/kcsan/kcsan_test.c +++ b/kernel/kcsan/kcsan_test.c @@ -699,12 +699,9 @@ static void test_barrier_nothreads(struct kunit *test) KCSAN_EXPECT_RW_BARRIER(spin_unlock(&test_spinlock), true); KCSAN_EXPECT_RW_BARRIER(mutex_lock(&test_mutex), false); KCSAN_EXPECT_RW_BARRIER(mutex_unlock(&test_mutex), true); - -#ifdef xor_unlock_is_negative_byte KCSAN_EXPECT_READ_BARRIER(xor_unlock_is_negative_byte(1, &test_var), true); KCSAN_EXPECT_WRITE_BARRIER(xor_unlock_is_negative_byte(1, &test_var), true); KCSAN_EXPECT_RW_BARRIER(xor_unlock_is_negative_byte(1, &test_var), true); -#endif kcsan_nestable_atomic_end(); } diff --git a/kernel/kcsan/selftest.c b/kernel/kcsan/selftest.c index 619be7417420..84a1200271af 100644 --- a/kernel/kcsan/selftest.c +++ b/kernel/kcsan/selftest.c @@ -227,12 +227,9 @@ static bool __init test_barrier(void) KCSAN_CHECK_RW_BARRIER(arch_spin_unlock(&arch_spinlock)); spin_lock(&test_spinlock); KCSAN_CHECK_RW_BARRIER(spin_unlock(&test_spinlock)); - -#ifdef xor_unlock_is_negative_byte KCSAN_CHECK_RW_BARRIER(xor_unlock_is_negative_byte(1, &test_var)); KCSAN_CHECK_READ_BARRIER(xor_unlock_is_negative_byte(1, &test_var)); KCSAN_CHECK_WRITE_BARRIER(xor_unlock_is_negative_byte(1, &test_var)); -#endif kcsan_nestable_atomic_end(); return ret; diff --git a/mm/filemap.c b/mm/filemap.c index 7da465616317..ab8f798eb0af 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1484,34 +1484,6 @@ void folio_add_wait_queue(struct folio *folio, wait_queue_entry_t *waiter) } EXPORT_SYMBOL_GPL(folio_add_wait_queue); -#ifdef xor_unlock_is_negative_byte -#define clear_bit_unlock_is_negative_byte(nr, p) \ - xor_unlock_is_negative_byte(1 << nr, p) -#endif - -#ifndef clear_bit_unlock_is_negative_byte - -/* - * PG_waiters is the high bit in the same byte as PG_lock. - * - * On x86 (and on many other architectures), we can clear PG_lock and - * test the sign bit at the same time. But if the architecture does - * not support that special operation, we just do this all by hand - * instead. - * - * The read of PG_waiters has to be after (or concurrently with) PG_locked - * being cleared, but a memory barrier should be unnecessary since it is - * in the same byte as PG_locked. - */ -static inline bool clear_bit_unlock_is_negative_byte(long nr, volatile void *mem) -{ - clear_bit_unlock(nr, mem); - /* smp_mb__after_atomic(); */ - return test_bit(PG_waiters, mem); -} - -#endif - /** * folio_unlock - Unlock a locked folio. * @folio: The folio. @@ -1527,7 +1499,7 @@ void folio_unlock(struct folio *folio) BUILD_BUG_ON(PG_waiters != 7); BUILD_BUG_ON(PG_locked > 7); VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); - if (clear_bit_unlock_is_negative_byte(PG_locked, folio_flags(folio, 0))) + if (xor_unlock_is_negative_byte(1 << PG_locked, folio_flags(folio, 0))) folio_wake_bit(folio, PG_locked); } EXPORT_SYMBOL(folio_unlock); diff --git a/mm/kasan/kasan_test.c b/mm/kasan/kasan_test.c index c1de091dfc92..b241b57427b1 100644 --- a/mm/kasan/kasan_test.c +++ b/mm/kasan/kasan_test.c @@ -1097,12 +1097,9 @@ static void kasan_bitops_test_and_modify(struct kunit *test, int nr, void *addr) KUNIT_EXPECT_KASAN_FAIL(test, test_and_change_bit(nr, addr)); KUNIT_EXPECT_KASAN_FAIL(test, __test_and_change_bit(nr, addr)); KUNIT_EXPECT_KASAN_FAIL(test, kasan_int_result = test_bit(nr, addr)); - -#if defined(xor_unlock_is_negative_byte) if (nr < 7) KUNIT_EXPECT_KASAN_FAIL(test, kasan_int_result = xor_unlock_is_negative_byte(1 << nr, addr)); -#endif } static void kasan_bitops_generic(struct kunit *test) From patchwork Wed Oct 4 16:53:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13409067 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A91D4E7C4EA for ; Wed, 4 Oct 2023 16:53:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6C9446B0270; Wed, 4 Oct 2023 12:53:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 62C6B6B026F; Wed, 4 Oct 2023 12:53:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4143C6B0273; Wed, 4 Oct 2023 12:53:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 0CE886B0271 for ; Wed, 4 Oct 2023 12:53:25 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id D578240397 for ; Wed, 4 Oct 2023 16:53:24 +0000 (UTC) X-FDA: 81308374728.14.754A69D Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf19.hostedemail.com (Postfix) with ESMTP id 429761A000C for ; Wed, 4 Oct 2023 16:53:21 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=WlYR7SVy; dmarc=none; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696438402; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VjQq5vIRQg04dws7PyaymhwSTjm3kTO8DTKw00f2iiE=; b=Et2cQ66LnRSRzrfKpMX01ZINYBzQxl2xtDxv68A+26Kt3DmxEa8rvklSzc+ZzHxLaKtwE9 h8eoKlp1ZPGqXKRwEwmXQK35plGFfcJJLRBeJl565u/cFkH7UHBEDJzuld9p4tMSDBnz1v TtS8cHUE60aCbic5hJTvd0+TngK0omU= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=WlYR7SVy; dmarc=none; spf=none (imf19.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696438402; a=rsa-sha256; cv=none; b=eAiDJyivr+6ypIFjFap+tIB/4yvSvwIKfRVCBzymnAS+CPimQq6c7ZD/662Lm1MB/aeBcC Mnp+a3DWML7bP0/t9ccl8MIRyNbvSsB3hid8rxJxZu7k5MXUl48h406VElcRbq9OrN1TQm WfXubScCMdj3ZK6rraGXXRXgwp7XIbw= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=VjQq5vIRQg04dws7PyaymhwSTjm3kTO8DTKw00f2iiE=; b=WlYR7SVyJnVNGNohBfF4SgXWK4 eiC2zmNKFtHyl0x9NHoD2Qj1GbLuvT9EfwgRXBH1bL+XUgJojOeePsBlj7eT+NoeJgfEXHAG7GTYQ oEZxQCk9OpT6d54J6R4sPQVYiJF8CjbPIXDfQ5fDNIaV7apCBlDD45D/7PZfXW3GGnYexS6Kergtq y7T4LA9JNaQ10uu2a1zCent9hOjzxoZi1Hsu2yv63P1xWF0B0ohpBTJRPXqYq9fdYxS6Xvw08aGHc 9cYF6pUKSBX41H5rDJac/ghEXLdcGM5d5+ihGxsw7ysvqGc4z7KoWUcGQDspm4pNnmE0KD9tDht/U HUI2ig1A==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qo57g-004SG6-Og; Wed, 04 Oct 2023 16:53:20 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, npiggin@gmail.com Subject: [PATCH v2 15/17] mm: Add folio_xor_flags_has_waiters() Date: Wed, 4 Oct 2023 17:53:15 +0100 Message-Id: <20231004165317.1061855-16-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231004165317.1061855-1-willy@infradead.org> References: <20231004165317.1061855-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: jb46m9m1cd88ihqtxytf4d47jbnbek67 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 429761A000C X-HE-Tag: 1696438401-961857 X-HE-Meta: U2FsdGVkX19GrEtaSfepzY5f0LAUmYX5RDUs3eI64mmxoCxDnxnt+dpbVkSXLasNtAIO7UAI9o1Y/n1t2CUKJbpCPRBpaRd2zLbO7n6DaN/JkKkpdBmFMaDVISJYPVudcbkJyor18SfDzzQB3YJAJbrQ5VVjkp4SUuH5wr6i/3UTvUpYmE4sexleeLLDTwH5M0d+FX5giC3M6V2fK53HVcaz5sePu82QnYaonUSgvPrydgkjdLaMsKp5rRbOtZwHt8hy3af/3OsOvZF/HS9Cj9KK52hJ2ff8IQ/q/da8LuKBLwrC3oO00Zvfim9Crh4M7BEFGRKnxUvEh5c0BPXanni8LCY2ASvWmOnHDXP3YbOvx+1TZfnTBkGcx1hyNew3wI6ahfrj6DEh8LjD6MUvwMdIe7amO/MyNE/c5dTeJQgyNyezxYM8hDCxckNxumi/uM3M50tu1+LYG5HZgRMV1vKvPocRouyuxEPtSklr5qCBsZnBFouh6YnrOTjsH8gVgZeZY3CIkyvpbly2sdm1lTfYb5JgcAEIBaWvS+PKr48GIqCMhVsMkM848nyXLa6Zce/93zHtdLdx3S1aOp55vKq5CyZ3sY0cTKn863RZK1OBv3SPh608KspCkxxNLNVT6j7H4+yiquGe2MoVIzVnzEAhgPCbpXsPMV2ia0WfWKA3NE3QobVa1C1lfvNfU70zwCEu3KcXpIIq7H7x/124TP9obty4Cj6eMFICmTLWLnnJ5KZAxi90jSbWQW6poxx2Ba2RZH24XRd19mnD9MjiRiGwDFzz0s1C5CQIppbiKpaQq75UHDr0Aq6+c994aPdR2z09vtwSrvYoVmqirvlP7GFx9dpBUWEGXJNBNRWkVF0/M1dEgh7qZchxdlJG6gLxaLkpQHshC6DNoVL1ZrcFocuDmifWUIWjKInZUB0Ybl9bcFEh8+uFvNyk7+n5b6910nm2PHjyEO/qlDxZnhZ hXcJsezo XUeSj93+FZTjKIyUOSe6E9Sn4XbAunEW55sBk3A8V783Hz9LdwU6GzlCiPJeVgko4QdNiiW17uLqorQ7VQekLfJwxQRyRQAKg/VE1la4zi+eUuMsdwhA5YEXCv+pfk1m3lu83MvjRuETcG2d7xNHr1Vnj4Sni30/A8aFLaoCFb9kKFG/DPcyGhy1Hjtpez1UhwfKx5JRvGR+4ix30bzdKirHcJkpRVep1VWBpOFsnvk/ySHaIiu1JHLyupUfWb9dEGKdp X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Optimise folio_end_read() by setting the uptodate bit at the same time we clear the unlock bit. This saves at least one memory barrier and one write-after-write hazard. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/page-flags.h | 19 +++++++++++++++++++ mm/filemap.c | 14 +++++++++++--- 2 files changed, 30 insertions(+), 3 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 5c02720c53a5..a88e64acebfe 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -692,6 +692,25 @@ TESTPAGEFLAG_FALSE(Ksm, ksm) u64 stable_page_flags(struct page *page); +/** + * folio_xor_flags_has_waiters - Change some folio flags. + * @folio: The folio. + * @mask: Bits set in this word will be changed. + * + * This must only be used for flags which are changed with the folio + * lock held. For example, it is unsafe to use for PG_dirty as that + * can be set without the folio lock held. It can also only be used + * on flags which are in the range 0-6 as some of the implementations + * only affect those bits. + * + * Return: Whether there are tasks waiting on the folio. + */ +static inline bool folio_xor_flags_has_waiters(struct folio *folio, + unsigned long mask) +{ + return xor_unlock_is_negative_byte(mask, folio_flags(folio, 0)); +} + /** * folio_test_uptodate - Is this folio up to date? * @folio: The folio. diff --git a/mm/filemap.c b/mm/filemap.c index ab8f798eb0af..3dad2615af41 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1499,7 +1499,7 @@ void folio_unlock(struct folio *folio) BUILD_BUG_ON(PG_waiters != 7); BUILD_BUG_ON(PG_locked > 7); VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); - if (xor_unlock_is_negative_byte(1 << PG_locked, folio_flags(folio, 0))) + if (folio_xor_flags_has_waiters(folio, 1 << PG_locked)) folio_wake_bit(folio, PG_locked); } EXPORT_SYMBOL(folio_unlock); @@ -1520,9 +1520,17 @@ EXPORT_SYMBOL(folio_unlock); */ void folio_end_read(struct folio *folio, bool success) { + unsigned long mask = 1 << PG_locked; + + /* Must be in bottom byte for x86 to work */ + BUILD_BUG_ON(PG_uptodate > 7); + VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); + VM_BUG_ON_FOLIO(folio_test_uptodate(folio), folio); + if (likely(success)) - folio_mark_uptodate(folio); - folio_unlock(folio); + mask |= 1 << PG_uptodate; + if (folio_xor_flags_has_waiters(folio, mask)) + folio_wake_bit(folio, PG_locked); } EXPORT_SYMBOL(folio_end_read); From patchwork Wed Oct 4 16:53:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13409065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D1AEE7C4E9 for ; Wed, 4 Oct 2023 16:53:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE0486B026C; Wed, 4 Oct 2023 12:53:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E8F5C6B026E; Wed, 4 Oct 2023 12:53:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D57BA6B0270; Wed, 4 Oct 2023 12:53:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id BAC536B026C for ; Wed, 4 Oct 2023 12:53:24 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 75CBF40397 for ; Wed, 4 Oct 2023 16:53:24 +0000 (UTC) X-FDA: 81308374728.17.A382495 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf03.hostedemail.com (Postfix) with ESMTP id B498220004 for ; Wed, 4 Oct 2023 16:53:22 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=VIQWZrcE; dmarc=none; spf=none (imf03.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696438402; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wx30M72VT+hJbOrAJeEy9IuygKjiZCeaf/wYxmq+w0o=; b=qALXCzi4GxLaMBdblWxHCI91dqOnzwaKJhrDtZCyRQnccnAy7tcDeiOlGjgWWsiekxqr6e xE2dt3Ye595IiWRpM/ijbcncp+XxzgvMpBzAUCmpt87Ryce8dSSBZc/tDmn2F1JQ/2fyyB 9iTHnx2qLYr3zx5kGEtKXvd9vVgpEhw= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=VIQWZrcE; dmarc=none; spf=none (imf03.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696438402; a=rsa-sha256; cv=none; b=cqe02Fh/5L9zBmSYnMkZgF4epIsda2nU3MRLGK1PJvHpr1gAvI+oNp9kUhgqc1r2CkQ/E3 4lS7kHM8O11TuCW07OsRqa4dcI/QqgLFl6kwW2d+DyXjFRnRWWA8tm+iL9OaRFzI+BhogC 8K6FkXUvd4SlPZ4yONi6AsyVyP9jChA= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=wx30M72VT+hJbOrAJeEy9IuygKjiZCeaf/wYxmq+w0o=; b=VIQWZrcE6oEY3co/PM//ME4nfa BtOcRyfUcLSgcqVeWTd40a6YYpq9zruZ2EDbrJbmBafh8zsH7eeVm0d385zTV7lvkjwYpIuH1gXmc /7xMagbzbtAlndSsVr7ws4+bR9mH8HrlwlIaqPukYXoRXMB7076uS4Oqi+3nabQ9AQNj6vDba2iYI XHcX5iFBH8JvdaVdpO0L18ek+zaY3OlnI35kE36eSbs96v7DxPJ62EXK++7d2ng/dqibc/kOKzwGd +Tn+IMnQYAyWHEnPgL4SxTUd1C1Sc0g2Gjfl2etMmn4Ex8NFsX3RKo2oo+ZBxXlqrqnJCJTi8B3WE 9n78tXEg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qo57g-004SGG-Sx; Wed, 04 Oct 2023 16:53:20 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, npiggin@gmail.com Subject: [PATCH v2 16/17] mm: Make __end_folio_writeback() return void Date: Wed, 4 Oct 2023 17:53:16 +0100 Message-Id: <20231004165317.1061855-17-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231004165317.1061855-1-willy@infradead.org> References: <20231004165317.1061855-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: B498220004 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: t13qh59tgpfrm99orm8qgpb5xe3och41 X-HE-Tag: 1696438402-318256 X-HE-Meta: U2FsdGVkX187Z9gXrEbjRsddno4fXa9ipLzitmB0MhAAA/pKVQ9BvynioP86apoBUwd939PNzyoP6BwyHZsc47L77RKwXcHPEaesSC0QhJPgxthbBLm6jTg1d2tfLGsaIo6Oxg84lGsC3m0MNr0i684wVIeEE0kiC6H3IDgP3iVzFwZoQvRkpfM6fB68jZx57uU6HjEcskiuEKRBwENyxeqItiFecuZXPMwQbHsRApsr4Tl7rG7Fxen3zRsy8MJskklFbs17IpARsATRG9nLKYCEeRxkYRIok9UzQzTrUqbQZUPf3ihcJF7kMTVDOWaVJ76qQRd5KXL0zygS8zXfXQFM6KuC7FKwqk3H+vBDNsCL1Kdkvf4zPG6jGqCHxLTK2RlvvJD4oGoFqyKbePizV2CwjsUdiV5cE0QOIREmO5MRORA+BFWdUbrQ+3JkaixTAZCn3vLgjBcdcM4sQ9Go/UHtkw89JBGtcBg0IR6/wjTbJZ7W8VbPGPEwgPSEdxRMkbtxqlXaoWyhurB5RKqa7xPKwk5DrWIYT/WIrozp9lp4caRH9L3k51wDqFhSoOwXdrW++z0vBRYo4SbrxcffzqHo3LX5LJ+ec4M288tDVckNMvAGDBkgP6KNNOyrf1rYsf3KcJK2MZiekQ9Aax62ACez2TwWfZlOn1vOaSxS5q/rAsJ6Wid2sM1BK3ACVOKLlzJ5ZAiBxIGRvKh/QRXuO+v45GBvMS9hT9zwJ26ymCkwbkw2krG9vTkS3Wuzv3SQXWrdh+jacJqlM4Ir3arrH1QRl4pq/f21UcLcVz7duMZ3NSOsuMljccTSzPIqXb4m8Z+6dS+QqxDTz2r/fYEclXQ8/woIu3yrV7KYpOnz3hXdx7Iz+GMBlASm0CtCQlOUgTH43jmV8qX/DakPbkOAT9xQQK+uEP9GQE1AKG12z0/+FwfE+znPxgiAAWUVQRYZGlajMZNgNwa1/E8R2QB HAFRazVI Pn+tSykBkMYMaV09LgGzek61D7eoeubQDJHocjgzGEwiUwVfS+ckUJObUysJqRifBd9+KggTirVO06bNfEvoqGC4Frb75vgQLPxz/CmauZQ8c52AbBt3/0ol3+Qw45vktAqn/BJULOBzJajRMA9isGpdwy93DeOuevMcWk4n7sgzhb0z5D1mlTyYEkaD4ryzfqE7pipruiHa5vuCMTC4DmWXrJQb00bWTPrJE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Rather than check the result of test-and-clear, just check that we have the writeback bit set at the start. This wouldn't catch every case, but it's good enough (and enables the next patch). Signed-off-by: Matthew Wilcox (Oracle) --- mm/filemap.c | 9 +++++++-- mm/internal.h | 2 +- mm/page-writeback.c | 38 ++++++++++++++++---------------------- 3 files changed, 24 insertions(+), 25 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 3dad2615af41..ddcced4638b5 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1595,9 +1595,15 @@ EXPORT_SYMBOL(folio_wait_private_2_killable); /** * folio_end_writeback - End writeback against a folio. * @folio: The folio. + * + * The folio must actually be under writeback. + * + * Context: May be called from process or interrupt context. */ void folio_end_writeback(struct folio *folio) { + VM_BUG_ON_FOLIO(!folio_test_writeback(folio), folio); + /* * folio_test_clear_reclaim() could be used here but it is an * atomic operation and overkill in this particular case. Failing @@ -1617,8 +1623,7 @@ void folio_end_writeback(struct folio *folio) * reused before the folio_wake(). */ folio_get(folio); - if (!__folio_end_writeback(folio)) - BUG(); + __folio_end_writeback(folio); smp_mb__after_atomic(); folio_wake(folio, PG_writeback); diff --git a/mm/internal.h b/mm/internal.h index 30cf724ddbce..ccb08dd9b5ec 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -105,7 +105,7 @@ static inline void wake_throttle_isolated(pg_data_t *pgdat) vm_fault_t do_swap_page(struct vm_fault *vmf); void folio_rotate_reclaimable(struct folio *folio); -bool __folio_end_writeback(struct folio *folio); +void __folio_end_writeback(struct folio *folio); void deactivate_file_folio(struct folio *folio); void folio_activate(struct folio *folio); diff --git a/mm/page-writeback.c b/mm/page-writeback.c index b8d3d7040a50..410b53e888e3 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -2940,11 +2940,10 @@ static void wb_inode_writeback_end(struct bdi_writeback *wb) spin_unlock_irqrestore(&wb->work_lock, flags); } -bool __folio_end_writeback(struct folio *folio) +void __folio_end_writeback(struct folio *folio) { long nr = folio_nr_pages(folio); struct address_space *mapping = folio_mapping(folio); - bool ret; folio_memcg_lock(folio); if (mapping && mapping_use_writeback_tags(mapping)) { @@ -2953,19 +2952,16 @@ bool __folio_end_writeback(struct folio *folio) unsigned long flags; xa_lock_irqsave(&mapping->i_pages, flags); - ret = folio_test_clear_writeback(folio); - if (ret) { - __xa_clear_mark(&mapping->i_pages, folio_index(folio), - PAGECACHE_TAG_WRITEBACK); - if (bdi->capabilities & BDI_CAP_WRITEBACK_ACCT) { - struct bdi_writeback *wb = inode_to_wb(inode); - - wb_stat_mod(wb, WB_WRITEBACK, -nr); - __wb_writeout_add(wb, nr); - if (!mapping_tagged(mapping, - PAGECACHE_TAG_WRITEBACK)) - wb_inode_writeback_end(wb); - } + folio_test_clear_writeback(folio); + __xa_clear_mark(&mapping->i_pages, folio_index(folio), + PAGECACHE_TAG_WRITEBACK); + if (bdi->capabilities & BDI_CAP_WRITEBACK_ACCT) { + struct bdi_writeback *wb = inode_to_wb(inode); + + wb_stat_mod(wb, WB_WRITEBACK, -nr); + __wb_writeout_add(wb, nr); + if (!mapping_tagged(mapping, PAGECACHE_TAG_WRITEBACK)) + wb_inode_writeback_end(wb); } if (mapping->host && !mapping_tagged(mapping, @@ -2974,15 +2970,13 @@ bool __folio_end_writeback(struct folio *folio) xa_unlock_irqrestore(&mapping->i_pages, flags); } else { - ret = folio_test_clear_writeback(folio); - } - if (ret) { - lruvec_stat_mod_folio(folio, NR_WRITEBACK, -nr); - zone_stat_mod_folio(folio, NR_ZONE_WRITE_PENDING, -nr); - node_stat_mod_folio(folio, NR_WRITTEN, nr); + folio_test_clear_writeback(folio); } + + lruvec_stat_mod_folio(folio, NR_WRITEBACK, -nr); + zone_stat_mod_folio(folio, NR_ZONE_WRITE_PENDING, -nr); + node_stat_mod_folio(folio, NR_WRITTEN, nr); folio_memcg_unlock(folio); - return ret; } bool __folio_start_writeback(struct folio *folio, bool keep_write) From patchwork Wed Oct 4 16:53:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13409066 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E60CE7C4E2 for ; Wed, 4 Oct 2023 16:53:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 26AB36B026E; Wed, 4 Oct 2023 12:53:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 131C26B026F; Wed, 4 Oct 2023 12:53:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DCD096B026F; Wed, 4 Oct 2023 12:53:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id CA50F6B026E for ; Wed, 4 Oct 2023 12:53:24 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 916691A03D1 for ; Wed, 4 Oct 2023 16:53:24 +0000 (UTC) X-FDA: 81308374728.19.E502F0B Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf25.hostedemail.com (Postfix) with ESMTP id C0FF0A0027 for ; Wed, 4 Oct 2023 16:53:22 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Kf1rS9d9; dmarc=none; spf=none (imf25.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696438402; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8+vjrjpTMU20NjTh21ymbSWyQ+A6oOAP0PoW8Flxz4A=; b=Z4AMq1aEu7q+cD0waapP50VpVaWOhxjRFAOso7ZuxmVXgK+JuyVqBr9Hdhor0zph3zDrI5 yLInR+1GAEZ/hFwZksLhtP2eDnGL+CmQOTs94TsRz77k2ktNvCKuF40kKrIlil1HBDXkwm IthEaTYK34Xzvkk9kW9fiVSfDmIF0bA= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Kf1rS9d9; dmarc=none; spf=none (imf25.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696438402; a=rsa-sha256; cv=none; b=k0h8MxYj0OfoMv5eWP0j6RHUDiRhPPNBa1fPrq2ZR0boRA6pFOEtrkxvLJOpHsRQccETTI B5J2tOXjJAZvTGpAMQnU7dHxNUC7MYbR7XQnjOGalQpVY4b2mNAVstHqVdWiR4IoszKiBo fA/slOhK5sfVgQ79oaL1SWqqkveTrQo= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=8+vjrjpTMU20NjTh21ymbSWyQ+A6oOAP0PoW8Flxz4A=; b=Kf1rS9d9vz2PJFuFKvV7rph0p1 HCyHPGHA+FbW1KAMoLhpQu/T9u3tpDlH9vyzTv4mDkTX0yV9Dpe2uN3ldxUDZ+9/i+xmwogS5qTLX kqFEqo9od39oL/2d6fXstvwLvJTaE4kIBipi7WvWwrlGAP/Qsrxq0k8cweo7AhCb9Y7zHrF2PA72x D9qgc0osOC51gblLOkrYKh983huTeODe/DozdfPHgqA/F5MUrc8QAVjCmkZY6+l0ikP6FRn5y5JLs IxZNcoqdFknci4ZLy6qYxJXmgtgyvGn5D5UmxyUKH2SeMwDmUoeBQ4QZRdI8YDQgrgROf03t6vi4R La+kEGjg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qo57h-004SGM-0x; Wed, 04 Oct 2023 16:53:21 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, torvalds@linux-foundation.org, npiggin@gmail.com Subject: [PATCH v2 17/17] mm: Use folio_xor_flags_has_waiters() in folio_end_writeback() Date: Wed, 4 Oct 2023 17:53:17 +0100 Message-Id: <20231004165317.1061855-18-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231004165317.1061855-1-willy@infradead.org> References: <20231004165317.1061855-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: C0FF0A0027 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: h8hifuqorw38hysuj7gtta64b5mt36oe X-HE-Tag: 1696438402-171243 X-HE-Meta: U2FsdGVkX19m1QsudW0+HdPfuk/fZmzzAFl8MJckxEPOzx0kg7dzUxwa7O8Nszv0L1+Gdcs5c9nUZHQ6JivyAyo+yPlvfR+jjdICRrA2aNJzAM62rk+606vrwovD+oWc9BxMFXnTxmK01mKpWUnxBG1L0cz7nq30FmLWE6+TM3qBfetYLepOTONXz/96/z0oSPMru3QNzMLEKWm+WnDvxkNPQcz11b1LilAIxjPhHJb/3MXwUO04VuQOmMizSZ4VwhnyPac7IpzTFe+G9KyAsNGXKcgoPI/s1jFJKSBlkN/4dGEs1ipxsUlCIZ9xSLORsXeUqWH3X4FpNeN6SRsl4FfB4p7OgLH1zxvRuPe/254AeK0oe3fsaSib1w/z2E0RUWIlnJ9TleXYnfhtX+VaLe6PN20c6afaWKpKCPht/5P9F334FRUq6zgjsllDyBN2mJgEzH7AvmX5tYwQ03rjmR93EoJp5nvZx71WfiRjE96wg8DfQJb3P0332b97UTV/fFjgZZLbfpcMOCi6kJ0BKa92weB0JShzHTTpmWXzPzOvbGOPeAcMYm7YUsL8IUFIsVZOo7XENQi4+P8l6kf3Z+pUPavIhKODHs8zQYm5ASYOvIWWbLQTyLYmEDLC8ppvm4ertKqR3x5IeglMt8EaEjGXFQalI51y31UoIMz/MvdqdUZhSN6146FahMSnxAZy50bgrXQ/vEL5SBbv6NAEGdI6J8lUOqqwMkAZ2BLxD5Fkxc3cTvVoDBEq5PtOuqC5JsaH/L35Wi043v69m4rJbv46FHVD+sLN+2RcPIWXWjf2U8yMMw8BhlGv9u4XxIGKrzlBDOU/d7/4crsvqNVRxTPoEldhG35/EoA2fzqacvU3n8XZKDLzA6BDi4b3a8vi3PKjMAf+eveHYLJSoqVAlVs5wgFagCl3Xqim4jzr1eGZiiJjtp+jRZdgZTd6ve/lOQRK2Xaez7k14ThFUgK RTFqC9Nl 8tjnVvxTYFgb/F2wwCPpxlfnmuF6X4gfZLHMKXPg+wZj1xpOkItQSeAdsckm4BJjZFuGWJqes4ow8pzTUmLQP4sTOZLJvNM+Yj4wxM05v22JRf7jQzWku3y1nr/p2nwo1ut16RowERcsAIGK7i2VE4LqNOFTjUUw4jNlAkepOQtaoMkBCocJulrpnd1RJdO6Yu9MK98AmlsYI0oCmUi3sqpIjEeseM2zPLGsH X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Match how folio_unlock() works by combining the test for PG_waiters with the clearing of PG_writeback. This should have a small performance win, and removes the last user of folio_wake(). Signed-off-by: Matthew Wilcox (Oracle) --- mm/filemap.c | 15 +++------------ mm/internal.h | 2 +- mm/page-writeback.c | 9 ++++++--- 3 files changed, 10 insertions(+), 16 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index ddcced4638b5..56bfba546be7 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1177,13 +1177,6 @@ static void folio_wake_bit(struct folio *folio, int bit_nr) spin_unlock_irqrestore(&q->lock, flags); } -static void folio_wake(struct folio *folio, int bit) -{ - if (!folio_test_waiters(folio)) - return; - folio_wake_bit(folio, bit); -} - /* * A choice of three behaviors for folio_wait_bit_common(): */ @@ -1620,13 +1613,11 @@ void folio_end_writeback(struct folio *folio) * Writeback does not hold a folio reference of its own, relying * on truncation to wait for the clearing of PG_writeback. * But here we must make sure that the folio is not freed and - * reused before the folio_wake(). + * reused before the folio_wake_bit(). */ folio_get(folio); - __folio_end_writeback(folio); - - smp_mb__after_atomic(); - folio_wake(folio, PG_writeback); + if (__folio_end_writeback(folio)) + folio_wake_bit(folio, PG_writeback); acct_reclaim_writeback(folio); folio_put(folio); } diff --git a/mm/internal.h b/mm/internal.h index ccb08dd9b5ec..30cf724ddbce 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -105,7 +105,7 @@ static inline void wake_throttle_isolated(pg_data_t *pgdat) vm_fault_t do_swap_page(struct vm_fault *vmf); void folio_rotate_reclaimable(struct folio *folio); -void __folio_end_writeback(struct folio *folio); +bool __folio_end_writeback(struct folio *folio); void deactivate_file_folio(struct folio *folio); void folio_activate(struct folio *folio); diff --git a/mm/page-writeback.c b/mm/page-writeback.c index 410b53e888e3..c52514fe1d23 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -2940,10 +2940,11 @@ static void wb_inode_writeback_end(struct bdi_writeback *wb) spin_unlock_irqrestore(&wb->work_lock, flags); } -void __folio_end_writeback(struct folio *folio) +bool __folio_end_writeback(struct folio *folio) { long nr = folio_nr_pages(folio); struct address_space *mapping = folio_mapping(folio); + bool ret; folio_memcg_lock(folio); if (mapping && mapping_use_writeback_tags(mapping)) { @@ -2952,7 +2953,7 @@ void __folio_end_writeback(struct folio *folio) unsigned long flags; xa_lock_irqsave(&mapping->i_pages, flags); - folio_test_clear_writeback(folio); + ret = folio_xor_flags_has_waiters(folio, 1 << PG_writeback); __xa_clear_mark(&mapping->i_pages, folio_index(folio), PAGECACHE_TAG_WRITEBACK); if (bdi->capabilities & BDI_CAP_WRITEBACK_ACCT) { @@ -2970,13 +2971,15 @@ void __folio_end_writeback(struct folio *folio) xa_unlock_irqrestore(&mapping->i_pages, flags); } else { - folio_test_clear_writeback(folio); + ret = folio_xor_flags_has_waiters(folio, 1 << PG_writeback); } lruvec_stat_mod_folio(folio, NR_WRITEBACK, -nr); zone_stat_mod_folio(folio, NR_ZONE_WRITE_PENDING, -nr); node_stat_mod_folio(folio, NR_WRITTEN, nr); folio_memcg_unlock(folio); + + return ret; } bool __folio_start_writeback(struct folio *folio, bool keep_write)