From patchwork Fri Nov 17 14:51:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13459013 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DCBFC5AE4B for ; Fri, 17 Nov 2023 14:52:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 682016B04D7; Fri, 17 Nov 2023 09:52:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 59E226B04DC; Fri, 17 Nov 2023 09:52:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2B6E86B04D7; Fri, 17 Nov 2023 09:52:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id C0E276B04DA for ; Fri, 17 Nov 2023 09:51:59 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 8D83CC020B for ; Fri, 17 Nov 2023 14:51:59 +0000 (UTC) X-FDA: 81467735958.12.14B9364 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf11.hostedemail.com (Postfix) with ESMTP id 6922640008 for ; Fri, 17 Nov 2023 14:51:57 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=j0pCGrSv; spf=none (imf11.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700232718; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JrgaNSZkbdttPU8m7IV9E3SqR4RsniiNUF24kh9JfO8=; b=ADlAKfskXG+RSByQcs2m8eqF6cuwAEkrcygb3qeVbzT0yq6zqrHw6tQ3rCrzneffF/C8oT B7u2hYX6n7TSciMqNKHroqg2GSO3TJcsiqrW1SySx+1vWfDQTVnyofxXoZ4M9nPrhdn7Z1 Ut2rtookesL/vuyFtlwxc4pAArFrO8Q= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700232718; a=rsa-sha256; cv=none; b=7YK7flIa97PRnpcG7E57bpgwG4DaP39TsekmBjRDRavgzRptXpEi88xBjptBFoqqHx0mim Ko77c2X381RIUl3F+P/X5c5ZIGS0D4VzfJTTk/YgHy8D6gvWXe2mTsjy0GjvqvckwBlxX4 UN+39KmcDQgZ0o3G7btOSLDBQOhkJVA= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=j0pCGrSv; spf=none (imf11.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=JrgaNSZkbdttPU8m7IV9E3SqR4RsniiNUF24kh9JfO8=; b=j0pCGrSveYKYRXtt4HIE3G3IaJ /XXSDQT1KRDKsvIp0aEw1OPQ3r6ItpfPvYmp9t5iO2Dt1LT3Nm+QMU4PLIh2lNQ/Whkvl2XHXKcg/ NUmkcSyCUy2Pw1gzXv5O0ETHuP1cxAfVP4uISEcYEggWj4OEXXBdm4BVxFum5Nq1HfBFmaZQAVFUs hwq+cjIJVhokWwYI+yme5U4DfMhLpSRVRm1vIS0Jjg3qGTkiUhTYpYJQ6ge1tExmFzieL1U4DfYdx yDJC32wJJnGckiV08EUX7cyJrxzf+PZirjLBGP/VtznM+WnX/sAgooLYKkDbVDax43ZtDkrlNG2qB q7D+hQNg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1r40C7-009ypx-OE; Fri, 17 Nov 2023 14:51:43 +0000 From: "Matthew Wilcox (Oracle)" To: Peter Zijlstra , Ingo Molnar , Will Deacon , Waiman Long Cc: "Matthew Wilcox (Oracle)" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Chandan Babu R , "Darrick J . Wong" , linux-xfs@vger.kernel.org, Mateusz Guzik Subject: [PATCH v4 1/4] locking: Add rwsem_assert_held() and rwsem_assert_held_write() Date: Fri, 17 Nov 2023 14:51:39 +0000 Message-Id: <20231117145142.2378800-2-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231117145142.2378800-1-willy@infradead.org> References: <20231117145142.2378800-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6922640008 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: s4s6ffwyswgddjgge74hm1u1qpmfkkef X-HE-Tag: 1700232717-587477 X-HE-Meta: U2FsdGVkX1/0/9tFoQKem3gVl1GsQk9gUomXAiG9WzjWHKn68PCfFaqjjttXQj/OnDIA49I09Hp+B57LQDExf8CAe1Zt+nYln7T5vXHTvLL1KUbIGPZ6r+a+0BcF1ECgoKPlQ87Xvd2Zal1SrZzvjDrVCh+8e2ZmUdCN76nf2APlzfSt8NGQ2mFcygn1vdfBsOvw1fOMFXejIRlT1/EQW2CUHFdHysV6nZB4VmXNwCDTy35dbw+9X1KZFXHGkVuGVXVqI4r4/rE6odC0jLU35y7bUzYrev7vuDPFweifz2UmlUgcyUW10Evk6gFFE1EsbVsA0Xoyxgg2RdMZpLWzhZHkUkTB+qQf2orGMtPOKaZbzYpBuLRiAwbdv7+uKF28N2X8I+Kc0LtKzPlPD5o+ZCycTCNFjJPJFCEdGik81x2VSHzfmgsDedUEFZ5PfDTrNR1L1FoB2tH2gtP48zraNve4FyScX2Eybr87fG8dZu/YTRCVVEMvQxP+H2WvTkW4iJnAKMq7TxrPToAWMWXyOflSITAGdDy+QkewlbuahNu+cCBq860HrQ96wXGlezITD6NrBQmuBB6Hn3qJLsw6RGUzdBbDbRzax0+OYOKXqSUyrFxYtFhpL+5hjhRdiOx2vaC6Dk81mXpdHJ10iyPsT5WC8msfG6VgWyDQ4npVsHTyEpqVc/+ydB1LqRLmXyd8tvf4CkBTTfM8cDthgrFICM+EIwXVhtiUv6lW8G+4fvJoXvxo3x5Nb/ek25f29CLdXoLWhhRvJzTw6K1lNuxtJPZITPQ1AIMUlS+CBLMSCa/PFOUU+Pgtv1udEW5M1ncWAAUG+C6Ork/Y0la+YYMCd3n0z83IUftsUi0q9ova9mz3wzM90dus3xmcsRJCUU+pJ5gBKywOlXdbspBupVhqVnDL6/ZBJvjE5AxV6pxw64TC8BSRXVM1OmDloaj1BT3rB+hDnwK1NbiVjWfJadX VK3sqEEv BkFw7J+wzMvji73EOaaynccsZ0eu44LwoHvcJcXW6hz/CEtz4uyDFWuzD9vXwAr8jRvF4sZScqDlaPFfo62Y9cWB8lqObVz35NP60IKVRBr9C3e9MlJqcz7t9Se7gj7laAPIK1l1dqalzLBclg646zur9R54MbmxuHfnEaIj4COY51EPX+G7wlUj/jeoougrMKHSSD/gIdIfSYJxDUD1xDmjepm7K9k7LMK2gkmtbJ/diuqk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Modelled after lockdep_assert_held() and lockdep_assert_held_write(), but are always active, even when lockdep is disabled. Of course, they don't test that _this_ thread is the owner, but it's sufficient to catch many bugs and doesn't incur the same performance penalty as lockdep. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Peter Zijlstra (Intel) --- include/linux/rwbase_rt.h | 9 ++++++-- include/linux/rwsem.h | 46 ++++++++++++++++++++++++++++++++++----- 2 files changed, 48 insertions(+), 7 deletions(-) diff --git a/include/linux/rwbase_rt.h b/include/linux/rwbase_rt.h index 1d264dd08625..29c4e4f243e4 100644 --- a/include/linux/rwbase_rt.h +++ b/include/linux/rwbase_rt.h @@ -26,12 +26,17 @@ struct rwbase_rt { } while (0) -static __always_inline bool rw_base_is_locked(struct rwbase_rt *rwb) +static __always_inline bool rw_base_is_locked(const struct rwbase_rt *rwb) { return atomic_read(&rwb->readers) != READER_BIAS; } -static __always_inline bool rw_base_is_contended(struct rwbase_rt *rwb) +static inline void rw_base_assert_held_write(const struct rwbase_rt *rwb) +{ + WARN_ON(atomic_read(&rwb->readers) != WRITER_BIAS); +} + +static __always_inline bool rw_base_is_contended(const struct rwbase_rt *rwb) { return atomic_read(&rwb->readers) > 0; } diff --git a/include/linux/rwsem.h b/include/linux/rwsem.h index 1dd530ce8b45..e032b9644af5 100644 --- a/include/linux/rwsem.h +++ b/include/linux/rwsem.h @@ -66,14 +66,24 @@ struct rw_semaphore { #endif }; -/* In all implementations count != 0 means locked */ +#define RWSEM_UNLOCKED_VALUE 0UL +#define RWSEM_WRITER_LOCKED (1UL << 0) +#define __RWSEM_COUNT_INIT(name) .count = ATOMIC_LONG_INIT(RWSEM_UNLOCKED_VALUE) + static inline int rwsem_is_locked(struct rw_semaphore *sem) { - return atomic_long_read(&sem->count) != 0; + return atomic_long_read(&sem->count) != RWSEM_UNLOCKED_VALUE; } -#define RWSEM_UNLOCKED_VALUE 0L -#define __RWSEM_COUNT_INIT(name) .count = ATOMIC_LONG_INIT(RWSEM_UNLOCKED_VALUE) +static inline void rwsem_assert_held_nolockdep(const struct rw_semaphore *sem) +{ + WARN_ON(atomic_long_read(&sem->count) == RWSEM_UNLOCKED_VALUE); +} + +static inline void rwsem_assert_held_write_nolockdep(const struct rw_semaphore *sem) +{ + WARN_ON(!(atomic_long_read(&sem->count) & RWSEM_WRITER_LOCKED)); +} /* Common initializer macros and functions */ @@ -152,11 +162,21 @@ do { \ __init_rwsem((sem), #sem, &__key); \ } while (0) -static __always_inline int rwsem_is_locked(struct rw_semaphore *sem) +static __always_inline int rwsem_is_locked(const struct rw_semaphore *sem) { return rw_base_is_locked(&sem->rwbase); } +static inline void rwsem_assert_held_nolockdep(const struct rw_semaphore *sem) +{ + WARN_ON(!rwsem_is_locked(sem)); +} + +static inline void rwsem_assert_held_write_nolockdep(const struct rw_semaphore *sem) +{ + rw_base_assert_held_write(sem); +} + static __always_inline int rwsem_is_contended(struct rw_semaphore *sem) { return rw_base_is_contended(&sem->rwbase); @@ -169,6 +189,22 @@ static __always_inline int rwsem_is_contended(struct rw_semaphore *sem) * the RT specific variant. */ +static inline void rwsem_assert_held(const struct rw_semaphore *sem) +{ + if (IS_ENABLED(CONFIG_LOCKDEP)) + lockdep_assert_held(sem); + else + rwsem_assert_held_nolockdep(sem); +} + +static inline void rwsem_assert_held_write(const struct rw_semaphore *sem) +{ + if (IS_ENABLED(CONFIG_LOCKDEP)) + lockdep_assert_held_write(sem); + else + rwsem_assert_held_write_nolockdep(sem); +} + /* * lock for reading */ From patchwork Fri Nov 17 14:51:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13459015 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFB6DC5AD4C for ; Fri, 17 Nov 2023 14:52:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 61BDB6B04DB; Fri, 17 Nov 2023 09:52:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F1E36B04E1; Fri, 17 Nov 2023 09:52:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1A6526B04DF; Fri, 17 Nov 2023 09:52:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E67136B04DB for ; Fri, 17 Nov 2023 09:52:04 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id C36611A05D3 for ; Fri, 17 Nov 2023 14:52:04 +0000 (UTC) X-FDA: 81467736168.22.73608B5 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf26.hostedemail.com (Postfix) with ESMTP id 26222140029 for ; Fri, 17 Nov 2023 14:52:02 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=G81dPiQI; dmarc=none; spf=none (imf26.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700232723; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=25QfWharHbMWrIl6caRn0CZ+MZ75oP4Z9qoseEg2zjc=; b=kN7pUhm1dA7NAyOQDTxQKPOpbLC5B5hqZPZZw9ju68VeK/J812eKMhEavzn5cw41p3hIR4 J0iyVkgFJQmc3jFJYcqQeIPJexvn0kJ/TJaIZTuEBhookzAm2/uTgoGRnFRZqtmMZDUtCG YW4+mMF8QjSawJqCGH3mVASQXuVHym4= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=G81dPiQI; dmarc=none; spf=none (imf26.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700232723; a=rsa-sha256; cv=none; b=EkE6qzQ/MbsC7HzECP5CgOSDXXPnBSG4W0Dk1hd7TvLl7y5v7B9uSL1ATQ3ZpWbVJMm0qK B3LF5X5R3TJfJP6P/dUmYvhB5Rex4R5OdvvL5uCNfH5EIwOcW4g73iaEKUntX7CHzg7DGB zcvxFONS8UGlxKkx8a6TS4x78vhU11g= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=25QfWharHbMWrIl6caRn0CZ+MZ75oP4Z9qoseEg2zjc=; b=G81dPiQIHl0SfOKIPMZ8I9sw2V d5ARoUWM1/yrnIdvjjTz4FsVu4uYDcbTLzSzH9bRwHDPxLfwbH/ILpx4RNS/G5aRerWYAFKNQa0oV +3SrFGRIhq2VzyXgwrmZZKHOs50oadM/vEMDA5KZRzEySyyeS9Pt0/H+jP3ODZaK5NZnH/XcIZC+U HWiWKF4SQx8ge+N3F+EDUFb9KgYvH/Z4E0pVcv0QrGrimiNp98ZrAKM+28ysPuxS7mlOvyv+eIcMj kmgYY9P65vTsHGeZz3gSLjNBxHZcWLLJ3eRMZMCL+gfHEXD4THepQSNv0oPCJaAG0deVfJWPq+i2d QnOeX+/g==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1r40C7-009ypz-Ry; Fri, 17 Nov 2023 14:51:43 +0000 From: "Matthew Wilcox (Oracle)" To: Peter Zijlstra , Ingo Molnar , Will Deacon , Waiman Long Cc: "Matthew Wilcox (Oracle)" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Chandan Babu R , "Darrick J . Wong" , linux-xfs@vger.kernel.org, Mateusz Guzik Subject: [PATCH v4 2/4] mm: Use rwsem assertion macros for mmap_lock Date: Fri, 17 Nov 2023 14:51:40 +0000 Message-Id: <20231117145142.2378800-3-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231117145142.2378800-1-willy@infradead.org> References: <20231117145142.2378800-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 26222140029 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: pas4zpzsacr9j8xy5534tnu7j34j85n3 X-HE-Tag: 1700232722-980520 X-HE-Meta: U2FsdGVkX1/fNK0tutAAMtK1/GNiGU4ZcGfuMhS3kjisjptpR5G9iSi27H1pWLACzYLQJ1kuqikTc2TFKeAZGB0DzmhNXb5zmawa0X2f1H4XXg8H4HKh18bx4YolyjVfFO4vMEKbp3gPnhquuMNpJ7x/wgQCmVgY82R95g8imktETJgqfx6IA5/Fqz1EQlAdwlmXkU3H28Wy2jcz5oGC03CZ0QE8GJlqFVRVwBNB2qXw5pd7V6AmrKvoo3e9y9UsngcBkIYpl0JBX82F+87fC6d1Lwd7Bylp2ASnBdlkE4jyssuF69X8dRkRgmIKY+3nG3EkEQyTos5CNYONITQFNShpnJk0eJ2HyWDQPz7yt+SCKbKrOCvuohd3sZVZQmNTBJE2hsh77zWi5vKFll4uVh21zNOpXmi58Bi1RC56Al+QCNcL84xjZcq9F48SGIXhh71NqbChQbnMj7YKR5eJDUOILJu8/J0JrnOr4CZFSpbRDGoAjqVqUp7e5lp6vFSyUKX7TZz61MbkQJzJyDsiFxN4nAnr+kd2WPeKRbL1KWB216xloYic4w7u9mPpue8ANDJtztKQVt1IfQXt90Ckh/XDRaBWDuXdnIO/9jVH3gzkpjPNfYSxfbTJs6cgd/MGWqYMW6pCfVCqWho6XnFo685DtPr5fAzJ69WHd11ajeyjRUOfk5cxL52IKNT0evInGMlnNDX9CMZ1ZX5wPRqX2Cm1hl5nkmjxIpl8jboa9oAYsn3zOH8hjD6FfEoy/TucHN0EMJOcVZgEgittt8Awht/E7gkWZn4gxyn6E+l7QOfrToaJJm2SM6mcpuonKGTdcyYbuPIjs0DAy/qT3r6p+uSo+97W+R5AVqEsP9FdKkS+Xu4RvZowho5xvmv4xNbPs4D2yoHQknLEa0jOnWKmeuGovOlXTFl4CNsU3syOInJG428jbwPf2p+tWWJE9Gfz9gd7enqRiHnu1txpqCV o7ePm2H0 jQOM8JnMEkOUnAf6uoq1WNEedlBmeZ/SEQ6REcCp4ICMywiAgHPNyGNCRxuVZb7XB7SzcviLLkx2UaDE5INq27oIbPFZyVDFSxkM4MhJXcFA/Isjclmq17IM+99Tk+uKB/Hj3PV/tBkQyACW6OMngIUN0jMA9ldlOMsX7it0ZOMHNJzt8+c5qM+iQRF+C1Hf5kWNGmXQZy4UQfQ7WCS9UNxdisWn7prktZ5Nk X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This slightly strengthens our write assertion when lockdep is disabled. It also downgrades us from BUG_ON to WARN_ON, but I think that's an improvement. I don't think dumping the mm_struct was all that valuable; the call chain is what's important. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/mmap_lock.h | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h index 8d38dcb6d044..de9dc20b01ba 100644 --- a/include/linux/mmap_lock.h +++ b/include/linux/mmap_lock.h @@ -60,16 +60,14 @@ static inline void __mmap_lock_trace_released(struct mm_struct *mm, bool write) #endif /* CONFIG_TRACING */ -static inline void mmap_assert_locked(struct mm_struct *mm) +static inline void mmap_assert_locked(const struct mm_struct *mm) { - lockdep_assert_held(&mm->mmap_lock); - VM_BUG_ON_MM(!rwsem_is_locked(&mm->mmap_lock), mm); + rwsem_assert_held(&mm->mmap_lock); } -static inline void mmap_assert_write_locked(struct mm_struct *mm) +static inline void mmap_assert_write_locked(const struct mm_struct *mm) { - lockdep_assert_held_write(&mm->mmap_lock); - VM_BUG_ON_MM(!rwsem_is_locked(&mm->mmap_lock), mm); + rwsem_assert_held_write(&mm->mmap_lock); } #ifdef CONFIG_PER_VMA_LOCK From patchwork Fri Nov 17 14:51:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13459012 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A5C1C5AD4C for ; Fri, 17 Nov 2023 14:52:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2BC0D6B04DA; Fri, 17 Nov 2023 09:52:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 268E66B04DB; Fri, 17 Nov 2023 09:52:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EA3336B04DC; Fri, 17 Nov 2023 09:51:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B3ED36B04D7 for ; Fri, 17 Nov 2023 09:51:59 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 83B8F1CBBA0 for ; Fri, 17 Nov 2023 14:51:59 +0000 (UTC) X-FDA: 81467735958.17.A5537EA Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf20.hostedemail.com (Postfix) with ESMTP id 8591F1C000F for ; Fri, 17 Nov 2023 14:51:57 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="kIJ/pJUv"; spf=none (imf20.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700232717; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zIWhQPPVGGNsaH/zRjrbtFp7YhWrYOd/f9YxTGaAZsk=; b=FCHgbRM3vdlLMmKF0jfDMCEEIwyKZnda7Q7rFaXkBgKMaWlI6KWtOm3NPLQSaJyZwvRur/ CFb+E4qOLwjDpw3cXjBYihb7V5haojje8+tEwTdQ1xae3f/+GVz1Rl/rOXZdwv5v1V09wE Bxs+SCLnEWG2Dp/vX/SLdi8L+PezgBk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700232718; a=rsa-sha256; cv=none; b=CaPDuwzNkG1obu7unKKWmFyZVJDsBuGABZoYRiMfv8oFyf/l/LdVXaT0OaKtdnpFWknFbW zzENhoTUS2Xa/eeZO1pBnUyaaaq+MAC0aohQ0S+ydLWjCMmgUPtU7uh0c/k2HvgDvsyfAr urXJ004116cWohwPS9MSXYyOcaET8Qo= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="kIJ/pJUv"; spf=none (imf20.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=zIWhQPPVGGNsaH/zRjrbtFp7YhWrYOd/f9YxTGaAZsk=; b=kIJ/pJUvSzvjSIPMKQsspGnDTZ 1C5xnvZh9UkDf+GK1wO/glLeHlDcdWjghj77Dn+2CPfES6jNKqlJwhkywPWEyy05+MbnGby92NlDv CA8G81oHSj9eQJ5ZytrYk0yeN3Cc8Gkf5LTw5GEkNoSgvKg3pOEKtNLRUPCKL8BV/Jl4UnoPU+Y8w ZNC9UPYIxoJAIiKqx0qELYhGMOgfCwQXZKHat4+pVCEypDTZUPSijVM+AfQ4jg6egGny3QBoXXWpa VwzLuHZYbqiKnW1Yg6aH9dhOpxZrpQtJCbtLWFAqf8GyRFtWdssrQIyv0gX8kSx41xIRyiGlaSk2w e5Niqi7Q==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1r40C7-009yq1-V0; Fri, 17 Nov 2023 14:51:44 +0000 From: "Matthew Wilcox (Oracle)" To: Peter Zijlstra , Ingo Molnar , Will Deacon , Waiman Long Cc: "Matthew Wilcox (Oracle)" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Chandan Babu R , "Darrick J . Wong" , linux-xfs@vger.kernel.org, Mateusz Guzik Subject: [PATCH v4 3/4] xfs: Replace xfs_isilocked with xfs_assert_ilocked Date: Fri, 17 Nov 2023 14:51:41 +0000 Message-Id: <20231117145142.2378800-4-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231117145142.2378800-1-willy@infradead.org> References: <20231117145142.2378800-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 8591F1C000F X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 9r1scrj65tiknjt71nttrfgrfiexfdwg X-HE-Tag: 1700232717-743844 X-HE-Meta: U2FsdGVkX18zqiLpiwIHp5XNz68piVdV9/63svH65paaiqQwVyqLeS9Hl7Syp71wvUx/UrNcNuj2l3t/7cBg+it1YNSoidGrl4kWon12WEvVUMvuJYGViH0wYeARGCIDbbYa6ROFl2DOCU4a1vVED2wRIKb9z9l4ESdK9rqN7ieQE/ScL1qb2npQJgq3mBXJtuFf/Q83j+0313PepRsSrL7d6UD4pQaFySI1G08TcB+iWdPIhW6eTJW5Dia5pMhLcGjDjGAa2hYs03V5sYN/Y2tA5dG+e3bZfEaFucx4QnVIy35Ek79xP7u9mAvcith0e+GFdOaUi416llYzvK9fhg2KiAp2G5DK/2BBIiCD3hBeOmMin2PzczLLXawwrUsAhU1PfxHwculrKT3JTvH5jFjWPasTt9TwJdScdn8jSpHXiWXFfockg2nO01z5+KWlWDe3Rag1F9GDwCQVeMHGyWZnJ23Mdo6v2+sXzS5mZ/gZI3CriKd7dd/IdbtWoRKdl88rC5maxtR0+2zwVwVSTxtmWUixlQbtsPLJ13Zqj7ML+iJ2EiYX948l8pa7iXzBiJwCJFE0pDvbmih8EHPJPu98UMIs7lMSXdmK2Xhv4mSeeu5youJl++H52juZTfgVawdon3D/N+Ni/vQdTChYcEWswnl8q90MM5KrREBFmlVwPhAjYMSi7IT2PCPkzgZ84FgUeSOZhBVJVIJUL2vb46xAq0SspjIXK+2BMldqKi/dzsBw+V1tLi6TKXZnH369cVbzPoOMVzuHtNwoBDZHgNNyn7C7Q984+/LLAPygr7zQ5YqC+4hnRRDswI7b47Q+95Y8UCzfq/8i5Lgx0TLTFJKW6ojSFQ2dTRt5qoSlESJMegGQGpv9A96elbr9ltmmHWmAZ5fl99ODJYbINoDN44K8UMN+FzzUGkRJCjGvbAozVqUPt8R4nsMwwduxHNWJczQDDvWjxTCeqvn2b8Y q8dGH/+f T30b0AiGR59V+Kdjbrf/joFdc6ol6L8u1myUNthMIWDro6fx0+HWZSJ9D7Wd9EEyeoD9w+pJDnpMIpkzt4Am4O60y45awcc3r5ODb2c/YVzDCBmzIpPnpwF5Uo+D3FkGLWGfDP8Rbp/cayUYZaCzX/B8fZq+CDwKna8M37cxk4El4kEqflLYrlyKcWuuzK+oJqYMdurwrivBWCacZ3B0V6VSXzIpMh1gsQSGB X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To use the new rwsem_assert_held()/rwsem_assert_held_write(), we can't use the existing ASSERT macro. Add a new xfs_assert_ilocked() and convert all the callers. Fix an apparent bug in xfs_isilocked(): If the caller specifies XFS_IOLOCK_EXCL | XFS_ILOCK_EXCL, xfs_assert_ilocked() will check both the IOLOCK and the ILOCK are held for write. xfs_isilocked() only checked that the ILOCK was held for write. xfs_assert_ilocked() is always on, even if DEBUG or XFS_WARN aren't defined. It's a cheap check, so I don't think it's worth defining it away. Signed-off-by: Matthew Wilcox (Oracle) --- fs/xfs/libxfs/xfs_attr.c | 2 +- fs/xfs/libxfs/xfs_attr_remote.c | 2 +- fs/xfs/libxfs/xfs_bmap.c | 19 +++++---- fs/xfs/libxfs/xfs_defer.c | 2 +- fs/xfs/libxfs/xfs_inode_fork.c | 2 +- fs/xfs/libxfs/xfs_rtbitmap.c | 2 +- fs/xfs/libxfs/xfs_trans_inode.c | 6 +-- fs/xfs/scrub/readdir.c | 4 +- fs/xfs/xfs_attr_list.c | 2 +- fs/xfs/xfs_bmap_util.c | 10 ++--- fs/xfs/xfs_dir2_readdir.c | 2 +- fs/xfs/xfs_dquot.c | 4 +- fs/xfs/xfs_file.c | 4 +- fs/xfs/xfs_inode.c | 72 +++++++++++---------------------- fs/xfs/xfs_inode.h | 2 +- fs/xfs/xfs_inode_item.c | 4 +- fs/xfs/xfs_iops.c | 3 +- fs/xfs/xfs_qm.c | 10 ++--- fs/xfs/xfs_reflink.c | 2 +- fs/xfs/xfs_rtalloc.c | 4 +- fs/xfs/xfs_symlink.c | 2 +- fs/xfs/xfs_trans_dquot.c | 2 +- 22 files changed, 66 insertions(+), 96 deletions(-) diff --git a/fs/xfs/libxfs/xfs_attr.c b/fs/xfs/libxfs/xfs_attr.c index e28d93d232de..ebf0722d8963 100644 --- a/fs/xfs/libxfs/xfs_attr.c +++ b/fs/xfs/libxfs/xfs_attr.c @@ -224,7 +224,7 @@ int xfs_attr_get_ilocked( struct xfs_da_args *args) { - ASSERT(xfs_isilocked(args->dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(args->dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL); if (!xfs_inode_hasattr(args->dp)) return -ENOATTR; diff --git a/fs/xfs/libxfs/xfs_attr_remote.c b/fs/xfs/libxfs/xfs_attr_remote.c index d440393b40eb..1c007ebf153a 100644 --- a/fs/xfs/libxfs/xfs_attr_remote.c +++ b/fs/xfs/libxfs/xfs_attr_remote.c @@ -545,7 +545,7 @@ xfs_attr_rmtval_stale( struct xfs_buf *bp; int error; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (XFS_IS_CORRUPT(mp, map->br_startblock == DELAYSTARTBLOCK) || XFS_IS_CORRUPT(mp, map->br_startblock == HOLESTARTBLOCK)) diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c index be62acffad6c..eaa1dbe9a6b3 100644 --- a/fs/xfs/libxfs/xfs_bmap.c +++ b/fs/xfs/libxfs/xfs_bmap.c @@ -1189,7 +1189,7 @@ xfs_iread_extents( if (!xfs_need_iread_extents(ifp)) return 0; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ir.loaded = 0; xfs_iext_first(ifp, &ir.icur); @@ -3883,7 +3883,7 @@ xfs_bmapi_read( ASSERT(*nmap >= 1); ASSERT(!(flags & ~(XFS_BMAPI_ATTRFORK | XFS_BMAPI_ENTIRE))); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_SHARED|XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_SHARED|XFS_ILOCK_EXCL); if (WARN_ON_ONCE(!ifp)) return -EFSCORRUPTED; @@ -4354,7 +4354,7 @@ xfs_bmapi_write( ASSERT(tp != NULL); ASSERT(len > 0); ASSERT(ifp->if_format != XFS_DINODE_FMT_LOCAL); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(!(flags & XFS_BMAPI_REMAP)); /* zeroing is for currently only for data extents, not metadata */ @@ -4651,7 +4651,7 @@ xfs_bmapi_remap( ifp = xfs_ifork_ptr(ip, whichfork); ASSERT(len > 0); ASSERT(len <= (xfs_filblks_t)XFS_MAX_BMBT_EXTLEN); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(!(flags & ~(XFS_BMAPI_ATTRFORK | XFS_BMAPI_PREALLOC | XFS_BMAPI_NORMAP))); ASSERT((flags & (XFS_BMAPI_ATTRFORK | XFS_BMAPI_PREALLOC)) != @@ -5287,7 +5287,7 @@ __xfs_bunmapi( if (xfs_is_shutdown(mp)) return -EIO; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(len > 0); ASSERT(nexts >= 0); @@ -5631,8 +5631,7 @@ xfs_bmse_merge( blockcount = left->br_blockcount + got->br_blockcount; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL)); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_ILOCK_EXCL); ASSERT(xfs_bmse_can_merge(left, got, shift)); new = *left; @@ -5760,7 +5759,7 @@ xfs_bmap_collapse_extents( if (xfs_is_shutdown(mp)) return -EIO; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_ILOCK_EXCL); error = xfs_iread_extents(tp, ip, whichfork); if (error) @@ -5833,7 +5832,7 @@ xfs_bmap_can_insert_extents( int is_empty; int error = 0; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL); if (xfs_is_shutdown(ip->i_mount)) return -EIO; @@ -5875,7 +5874,7 @@ xfs_bmap_insert_extents( if (xfs_is_shutdown(mp)) return -EIO; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_ILOCK_EXCL); error = xfs_iread_extents(tp, ip, whichfork); if (error) diff --git a/fs/xfs/libxfs/xfs_defer.c b/fs/xfs/libxfs/xfs_defer.c index bcfb6a4203cd..7927a721dc86 100644 --- a/fs/xfs/libxfs/xfs_defer.c +++ b/fs/xfs/libxfs/xfs_defer.c @@ -744,7 +744,7 @@ xfs_defer_ops_capture( * transaction. */ for (i = 0; i < dfc->dfc_held.dr_inos; i++) { - ASSERT(xfs_isilocked(dfc->dfc_held.dr_ip[i], XFS_ILOCK_EXCL)); + xfs_assert_ilocked(dfc->dfc_held.dr_ip[i], XFS_ILOCK_EXCL); ihold(VFS_I(dfc->dfc_held.dr_ip[i])); } diff --git a/fs/xfs/libxfs/xfs_inode_fork.c b/fs/xfs/libxfs/xfs_inode_fork.c index 5a2e7ddfa76d..14193e044f3d 100644 --- a/fs/xfs/libxfs/xfs_inode_fork.c +++ b/fs/xfs/libxfs/xfs_inode_fork.c @@ -563,7 +563,7 @@ xfs_iextents_copy( struct xfs_bmbt_irec rec; int64_t copied = 0; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL | XFS_ILOCK_SHARED)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL | XFS_ILOCK_SHARED); ASSERT(ifp->if_bytes > 0); for_each_xfs_iext(ifp, &icur, &rec) { diff --git a/fs/xfs/libxfs/xfs_rtbitmap.c b/fs/xfs/libxfs/xfs_rtbitmap.c index c269d704314d..0b094a44c6e2 100644 --- a/fs/xfs/libxfs/xfs_rtbitmap.c +++ b/fs/xfs/libxfs/xfs_rtbitmap.c @@ -946,7 +946,7 @@ xfs_rtfree_extent( struct timespec64 atime; ASSERT(mp->m_rbmip->i_itemp != NULL); - ASSERT(xfs_isilocked(mp->m_rbmip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(mp->m_rbmip, XFS_ILOCK_EXCL); error = xfs_rtcheck_alloc_range(&args, start, len); if (error) diff --git a/fs/xfs/libxfs/xfs_trans_inode.c b/fs/xfs/libxfs/xfs_trans_inode.c index 70e97ea6eee7..69fc5b981352 100644 --- a/fs/xfs/libxfs/xfs_trans_inode.c +++ b/fs/xfs/libxfs/xfs_trans_inode.c @@ -31,7 +31,7 @@ xfs_trans_ijoin( { struct xfs_inode_log_item *iip; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (ip->i_itemp == NULL) xfs_inode_item_init(ip, ip->i_mount); iip = ip->i_itemp; @@ -60,7 +60,7 @@ xfs_trans_ichgtime( struct timespec64 tv; ASSERT(tp); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); tv = current_time(inode); @@ -90,7 +90,7 @@ xfs_trans_log_inode( struct inode *inode = VFS_I(ip); ASSERT(iip); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(!xfs_iflags_test(ip, XFS_ISTALE)); tp->t_flags |= XFS_TRANS_DIRTY; diff --git a/fs/xfs/scrub/readdir.c b/fs/xfs/scrub/readdir.c index e51c1544be63..0b200bab04c8 100644 --- a/fs/xfs/scrub/readdir.c +++ b/fs/xfs/scrub/readdir.c @@ -283,7 +283,7 @@ xchk_dir_walk( return -EIO; ASSERT(S_ISDIR(VFS_I(dp)->i_mode)); - ASSERT(xfs_isilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL); if (dp->i_df.if_format == XFS_DINODE_FMT_LOCAL) return xchk_dir_walk_sf(sc, dp, dirent_fn, priv); @@ -334,7 +334,7 @@ xchk_dir_lookup( return -EIO; ASSERT(S_ISDIR(VFS_I(dp)->i_mode)); - ASSERT(xfs_isilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL); if (dp->i_df.if_format == XFS_DINODE_FMT_LOCAL) { error = xfs_dir2_sf_lookup(&args); diff --git a/fs/xfs/xfs_attr_list.c b/fs/xfs/xfs_attr_list.c index 99bbbe1a0e44..235dd125c92f 100644 --- a/fs/xfs/xfs_attr_list.c +++ b/fs/xfs/xfs_attr_list.c @@ -505,7 +505,7 @@ xfs_attr_list_ilocked( { struct xfs_inode *dp = context->dp; - ASSERT(xfs_isilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL); /* * Decide on what work routines to call based on the inode size. diff --git a/fs/xfs/xfs_bmap_util.c b/fs/xfs/xfs_bmap_util.c index 731260a5af6d..0887fb37d84f 100644 --- a/fs/xfs/xfs_bmap_util.c +++ b/fs/xfs/xfs_bmap_util.c @@ -649,8 +649,8 @@ xfs_can_free_eofblocks( * Caller must either hold the exclusive io lock; or be inactivating * the inode, which guarantees there are no other users of the inode. */ - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL) || - (VFS_I(ip)->i_state & I_FREEING)); + if (!(VFS_I(ip)->i_state & I_FREEING)) + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL); /* prealloc/delalloc exists only on regular files */ if (!S_ISREG(VFS_I(ip)->i_mode)) @@ -1106,8 +1106,7 @@ xfs_collapse_file_space( xfs_fileoff_t shift_fsb = XFS_B_TO_FSB(mp, len); bool done = false; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL)); - ASSERT(xfs_isilocked(ip, XFS_MMAPLOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_MMAPLOCK_EXCL); trace_xfs_collapse_file_space(ip); @@ -1176,8 +1175,7 @@ xfs_insert_file_space( xfs_fileoff_t shift_fsb = XFS_B_TO_FSB(mp, len); bool done = false; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL)); - ASSERT(xfs_isilocked(ip, XFS_MMAPLOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_MMAPLOCK_EXCL); trace_xfs_insert_file_space(ip); diff --git a/fs/xfs/xfs_dir2_readdir.c b/fs/xfs/xfs_dir2_readdir.c index 9f3ceb461515..95cd8b9cf3dc 100644 --- a/fs/xfs/xfs_dir2_readdir.c +++ b/fs/xfs/xfs_dir2_readdir.c @@ -521,7 +521,7 @@ xfs_readdir( return -EIO; ASSERT(S_ISDIR(VFS_I(dp)->i_mode)); - ASSERT(xfs_isilocked(dp, XFS_IOLOCK_SHARED | XFS_IOLOCK_EXCL)); + xfs_assert_ilocked(dp, XFS_IOLOCK_SHARED | XFS_IOLOCK_EXCL); XFS_STATS_INC(dp->i_mount, xs_dir_getdents); args.dp = dp; diff --git a/fs/xfs/xfs_dquot.c b/fs/xfs/xfs_dquot.c index ac6ba646624d..4b2f1b82badc 100644 --- a/fs/xfs/xfs_dquot.c +++ b/fs/xfs/xfs_dquot.c @@ -949,7 +949,7 @@ xfs_qm_dqget_inode( if (error) return error; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(xfs_inode_dquot(ip, type) == NULL); id = xfs_qm_id_for_quotatype(ip, type); @@ -1006,7 +1006,7 @@ xfs_qm_dqget_inode( } dqret: - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); trace_xfs_dqget_miss(dqp); *O_dqpp = dqp; return 0; diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index e33e5e13b95f..632653e00906 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -879,7 +879,7 @@ xfs_break_dax_layouts( { struct page *page; - ASSERT(xfs_isilocked(XFS_I(inode), XFS_MMAPLOCK_EXCL)); + xfs_assert_ilocked(XFS_I(inode), XFS_MMAPLOCK_EXCL); page = dax_layout_busy_page(inode->i_mapping); if (!page) @@ -900,7 +900,7 @@ xfs_break_layouts( bool retry; int error; - ASSERT(xfs_isilocked(XFS_I(inode), XFS_IOLOCK_SHARED|XFS_IOLOCK_EXCL)); + xfs_assert_ilocked(XFS_I(inode), XFS_IOLOCK_SHARED | XFS_IOLOCK_EXCL); do { retry = false; diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index c0f1c89786c2..1ed6bed19bec 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -333,52 +333,26 @@ xfs_ilock_demote( trace_xfs_ilock_demote(ip, lock_flags, _RET_IP_); } -#if defined(DEBUG) || defined(XFS_WARN) -static inline bool -__xfs_rwsem_islocked( - struct rw_semaphore *rwsem, - bool shared) -{ - if (!debug_locks) - return rwsem_is_locked(rwsem); - - if (!shared) - return lockdep_is_held_type(rwsem, 0); - - /* - * We are checking that the lock is held at least in shared - * mode but don't care that it might be held exclusively - * (i.e. shared | excl). Hence we check if the lock is held - * in any mode rather than an explicit shared mode. - */ - return lockdep_is_held_type(rwsem, -1); -} - -bool -xfs_isilocked( +void +xfs_assert_ilocked( struct xfs_inode *ip, uint lock_flags) { - if (lock_flags & (XFS_ILOCK_EXCL|XFS_ILOCK_SHARED)) { - if (!(lock_flags & XFS_ILOCK_SHARED)) - return !!ip->i_lock.mr_writer; - return rwsem_is_locked(&ip->i_lock.mr_lock); - } - - if (lock_flags & (XFS_MMAPLOCK_EXCL|XFS_MMAPLOCK_SHARED)) { - return __xfs_rwsem_islocked(&VFS_I(ip)->i_mapping->invalidate_lock, - (lock_flags & XFS_MMAPLOCK_SHARED)); - } - - if (lock_flags & (XFS_IOLOCK_EXCL | XFS_IOLOCK_SHARED)) { - return __xfs_rwsem_islocked(&VFS_I(ip)->i_rwsem, - (lock_flags & XFS_IOLOCK_SHARED)); - } - - ASSERT(0); - return false; + if (lock_flags & XFS_ILOCK_SHARED) + rwsem_assert_held(&ip->i_lock.mr_lock); + else if (lock_flags & XFS_ILOCK_EXCL) + ASSERT(ip->i_lock.mr_writer); + + if (lock_flags & XFS_MMAPLOCK_SHARED) + rwsem_assert_held(&VFS_I(ip)->i_mapping->invalidate_lock); + else if (lock_flags & XFS_MMAPLOCK_EXCL) + rwsem_assert_held_write(&VFS_I(ip)->i_mapping->invalidate_lock); + + if (lock_flags & XFS_IOLOCK_SHARED) + rwsem_assert_held(&VFS_I(ip)->i_rwsem); + else if (lock_flags & XFS_IOLOCK_EXCL) + rwsem_assert_held_write(&VFS_I(ip)->i_rwsem); } -#endif /* * xfs_lockdep_subclass_ok() is only used in an ASSERT, so is only called when @@ -1342,9 +1316,9 @@ xfs_itruncate_extents_flags( xfs_filblks_t unmap_len; int error = 0; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); - ASSERT(!atomic_read(&VFS_I(ip)->i_count) || - xfs_isilocked(ip, XFS_IOLOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); + if (atomic_read(&VFS_I(ip)->i_count)) + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL); ASSERT(new_size <= XFS_ISIZE(ip)); ASSERT(tp->t_flags & XFS_TRANS_PERM_LOG_RES); ASSERT(ip->i_itemp != NULL); @@ -1605,7 +1579,7 @@ xfs_inactive_ifree( xfs_trans_ijoin(tp, ip, XFS_ILOCK_EXCL); error = xfs_ifree(tp, ip); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (error) { /* * If we fail to free the inode, shut down. The cancel @@ -2359,7 +2333,7 @@ xfs_ifree( struct xfs_inode_log_item *iip = ip->i_itemp; int error; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(VFS_I(ip)->i_nlink == 0); ASSERT(ip->i_df.if_nextents == 0); ASSERT(ip->i_disk_size == 0 || !S_ISREG(VFS_I(ip)->i_mode)); @@ -2428,7 +2402,7 @@ static void xfs_iunpin( struct xfs_inode *ip) { - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL|XFS_ILOCK_SHARED)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL | XFS_ILOCK_SHARED); trace_xfs_inode_unpin_nowait(ip, _RET_IP_); @@ -3189,7 +3163,7 @@ xfs_iflush( struct xfs_mount *mp = ip->i_mount; int error; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL|XFS_ILOCK_SHARED)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL|XFS_ILOCK_SHARED); ASSERT(xfs_iflags_test(ip, XFS_IFLUSHING)); ASSERT(ip->i_df.if_format != XFS_DINODE_FMT_BTREE || ip->i_df.if_nextents > XFS_IFORK_MAXEXT(ip, XFS_DATA_FORK)); diff --git a/fs/xfs/xfs_inode.h b/fs/xfs/xfs_inode.h index 3dc47937da5d..dd8b8339ba1b 100644 --- a/fs/xfs/xfs_inode.h +++ b/fs/xfs/xfs_inode.h @@ -523,7 +523,7 @@ void xfs_ilock(xfs_inode_t *, uint); int xfs_ilock_nowait(xfs_inode_t *, uint); void xfs_iunlock(xfs_inode_t *, uint); void xfs_ilock_demote(xfs_inode_t *, uint); -bool xfs_isilocked(struct xfs_inode *, uint); +void xfs_assert_ilocked(struct xfs_inode *, uint); uint xfs_ilock_data_map_shared(struct xfs_inode *); uint xfs_ilock_attr_map_shared(struct xfs_inode *); diff --git a/fs/xfs/xfs_inode_item.c b/fs/xfs/xfs_inode_item.c index cd7803fda8b1..3e125be356c1 100644 --- a/fs/xfs/xfs_inode_item.c +++ b/fs/xfs/xfs_inode_item.c @@ -649,7 +649,7 @@ xfs_inode_item_pin( { struct xfs_inode *ip = INODE_ITEM(lip)->ili_inode; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(lip->li_buf); trace_xfs_inode_pin(ip, _RET_IP_); @@ -755,7 +755,7 @@ xfs_inode_item_release( unsigned short lock_flags; ASSERT(ip->i_itemp != NULL); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); lock_flags = iip->ili_lock_flags; iip->ili_lock_flags = 0; diff --git a/fs/xfs/xfs_iops.c b/fs/xfs/xfs_iops.c index fdfda4fba12b..e711668d601b 100644 --- a/fs/xfs/xfs_iops.c +++ b/fs/xfs/xfs_iops.c @@ -796,8 +796,7 @@ xfs_setattr_size( uint lock_flags = 0; bool did_zeroing = false; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL)); - ASSERT(xfs_isilocked(ip, XFS_MMAPLOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_MMAPLOCK_EXCL); ASSERT(S_ISREG(inode->i_mode)); ASSERT((iattr->ia_valid & (ATTR_UID|ATTR_GID|ATTR_ATIME|ATTR_ATIME_SET| ATTR_MTIME_SET|ATTR_TIMES_SET)) == 0); diff --git a/fs/xfs/xfs_qm.c b/fs/xfs/xfs_qm.c index 94a7932ac570..45cf9a557eb9 100644 --- a/fs/xfs/xfs_qm.c +++ b/fs/xfs/xfs_qm.c @@ -254,7 +254,7 @@ xfs_qm_dqattach_one( struct xfs_dquot *dqp; int error; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); error = 0; /* @@ -322,7 +322,7 @@ xfs_qm_dqattach_locked( if (!xfs_qm_need_dqattach(ip)) return 0; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (XFS_IS_UQUOTA_ON(mp) && !ip->i_udquot) { error = xfs_qm_dqattach_one(ip, XFS_DQTYPE_USER, @@ -353,7 +353,7 @@ xfs_qm_dqattach_locked( * Don't worry about the dquots that we may have attached before any * error - they'll get detached later if it has not already been done. */ - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); return error; } @@ -1809,7 +1809,7 @@ xfs_qm_vop_chown( XFS_TRANS_DQ_RTBCOUNT : XFS_TRANS_DQ_BCOUNT; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(XFS_IS_QUOTA_ON(ip->i_mount)); /* old dquot */ @@ -1897,7 +1897,7 @@ xfs_qm_vop_create_dqattach( if (!XFS_IS_QUOTA_ON(mp)) return; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (udqp && XFS_IS_UQUOTA_ON(mp)) { ASSERT(ip->i_udquot == NULL); diff --git a/fs/xfs/xfs_reflink.c b/fs/xfs/xfs_reflink.c index 658edee8381d..d10eba9548dc 100644 --- a/fs/xfs/xfs_reflink.c +++ b/fs/xfs/xfs_reflink.c @@ -527,7 +527,7 @@ xfs_reflink_allocate_cow( int error; bool found; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (!ip->i_cowfp) { ASSERT(!xfs_is_reflink_inode(ip)); xfs_ifork_init_cow(ip); diff --git a/fs/xfs/xfs_rtalloc.c b/fs/xfs/xfs_rtalloc.c index 88c48de5c9c8..5e5eedff71c9 100644 --- a/fs/xfs/xfs_rtalloc.c +++ b/fs/xfs/xfs_rtalloc.c @@ -1182,7 +1182,7 @@ xfs_rtallocate_extent( int error; /* error value */ xfs_rtxnum_t r; /* result allocated rtext */ - ASSERT(xfs_isilocked(args.mp->m_rbmip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(args.mp->m_rbmip, XFS_ILOCK_EXCL); ASSERT(minlen > 0 && minlen <= maxlen); /* @@ -1425,7 +1425,7 @@ xfs_rtpick_extent( uint64_t seq; /* sequence number of file creation */ struct timespec64 ts; /* timespec in inode */ - ASSERT(xfs_isilocked(mp->m_rbmip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(mp->m_rbmip, XFS_ILOCK_EXCL); ts = inode_get_atime(VFS_I(mp->m_rbmip)); if (!(mp->m_rbmip->i_diflags & XFS_DIFLAG_NEWRTBM)) { diff --git a/fs/xfs/xfs_symlink.c b/fs/xfs/xfs_symlink.c index 85e433df6a3f..2ca157de4a73 100644 --- a/fs/xfs/xfs_symlink.c +++ b/fs/xfs/xfs_symlink.c @@ -43,7 +43,7 @@ xfs_readlink_bmap_ilocked( int fsblocks = 0; int offset; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL); fsblocks = xfs_symlink_blocks(mp, pathlen); error = xfs_bmapi_read(ip, 0, fsblocks, mval, &nmaps, 0); diff --git a/fs/xfs/xfs_trans_dquot.c b/fs/xfs/xfs_trans_dquot.c index aa00cf67ad72..9c159d016ecf 100644 --- a/fs/xfs/xfs_trans_dquot.c +++ b/fs/xfs/xfs_trans_dquot.c @@ -796,7 +796,7 @@ xfs_trans_reserve_quota_nblks( return 0; ASSERT(!xfs_is_quota_inode(&mp->m_sb, ip->i_ino)); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (force) qflags |= XFS_QMOPT_FORCE_RES; From patchwork Fri Nov 17 14:51:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13459014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC13BC47071 for ; Fri, 17 Nov 2023 14:52:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 333B86B04DC; Fri, 17 Nov 2023 09:52:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2E33B6B04DB; Fri, 17 Nov 2023 09:52:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 135206B04E0; Fri, 17 Nov 2023 09:52:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id EA9B56B04DC for ; Fri, 17 Nov 2023 09:52:04 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id C36241A0144 for ; Fri, 17 Nov 2023 14:52:04 +0000 (UTC) X-FDA: 81467736168.23.C9BEB61 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf21.hostedemail.com (Postfix) with ESMTP id 034411C0016 for ; Fri, 17 Nov 2023 14:52:02 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=YG8Z8q53; dmarc=none; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700232723; a=rsa-sha256; cv=none; b=RzX3DCmJvsqMSSfdWJXAR9iA9UKxi8tdotcr21guqC8FYNt1Ik/vcZiUOmcB6EA7AAktO8 cM7cIUSTIV9Y2wK+McjJSBn4Je9hzupdJMW7fs+ypqE682PpMlKwn8Z500xxvHHkomntv3 TfBw7YbQnyJY3KJI3fkjM4QRbamCfJI= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=YG8Z8q53; dmarc=none; spf=none (imf21.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700232723; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xbcUpDlbQzcZtzSAWcv3Q7LJrTanIECAc3h72m1YlgE=; b=IyRBZHjTOasizfY+2OFU0/fMJ0gEhh4VLNmgoL8q2aQnW/2Bie5Mpj+6q2Q9XfjolfmGcP sR3+gx6rbeTKG5GgUl3F95fACV2rV+9u8YZ0sRJFbriOw68Q0ATB4eBCAh307puc/7HOyp DB5ZQh0TYuipwEzqWE1+7DVOlKjKNOM= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=xbcUpDlbQzcZtzSAWcv3Q7LJrTanIECAc3h72m1YlgE=; b=YG8Z8q53nFvlTT1i5HTb7FSPvT tOurzVUWVOIZM6sXYGsXeluj9sSjVhsqqCccY1fK574M8KKH47ZlvWDdMTLvaN274V1eKlJ4CBBsK 2AwLvC0X+WIsV6Q93Wapc4vR6uGITHw4OoxDDmJZYZrS8qup2KU5X0pDhjgBYa+Smdc3wI8gQd/5c p8P4ZAF/x+FMGggLmvbwBbZMTovB8J++AYDIGYA5LSRL834xUpyeZCOS8I/H4gSJlIsUhoBI3aqoW B0gfLDIWFJnDFHP1zacyv5Gp4Sxj0FLcdmmB9O88KksY6RAXT18J/CduPplrtQCy+7g6MgC3zGbZb pH5XtzIw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1r40C8-009yq3-2D; Fri, 17 Nov 2023 14:51:44 +0000 From: "Matthew Wilcox (Oracle)" To: Peter Zijlstra , Ingo Molnar , Will Deacon , Waiman Long Cc: "Matthew Wilcox (Oracle)" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Chandan Babu R , "Darrick J . Wong" , linux-xfs@vger.kernel.org, Mateusz Guzik Subject: [PATCH v4 4/4] xfs: Remove mrlock wrapper Date: Fri, 17 Nov 2023 14:51:42 +0000 Message-Id: <20231117145142.2378800-5-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231117145142.2378800-1-willy@infradead.org> References: <20231117145142.2378800-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 034411C0016 X-Stat-Signature: hk5fizqw818c6ncp9m5ofiex8yjmbb7x X-HE-Tag: 1700232722-742142 X-HE-Meta: U2FsdGVkX1/CByjgTtyB2mhkglUR+RhY3qFOew/4kMlOTXEMWGGo7UcAW0ncJxYu5kFMETR2O+q2PHjClLBVbHHLjtSJ8xbe/qPBOvP4XCL9ShUFiytZpuD6OcJe6z5yojkNr8pSEcJ/cOtyB6PNHDrVJUD4omtSkfMX1u62oj7imtuHfgHaDeX4Tbp/PRlFWyXtAosPP+OtBhXJGUn5jT2HVq982hNQ3MFPadg9sc0vwWtvawo9835Co7RETngqPXCR19nBvk0sNLFhdLdpHPX0p2cCcEVmNqLZRq7qPkwwHo5exsTSyjzMhWltCiKSt4UhZ58Gu/HXrfr48cEl78gSVmcHCJsYUz+PhtMpB3eCCwevOqYcQsgqsxJF2YuG9V3ed3yACQBwBwvShB2/kwibbQUzE8xMehDsPUH0GYYDAgXd3lijOUjF2fDoaUKlhx9ubftm71S+lh40gkjCAA/EIvn5T4LAz1o7syOqx0oXdeKflPQWGiyMhilb9RLM5GZS1/yIJVvgwfUGXsMWeKOTIKpYixt/3RgHcqEXMRy/N3uVGkkQ1D+LwpHWw6Iu7xdjzpxyJP0yj6LDW/isEYCxS4WJ+ijb1Rgq9f1Wl8+O5Tkax+CDBj4XyDgACo9huTQrhLhVzFx/uErD0r4YkvOIhzKO69a7359QeZk7aSG+MRUIR4v2vJmP0CfYR9aUvbDsLSXtLo/7vmeq3/2zgG4QMnOEYFcMMlfg4fb20G2BsI+rYiRsZKKFYlW43nQTayCCb5FSCWp4d3zlLj1qWa0v52FiEuFSF839BfoU0NHAiQhLaPvA/GKtljCOUmqqIvYPAi/C6Rwu6NHAFEW4XiWnBGqcYNOS/x/oOqcF9mL0NqCqPRTQnj3wOLQKlidbkrrlwIsEieOCdezsIjW/hbGz6OYzEopAwA4QuNbFxUb/D/0/S0TMD5cwVyl7HFxZIJzirS80aXGezU6vp0R 9GD9tGl8 wyami69KlmRaPN5SNKQF3n6Ufmd1K4eO5d0o19TtQyWO5uqDN/axFdaeh/eAmO4zfqWBHyAJPfjfSNt575tj5WTX1TPfvY2PFGhQUgRrqZCIzEQ4igseGsw9TPZJM8UG+D/vyo1S3xJENrqyBdX56vnPYTx25vDcjMXCG+y/SBLTZOtbT5zYpVclfow== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: mrlock was an rwsem wrapper that also recorded whether the lock was held for read or write. Now that we can ask the generic code whether the lock is held for read or write, we can remove this wrapper and use an rwsem directly. As the comment says, we can't use lockdep to assert that the ILOCK is held for write, because we might be in a workqueue, and we aren't able to tell lockdep that we do in fact own the lock. Signed-off-by: Matthew Wilcox (Oracle) --- fs/xfs/mrlock.h | 78 ---------------------------------------------- fs/xfs/xfs_inode.c | 22 +++++++------ fs/xfs/xfs_inode.h | 2 +- fs/xfs/xfs_iops.c | 4 +-- fs/xfs/xfs_linux.h | 2 +- fs/xfs/xfs_super.c | 4 +-- 6 files changed, 18 insertions(+), 94 deletions(-) delete mode 100644 fs/xfs/mrlock.h diff --git a/fs/xfs/mrlock.h b/fs/xfs/mrlock.h deleted file mode 100644 index 79155eec341b..000000000000 --- a/fs/xfs/mrlock.h +++ /dev/null @@ -1,78 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Copyright (c) 2000-2006 Silicon Graphics, Inc. - * All Rights Reserved. - */ -#ifndef __XFS_SUPPORT_MRLOCK_H__ -#define __XFS_SUPPORT_MRLOCK_H__ - -#include - -typedef struct { - struct rw_semaphore mr_lock; -#if defined(DEBUG) || defined(XFS_WARN) - int mr_writer; -#endif -} mrlock_t; - -#if defined(DEBUG) || defined(XFS_WARN) -#define mrinit(mrp, name) \ - do { (mrp)->mr_writer = 0; init_rwsem(&(mrp)->mr_lock); } while (0) -#else -#define mrinit(mrp, name) \ - do { init_rwsem(&(mrp)->mr_lock); } while (0) -#endif - -#define mrlock_init(mrp, t,n,s) mrinit(mrp, n) -#define mrfree(mrp) do { } while (0) - -static inline void mraccess_nested(mrlock_t *mrp, int subclass) -{ - down_read_nested(&mrp->mr_lock, subclass); -} - -static inline void mrupdate_nested(mrlock_t *mrp, int subclass) -{ - down_write_nested(&mrp->mr_lock, subclass); -#if defined(DEBUG) || defined(XFS_WARN) - mrp->mr_writer = 1; -#endif -} - -static inline int mrtryaccess(mrlock_t *mrp) -{ - return down_read_trylock(&mrp->mr_lock); -} - -static inline int mrtryupdate(mrlock_t *mrp) -{ - if (!down_write_trylock(&mrp->mr_lock)) - return 0; -#if defined(DEBUG) || defined(XFS_WARN) - mrp->mr_writer = 1; -#endif - return 1; -} - -static inline void mrunlock_excl(mrlock_t *mrp) -{ -#if defined(DEBUG) || defined(XFS_WARN) - mrp->mr_writer = 0; -#endif - up_write(&mrp->mr_lock); -} - -static inline void mrunlock_shared(mrlock_t *mrp) -{ - up_read(&mrp->mr_lock); -} - -static inline void mrdemote(mrlock_t *mrp) -{ -#if defined(DEBUG) || defined(XFS_WARN) - mrp->mr_writer = 0; -#endif - downgrade_write(&mrp->mr_lock); -} - -#endif /* __XFS_SUPPORT_MRLOCK_H__ */ diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index 1ed6bed19bec..7661084ca568 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -208,9 +208,9 @@ xfs_ilock( } if (lock_flags & XFS_ILOCK_EXCL) - mrupdate_nested(&ip->i_lock, XFS_ILOCK_DEP(lock_flags)); + down_write_nested(&ip->i_lock, XFS_ILOCK_DEP(lock_flags)); else if (lock_flags & XFS_ILOCK_SHARED) - mraccess_nested(&ip->i_lock, XFS_ILOCK_DEP(lock_flags)); + down_read_nested(&ip->i_lock, XFS_ILOCK_DEP(lock_flags)); } /* @@ -251,10 +251,10 @@ xfs_ilock_nowait( } if (lock_flags & XFS_ILOCK_EXCL) { - if (!mrtryupdate(&ip->i_lock)) + if (!down_write_trylock(&ip->i_lock)) goto out_undo_mmaplock; } else if (lock_flags & XFS_ILOCK_SHARED) { - if (!mrtryaccess(&ip->i_lock)) + if (!down_read_trylock(&ip->i_lock)) goto out_undo_mmaplock; } return 1; @@ -303,9 +303,9 @@ xfs_iunlock( up_read(&VFS_I(ip)->i_mapping->invalidate_lock); if (lock_flags & XFS_ILOCK_EXCL) - mrunlock_excl(&ip->i_lock); + up_write(&ip->i_lock); else if (lock_flags & XFS_ILOCK_SHARED) - mrunlock_shared(&ip->i_lock); + up_read(&ip->i_lock); trace_xfs_iunlock(ip, lock_flags, _RET_IP_); } @@ -324,7 +324,7 @@ xfs_ilock_demote( ~(XFS_IOLOCK_EXCL|XFS_MMAPLOCK_EXCL|XFS_ILOCK_EXCL)) == 0); if (lock_flags & XFS_ILOCK_EXCL) - mrdemote(&ip->i_lock); + downgrade_write(&ip->i_lock); if (lock_flags & XFS_MMAPLOCK_EXCL) downgrade_write(&VFS_I(ip)->i_mapping->invalidate_lock); if (lock_flags & XFS_IOLOCK_EXCL) @@ -338,10 +338,14 @@ xfs_assert_ilocked( struct xfs_inode *ip, uint lock_flags) { + /* + * Sometimes we assert the ILOCK is held exclusively, but we're in + * a workqueue, so lockdep doesn't know we're the owner. + */ if (lock_flags & XFS_ILOCK_SHARED) - rwsem_assert_held(&ip->i_lock.mr_lock); + rwsem_assert_held(&ip->i_lock); else if (lock_flags & XFS_ILOCK_EXCL) - ASSERT(ip->i_lock.mr_writer); + rwsem_assert_held_write_nolockdep(&ip->i_lock); if (lock_flags & XFS_MMAPLOCK_SHARED) rwsem_assert_held(&VFS_I(ip)->i_mapping->invalidate_lock); diff --git a/fs/xfs/xfs_inode.h b/fs/xfs/xfs_inode.h index dd8b8339ba1b..3cade5903fda 100644 --- a/fs/xfs/xfs_inode.h +++ b/fs/xfs/xfs_inode.h @@ -39,7 +39,7 @@ typedef struct xfs_inode { /* Transaction and locking information. */ struct xfs_inode_log_item *i_itemp; /* logging information */ - mrlock_t i_lock; /* inode lock */ + struct rw_semaphore i_lock; /* inode lock */ atomic_t i_pincount; /* inode pin count */ struct llist_node i_gclist; /* deferred inactivation list */ diff --git a/fs/xfs/xfs_iops.c b/fs/xfs/xfs_iops.c index e711668d601b..79010d60ee21 100644 --- a/fs/xfs/xfs_iops.c +++ b/fs/xfs/xfs_iops.c @@ -1284,9 +1284,9 @@ xfs_setup_inode( */ lockdep_set_class(&inode->i_rwsem, &inode->i_sb->s_type->i_mutex_dir_key); - lockdep_set_class(&ip->i_lock.mr_lock, &xfs_dir_ilock_class); + lockdep_set_class(&ip->i_lock, &xfs_dir_ilock_class); } else { - lockdep_set_class(&ip->i_lock.mr_lock, &xfs_nondir_ilock_class); + lockdep_set_class(&ip->i_lock, &xfs_nondir_ilock_class); } /* diff --git a/fs/xfs/xfs_linux.h b/fs/xfs/xfs_linux.h index d7873e0360f0..ec3c6c138a63 100644 --- a/fs/xfs/xfs_linux.h +++ b/fs/xfs/xfs_linux.h @@ -22,7 +22,6 @@ typedef __u32 xfs_nlink_t; #include "xfs_types.h" #include "kmem.h" -#include "mrlock.h" #include #include @@ -51,6 +50,7 @@ typedef __u32 xfs_nlink_t; #include #include #include +#include #include #include #include diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c index 764304595e8b..cd0200ed51f0 100644 --- a/fs/xfs/xfs_super.c +++ b/fs/xfs/xfs_super.c @@ -724,9 +724,7 @@ xfs_fs_inode_init_once( /* xfs inode */ atomic_set(&ip->i_pincount, 0); spin_lock_init(&ip->i_flags_lock); - - mrlock_init(&ip->i_lock, MRLOCK_ALLOW_EQUAL_PRI|MRLOCK_BARRIER, - "xfsino", ip->i_ino); + init_rwsem(&ip->i_lock); } /*