From patchwork Sat Oct 7 20:35:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13412476 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10E5CE95A6B for ; Sat, 7 Oct 2023 20:36:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 07C568D0001; Sat, 7 Oct 2023 16:36:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F36168D0013; Sat, 7 Oct 2023 16:36:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DFC438D0012; Sat, 7 Oct 2023 16:36:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id D212F8D0001 for ; Sat, 7 Oct 2023 16:36:02 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 90B4514028C for ; Sat, 7 Oct 2023 20:36:02 +0000 (UTC) X-FDA: 81319822164.22.E0E0B2F Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf24.hostedemail.com (Postfix) with ESMTP id 000C818001F for ; Sat, 7 Oct 2023 20:36:00 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=WDbKmpqH; dmarc=none; spf=none (imf24.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696710961; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gK2PkVQedGYkiXZ+S6dDW/QdpinQQuQZI6/6Ogj39CE=; b=vmiB8vKM86Wd3jK1O8woyv/R9seVzT++v+7IBx7JE8mEuyjJN1u1H4gCsaQrZTxpZhNDSf 3A+WbTu/FGG+hAG5a6Lsd0fcdih72cs+bFkWXVNqt8FSzWP6s3F6qIicsUcrwYM8fPTxI6 OWsjOG9kVs2BVcEJj6rwEB039UVbFqU= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=WDbKmpqH; dmarc=none; spf=none (imf24.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696710961; a=rsa-sha256; cv=none; b=j4NDsRtf8uBs5OrW/UUWAN2kzrieRYKGkKd08sebavcjEWiI5oRY5HW4YrUCUY/fYj6VU9 KcCNm1KJCqG+3W9pXrXwkJ2oJmyMYflvdbBmv4B1c4h8NreOfiiAGONaIvaASfH41CZZBO XntxW+HClnP133FObUgbPrFAZQvNfVM= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=gK2PkVQedGYkiXZ+S6dDW/QdpinQQuQZI6/6Ogj39CE=; b=WDbKmpqHsfQKSeC/TKnzhmuJiI XlF8tSu64opY8lCrqTKdNZn4EJxfi2q/PChuCviUe9nlITFxCNCS8H6sg23IVfp1KEV4oa3Uabg4v k/8SyjPEtT9LSef6M4Fq2ekcazpitwdP4ZUd/vMn5CV1MqbJnh/xZvWaVpuiRjU3ljPr6twQONFF8 NPVe2/HFVL0mnVB8pEWRl6UlGUioGDu/ZwK+lDNBHqEMNs0Gu5XNP0n60hTF12hgDjq5gOsVl8Smz Ha3qXka1ZbBhtaqJ99sTPBCiH6gvEXWtXloFKTA0k3TS48IMtX2jSAs2ZqNaDwUuU5DyZDgB5P+kr f9RhCEsQ==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qpE1Y-005mLA-Ih; Sat, 07 Oct 2023 20:35:44 +0000 From: "Matthew Wilcox (Oracle)" To: Peter Zijlstra , Ingo Molnar , Will Deacon , Waiman Long Cc: "Matthew Wilcox (Oracle)" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Chandan Babu R , "Darrick J . Wong" , linux-xfs@vger.kernel.org, Mateusz Guzik Subject: [PATCH v2 1/5] locking: Add rwsem_assert_held() and rwsem_assert_held_write() Date: Sat, 7 Oct 2023 21:35:39 +0100 Message-Id: <20231007203543.1377452-2-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231007203543.1377452-1-willy@infradead.org> References: <20231007203543.1377452-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 000C818001F X-Stat-Signature: 6qiyigbunbq167qpcgefnprgufw3qqqq X-Rspam-User: X-HE-Tag: 1696710960-172896 X-HE-Meta: U2FsdGVkX1+nVfb6paK9U65CdjJE3a/lysdIH1p78zaBS04G7Bq20pco5B9bE18RMN4OeEzIQKuKkd1cEyO2P3GaKJxoLZb7vAW9zvpFvA+xUtPxsl7mPGq8p3pLHPFKMuOOLh/jJfS6mmkPftN5zmYlD/IXsbAvt+fn+gGn4fISSP9klSwFnrWYUj1ZfbeVYme3v8FguhRxK0WwMYb6ydFCV1PG+FaqvCKnWybNJMJAkhaCSZ8E/anfqBHZjFQdVnL4hXlkbaxV/m5jz9zA/PtzTBo8GcHYQLzPfHXsy84NyXm4P2G6llVL4Op0kdY9SPQ0c9Dz4hw95SwznUe6kcjnoavyYEPu8V8gesX1ELqLcPiNmpDnufC/HO/Weyd+TehRlI/AqlpC/kO7tG9fKtTIMpxbo5QRGoOtwJVrQMvYE2/9twJ5jHhKkZs6WrhVkJXkBA8ath1UQ5MrnJrbx1ebeOJ1DWjWQvI0YI1jac3XhG3gXJfVOoI8RQNAY2gdz6M5gsCwKZvUOFSvOhFW17B/r7hx0m5hUR/gIrGRvDuXHhRQP5TbboCwJzYOzj+g+2tzDG+b/gHEXsYB54n5p6b/Mph3c9RGjWcrb6/bekUO8hzWgdGN5kbBrU6NNBH92XNvfPCqodZd/16nGUE/E3o1mMO+iLRG6VSgoVKxC3/byI2P/YYeMHe/B5qdHtqPHMvYqLqhor5ZhyJmJNNPXbd2bwoUT+nyA7Cg1cUp+hC3UA1+Cx44E/3J25biXNopo3fJgNTPPfYLkgXOuOLTmirppajferFAwJoRefX81HiBB4SWaHVD0E5eOXUTYaqmyXe5/2Ew7AG6YDgGIrNpaRmsz+7ezg8Q3BYfNDMEEv/VljCbMepuHBtCWsQ/CWjhSD/CIUQ0GdLBeeWN2DJ3C3h9Vx9s6OBZOLfEj1Qjv3w/HgCtFi+/mNo8eqrWSO67TzSM8aTtUZFrisAtrHy Ma2c5Msm PiYSU0ZIHIpF6SalfXD+Jh6kbjVRLDhG+V/q896c2gTgfNwcyu+IQypHPGe143oet+c0B0lI+zBT6gcqiG0dxR9I2LVXlwfoKoYKj6tvvTtr4Gaey3mX+ShDisGMbd6Bp/f4P1Z8rCXVdxE9CovfpbLyA2QMgM63HShk3C4vBrciJC083Fu6hARN8GSGK1nTibxKZzE4CODn0IvXlmRbNvlC6vlIrHhNFTCh2OgiqfMhEsHw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Modelled after lockdep_assert_held() and lockdep_assert_held_write(), but are always active, even when lockdep is disabled. Of course, they don't test that _this_ thread is the owner, but it's sufficient to catch many bugs and doesn't incur the same performance penalty as lockdep. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/rwbase_rt.h | 9 +++++++-- include/linux/rwsem.h | 42 ++++++++++++++++++++++++++++++++++----- 2 files changed, 44 insertions(+), 7 deletions(-) diff --git a/include/linux/rwbase_rt.h b/include/linux/rwbase_rt.h index 1d264dd08625..a04acd85705b 100644 --- a/include/linux/rwbase_rt.h +++ b/include/linux/rwbase_rt.h @@ -26,12 +26,17 @@ struct rwbase_rt { } while (0) -static __always_inline bool rw_base_is_locked(struct rwbase_rt *rwb) +static __always_inline bool rw_base_is_locked(const struct rwbase_rt *rwb) { return atomic_read(&rwb->readers) != READER_BIAS; } -static __always_inline bool rw_base_is_contended(struct rwbase_rt *rwb) +static inline void rw_base_assert_held_write(const struct rwbase_rt *rwb) +{ + BUG_ON(atomic_read(&rwb->readers) != WRITER_BIAS); +} + +static __always_inline bool rw_base_is_contended(const struct rwbase_rt *rwb) { return atomic_read(&rwb->readers) > 0; } diff --git a/include/linux/rwsem.h b/include/linux/rwsem.h index 1dd530ce8b45..048149f781b3 100644 --- a/include/linux/rwsem.h +++ b/include/linux/rwsem.h @@ -66,14 +66,24 @@ struct rw_semaphore { #endif }; -/* In all implementations count != 0 means locked */ +#define RWSEM_UNLOCKED_VALUE 0UL +#define RWSEM_WRITER_LOCKED (1UL << 0) +#define __RWSEM_COUNT_INIT(name) .count = ATOMIC_LONG_INIT(RWSEM_UNLOCKED_VALUE) + static inline int rwsem_is_locked(struct rw_semaphore *sem) { - return atomic_long_read(&sem->count) != 0; + return atomic_long_read(&sem->count) != RWSEM_UNLOCKED_VALUE; } -#define RWSEM_UNLOCKED_VALUE 0L -#define __RWSEM_COUNT_INIT(name) .count = ATOMIC_LONG_INIT(RWSEM_UNLOCKED_VALUE) +static inline void __rwsem_assert_held(const struct rw_semaphore *sem) +{ + WARN_ON(atomic_long_read(&sem->count) == RWSEM_UNLOCKED_VALUE); +} + +static inline void __rwsem_assert_held_write(const struct rw_semaphore *sem) +{ + WARN_ON(!(atomic_long_read(&sem->count) & RWSEM_WRITER_LOCKED)); +} /* Common initializer macros and functions */ @@ -152,11 +162,21 @@ do { \ __init_rwsem((sem), #sem, &__key); \ } while (0) -static __always_inline int rwsem_is_locked(struct rw_semaphore *sem) +static __always_inline int rwsem_is_locked(const struct rw_semaphore *sem) { return rw_base_is_locked(&sem->rwbase); } +static inline void __rwsem_assert_held(const struct rw_semaphore *sem) +{ + BUG_ON(!rwsem_is_locked(sem)); +} + +static inline void __rwsem_assert_held_write(const struct rw_semaphore *sem) +{ + rw_base_assert_held_write(sem); +} + static __always_inline int rwsem_is_contended(struct rw_semaphore *sem) { return rw_base_is_contended(&sem->rwbase); @@ -169,6 +189,18 @@ static __always_inline int rwsem_is_contended(struct rw_semaphore *sem) * the RT specific variant. */ +static inline void rwsem_assert_held(const struct rw_semaphore *sem) +{ + lockdep_assert_held(sem); + __rwsem_assert_held(sem); +} + +static inline void rwsem_assert_held_write(const struct rw_semaphore *sem) +{ + lockdep_assert_held_write(sem); + __rwsem_assert_held_write(sem); +} + /* * lock for reading */ From patchwork Sat Oct 7 20:35:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13412480 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 496BAE95A6D for ; Sat, 7 Oct 2023 20:36:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D13D68D0016; Sat, 7 Oct 2023 16:36:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CC3E08D0013; Sat, 7 Oct 2023 16:36:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD9EC8D0016; Sat, 7 Oct 2023 16:36:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A6FD18D0013 for ; Sat, 7 Oct 2023 16:36:17 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 7E720A0297 for ; Sat, 7 Oct 2023 20:36:17 +0000 (UTC) X-FDA: 81319822794.06.85F2478 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf02.hostedemail.com (Postfix) with ESMTP id E2D0680004 for ; Sat, 7 Oct 2023 20:36:15 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Cr3o8RWM; spf=none (imf02.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696710976; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TkvZj289LjVT0INkRek2vP8VRKU8xKxNyMII+enlEWg=; b=g1F2nyCy460GZDX9vPYpzIzEBqrzXWiMmPxMyAXmqRnfA9huxurWFUd7ahwVZGRjrD58io 95tFXG0b4JtU7Hg9vSJtgvgkvaYAgLD5tQiOVrBayasNV3o7wrHyGdhS0SiX+nIbTo62yq 9L/03cZL7YHPLgKjl/sawUrQ7n/+Ov0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696710976; a=rsa-sha256; cv=none; b=ypFaeqs+lFsbIFzyjAtcupip5AVEnGoRzDk8IHdxIz+/1RJYElkj17Ius9M3WFl/HZhsZc nUD25UhornP/MndMlqnV14DL2D5ecoPyle1C+2SB+V+uHKVzRs0SlCEmtAfnrEeiXpWH2w zix4GyoBtXdpf3x/cx2v9Mvz6Sp50zs= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Cr3o8RWM; spf=none (imf02.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=TkvZj289LjVT0INkRek2vP8VRKU8xKxNyMII+enlEWg=; b=Cr3o8RWM58gE0W4E+dSE8SE0UL Xdcas8AZH3bSf8486FYpIVyUco+VLPvf8wArWBxyCMYcnkQZ5EHF6gBsxnToFtBVVVJGVweSpnVXG pVd5zoHUthMI/p+Fw5DlYC8zihzCaba7WMrYWvlsg35gA9HJoxVD5QlfNdxIbV6B49ZS27saqgbkn m0pDKlI9KhEULZMOZCA2a44zVKUNvfOAi/6aiqRxOCvy+CLRBqDu5K+/CSPv/tCHzTULgIAgcedI+ 8Oc/wngAVrGkFfEWscWUjCALfTLekMn69t9FAP5LxXoUB+utuJ1/nJjrk2DNbLHtZQDB8SnVbkP0O RMOIsMmw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qpE1Y-005mLC-LZ; Sat, 07 Oct 2023 20:35:44 +0000 From: "Matthew Wilcox (Oracle)" To: Peter Zijlstra , Ingo Molnar , Will Deacon , Waiman Long Cc: "Matthew Wilcox (Oracle)" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Chandan Babu R , "Darrick J . Wong" , linux-xfs@vger.kernel.org, Mateusz Guzik Subject: [PATCH v2 2/5] mm: Use rwsem assertion macros for mmap_lock Date: Sat, 7 Oct 2023 21:35:40 +0100 Message-Id: <20231007203543.1377452-3-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231007203543.1377452-1-willy@infradead.org> References: <20231007203543.1377452-1-willy@infradead.org> MIME-Version: 1.0 X-Stat-Signature: zkrckwj81oc4opgy3m5ybz4j43xaa1yr X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: E2D0680004 X-Rspam-User: X-HE-Tag: 1696710975-314758 X-HE-Meta: U2FsdGVkX18KYc68Xwy5Y9szCluDAeQ0ckp85EwEuBwInERuu+MdO1mEJkODsr4vJ7xZm9lvVTkLJhaDwFkUvppp5AxKfycwfcWk43CvBXQP1et4g3WP41AxDKMC8+zb4wjkp6Gj7SYnWUReJWD6tLUJpMIX006mIZK01aqnLmzZ1rDzCUZhqnRi1SJfGZcWAT4woiRZs496vIHOhnu59DyiovQFS5I8FhJm5mJNJ0I+Lqpz5q9SOKbBJgn22s1Znqb+3IK5dmX2q4VNPFLhNDi+o5L/rTIc/GgHXFvJv3FYY/FNbNDAQIy9cbeLQgX3hkg5Adsn6mRfo7RLKxgspbtnLYU5TKVfsoIAVknMCtvv2SWJgHftIOpZtwqkjQ3ceVEob98up6ZWIJZiZdOsvAXXzacP+aZcF+DWQtRxfVdDs4GuTx2lauqKbEcviiEUlXlTmdJp+X7sWiJzz9VYqpotZ+2Krwzi0YQmdp2j1i6bCUHHfS98aUyaeYYd9RQtc/lG8vLqDy3VNoVpml50da5l2AAeyfEKkqE7Ntb3ilBvyhnfOUTgDj3qDOklUyRA3QmShEdfcBIZ+xpSRp8hGicZL6D4n953Sjho0WiF8OyIHcKp4ALN71GAo58tALjqLBCNObrR4203UquqEQA1zN4JasDT7LhA2e1/zfKo5xV5TDTSK6bMmYmazvUPP42qAQoETZ6bhr3m+SbON2iMDbPZ1Vu5NwPj6IBeZ9H5hfGPrQFK+MVZ5fzsgcMsCFjwkRYQg8yJX9drxDsLu4ldtE640bXlPKX35I0zmiTWwuRi7563wGWUrx/UUdk3g39j9onFvrw3pxQ9MbR0N7n3adPphWa0PPRvcigV7RzWFoDRyJy4J9R4lSaTyZg0Y2dqWgFYFCY6Q9wBiNFWNCtDwnm8LyafCxwhZgz03DDWr4WRnXu2dSHV4zyjCXZedaUrEZ1nktVqkNSTlsOa6AJ 5DG7IG1R FwZqZnmRE1Z17/UgSf2uMQMesF9z/uFqj+k4W9oWemcutNaT0ywHftVmXwKEzElz5cp/OdITM7u8zVdTXr3Wo1D7aYmR0jn32iOUOi8oudTEdpEqXCY6nqxLI8vRFN/zHGHzTYMMIItMCPLHpYHkTgN/oJgrcZgWRtPJt4rsmqZwtm5yKrw6lScMhqvjY55Iu2F4Fz4Xhnw2+9OSusX92TP15g1jiUcFb/YUa X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This slightly strengthens our write assertion when lockdep is disabled. It also downgrades us from BUG_ON to WARN_ON, but I think that's an improvement. I don't think dumping the mm_struct was all that valuable; the call chain is what's important. Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/mmap_lock.h | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h index 8d38dcb6d044..de9dc20b01ba 100644 --- a/include/linux/mmap_lock.h +++ b/include/linux/mmap_lock.h @@ -60,16 +60,14 @@ static inline void __mmap_lock_trace_released(struct mm_struct *mm, bool write) #endif /* CONFIG_TRACING */ -static inline void mmap_assert_locked(struct mm_struct *mm) +static inline void mmap_assert_locked(const struct mm_struct *mm) { - lockdep_assert_held(&mm->mmap_lock); - VM_BUG_ON_MM(!rwsem_is_locked(&mm->mmap_lock), mm); + rwsem_assert_held(&mm->mmap_lock); } -static inline void mmap_assert_write_locked(struct mm_struct *mm) +static inline void mmap_assert_write_locked(const struct mm_struct *mm) { - lockdep_assert_held_write(&mm->mmap_lock); - VM_BUG_ON_MM(!rwsem_is_locked(&mm->mmap_lock), mm); + rwsem_assert_held_write(&mm->mmap_lock); } #ifdef CONFIG_PER_VMA_LOCK From patchwork Sat Oct 7 20:35:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13412479 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E096BE95A6A for ; Sat, 7 Oct 2023 20:36:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6FB2C8D0015; Sat, 7 Oct 2023 16:36:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6A9B18D0013; Sat, 7 Oct 2023 16:36:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 523B88D0015; Sat, 7 Oct 2023 16:36:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 3E3408D0013 for ; Sat, 7 Oct 2023 16:36:08 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 16CDB120246 for ; Sat, 7 Oct 2023 20:36:08 +0000 (UTC) X-FDA: 81319822416.11.5C475F3 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf11.hostedemail.com (Postfix) with ESMTP id 305A740006 for ; Sat, 7 Oct 2023 20:36:05 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=kCGf5OHK; dmarc=none; spf=none (imf11.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696710966; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tO4I1YN3rjgRtaLOwI5RhJx4suxpIK/NlbOhBn3dZ+E=; b=azh2yK1rjkFn0wLqYe6X7YXFlkaLrRstmm/7wlh5rXpDHEPN1jXoinPGy+fbJUn/wXAevy nmfaJC6g47FjvE/2JDYIwMC3ROJzAQR41ub1oLSS5pe8DMjcwHmumMx8SjQ9Djin5sP2RH unifiJ5B46b8P8BVFIf8W+2lWg7cSWY= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=kCGf5OHK; dmarc=none; spf=none (imf11.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696710966; a=rsa-sha256; cv=none; b=Nvs7uAH+zS1vH0hi2fTPni+IQrxI/dU5PPKgxUNlFar7HcE1lVjJhjKsUOY3C7+See9YDY PdZFVRVjnnhnNF05Ak3eXcBiUvpfY4SO2lji4LDnI6KLyDmE2/pn2YJk7aLVB3exYaBVad xmN4sS/Jcw6lcPPqlHyKPnCNu1XQRVg= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=tO4I1YN3rjgRtaLOwI5RhJx4suxpIK/NlbOhBn3dZ+E=; b=kCGf5OHKv1yLhAGaMJql4oo/aY oDrgdQTXvr2bazyv2FCtxnoyhilrDWQktqCsorTTPfXQ8CANTTikiR/c5LrqHxvnwV0TPPm+cLOcT cbZb3+jKkTqYoGLkPuflfJ4d+vjrlHf6EokCVmDD/NQTyXzDPUdp2ILpK9eKf1n4ZHCRs7vj/K5hW NAtOXWYKChX3rHQnOVxoK1wfIV0Z0IybOnaEAcHXJjtJK863oYBqC8gAnUY1czauBv5ZiNEvm0iGF pXMHml50dSk2YAiDSVT56nFM6/WnZX1uRBvZ32aBt3DRK6SFcpOGPJUuGOixBEQi0FyohEnh7w7pW EU3o0Vmw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qpE1Y-005mLE-OL; Sat, 07 Oct 2023 20:35:44 +0000 From: "Matthew Wilcox (Oracle)" To: Peter Zijlstra , Ingo Molnar , Will Deacon , Waiman Long Cc: "Matthew Wilcox (Oracle)" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Chandan Babu R , "Darrick J . Wong" , linux-xfs@vger.kernel.org, Mateusz Guzik Subject: [PATCH v2 3/5] xfs: Replace xfs_isilocked with xfs_assert_locked Date: Sat, 7 Oct 2023 21:35:41 +0100 Message-Id: <20231007203543.1377452-4-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231007203543.1377452-1-willy@infradead.org> References: <20231007203543.1377452-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 305A740006 X-Stat-Signature: hk6m4gz4cr946ri66oe8fnfq1empfbwc X-Rspam-User: X-HE-Tag: 1696710965-391443 X-HE-Meta: U2FsdGVkX1+1RXhMvItvC1F0DMGtcL78LNpXVzg7gscH/gcaA2nWvMAULa0UPXCNHTg5Jlwu+YyO/Wo4ezVgSriVNgQuLEaCpgKeJn1KZzx6slMRqsDmpaGmK+ooX05Jc1h+HTxCoR15gTyHtQNPRpNdDJebATOQ/VDtS+SAp0+HdwZN73iqWqFo1YXF6KI+ULw++SBeXZrEohMDQ6zqqOgDmD0HT0uHD4d99MYxZs2hHXBJgEcJN8P1jiH7orxCaHIdFVQSl97HwuBnTd1LM7gGHrWtIk21hm7NRmZEESX3lLS7Bhw5/trgWOIiIy8Xxuvd9QrK+HLzY1wnuEgbWPjULE5OtpBl0eqjAApnP8xgqXN+lm+iDR+5eMpJF2CgIdwKui9mrHxF2eEvjYqB96nTXjZWT7kYIIZubmVdJfeyiA2j4UkthtSF/PkxqB4TaYsaWhzjl1hSwWqjlpTnteXy7dzTWuq5VOob5XIZiscEut1P2YiYM5biU6VWbrefraoBMd3jO4oVsZqANq0AViytLBOILb3lSodDQlIepJJ1oUA6SGNVq7QaNT4blxQHx+e1RujXVst90vK2ktzJN8QOcMUAySkp7M4iF++l2uB8H3kyTF1mmj0QogwS9X/28qqhh6AFR11JM5hijt7Tket2qmSJpJui5zTUm9rlB2MlEwhJsJ8QAUwoeWOMsreJOzW1/Yu3NuonPv95jrHyyDCav4Ora3OJyK6hi4n+G9bC5t7sxoLHk+ZGwjCmwP3pC+bYdAruPLIzDyJjr3Ctsb8rnDpihGCzuomH7iFrDXUOHSOa5fpHLuijT041ODY6PVLsOZ4hptkXORBsYsuUgLycrWyX0PT9eUYl2oe2pZ/Hlof9k/7mSe/jCv/xSSi+JXhhcUdmJvtLnOV06XO7LBpUbQV0XAe7ynoi4lxeDvXcOYl5PqLSEEp89ssJ3K26/RpAlk96ot+Cf5nkjOw nycMF+AR XzpzJFXOl7jbNIgU+7EHRtYEhSYZJQhKO0kX1Bd8IvlUbGCzyCVNbf60NscALCBZWjgYBjfJz4Cm2XoEIOinh/TniGTrCOMK05VlT2W9Vj6xn9c3li72JMEH6SGe9ORrdXeD1ZrOgfHWF0SP6sixIFrynnceX/ZbsFOWPv0UtCjUkkGFHBTtQewdtrvGw+UVesOUqjNHjZSrkbyswpp+McBdXaCZUDn6wJEgE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: To use the new rwsem_assert_held()/rwsem_assert_held_write(), we can't use the existing ASSERT macro. xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_ILOCK_EXCL) checks both the IOLOCK and the ILOCK are held for write. xfs_isilocked() only checked that the ILOCK was held for write. xfs_assert_ilocked() is always on, even if DEBUG or XFS_WARN aren't defined. It's a cheap check, so I don't think it's worth defining it away. Signed-off-by: Matthew Wilcox (Oracle) --- fs/xfs/libxfs/xfs_attr.c | 2 +- fs/xfs/libxfs/xfs_attr_remote.c | 2 +- fs/xfs/libxfs/xfs_bmap.c | 19 +++++---- fs/xfs/libxfs/xfs_defer.c | 2 +- fs/xfs/libxfs/xfs_inode_fork.c | 2 +- fs/xfs/libxfs/xfs_rtbitmap.c | 2 +- fs/xfs/libxfs/xfs_trans_inode.c | 6 +-- fs/xfs/scrub/readdir.c | 4 +- fs/xfs/xfs_attr_list.c | 2 +- fs/xfs/xfs_bmap_util.c | 10 ++--- fs/xfs/xfs_dir2_readdir.c | 2 +- fs/xfs/xfs_dquot.c | 4 +- fs/xfs/xfs_file.c | 4 +- fs/xfs/xfs_inode.c | 72 +++++++++++---------------------- fs/xfs/xfs_inode.h | 2 +- fs/xfs/xfs_inode_item.c | 4 +- fs/xfs/xfs_iops.c | 3 +- fs/xfs/xfs_qm.c | 10 ++--- fs/xfs/xfs_reflink.c | 2 +- fs/xfs/xfs_rtalloc.c | 4 +- fs/xfs/xfs_symlink.c | 2 +- fs/xfs/xfs_trans_dquot.c | 2 +- 22 files changed, 66 insertions(+), 96 deletions(-) diff --git a/fs/xfs/libxfs/xfs_attr.c b/fs/xfs/libxfs/xfs_attr.c index e28d93d232de..ebf0722d8963 100644 --- a/fs/xfs/libxfs/xfs_attr.c +++ b/fs/xfs/libxfs/xfs_attr.c @@ -224,7 +224,7 @@ int xfs_attr_get_ilocked( struct xfs_da_args *args) { - ASSERT(xfs_isilocked(args->dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(args->dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL); if (!xfs_inode_hasattr(args->dp)) return -ENOATTR; diff --git a/fs/xfs/libxfs/xfs_attr_remote.c b/fs/xfs/libxfs/xfs_attr_remote.c index d440393b40eb..1c007ebf153a 100644 --- a/fs/xfs/libxfs/xfs_attr_remote.c +++ b/fs/xfs/libxfs/xfs_attr_remote.c @@ -545,7 +545,7 @@ xfs_attr_rmtval_stale( struct xfs_buf *bp; int error; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (XFS_IS_CORRUPT(mp, map->br_startblock == DELAYSTARTBLOCK) || XFS_IS_CORRUPT(mp, map->br_startblock == HOLESTARTBLOCK)) diff --git a/fs/xfs/libxfs/xfs_bmap.c b/fs/xfs/libxfs/xfs_bmap.c index 30c931b38853..d70cf543a52d 100644 --- a/fs/xfs/libxfs/xfs_bmap.c +++ b/fs/xfs/libxfs/xfs_bmap.c @@ -1189,7 +1189,7 @@ xfs_iread_extents( if (!xfs_need_iread_extents(ifp)) return 0; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ir.loaded = 0; xfs_iext_first(ifp, &ir.icur); @@ -3883,7 +3883,7 @@ xfs_bmapi_read( ASSERT(*nmap >= 1); ASSERT(!(flags & ~(XFS_BMAPI_ATTRFORK | XFS_BMAPI_ENTIRE))); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_SHARED|XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_SHARED|XFS_ILOCK_EXCL); if (WARN_ON_ONCE(!ifp)) return -EFSCORRUPTED; @@ -4354,7 +4354,7 @@ xfs_bmapi_write( ASSERT(tp != NULL); ASSERT(len > 0); ASSERT(ifp->if_format != XFS_DINODE_FMT_LOCAL); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(!(flags & XFS_BMAPI_REMAP)); /* zeroing is for currently only for data extents, not metadata */ @@ -4651,7 +4651,7 @@ xfs_bmapi_remap( ifp = xfs_ifork_ptr(ip, whichfork); ASSERT(len > 0); ASSERT(len <= (xfs_filblks_t)XFS_MAX_BMBT_EXTLEN); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(!(flags & ~(XFS_BMAPI_ATTRFORK | XFS_BMAPI_PREALLOC | XFS_BMAPI_NORMAP))); ASSERT((flags & (XFS_BMAPI_ATTRFORK | XFS_BMAPI_PREALLOC)) != @@ -5305,7 +5305,7 @@ __xfs_bunmapi( if (xfs_is_shutdown(mp)) return -EIO; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(len > 0); ASSERT(nexts >= 0); @@ -5648,8 +5648,7 @@ xfs_bmse_merge( blockcount = left->br_blockcount + got->br_blockcount; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL)); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_ILOCK_EXCL); ASSERT(xfs_bmse_can_merge(left, got, shift)); new = *left; @@ -5777,7 +5776,7 @@ xfs_bmap_collapse_extents( if (xfs_is_shutdown(mp)) return -EIO; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_ILOCK_EXCL); error = xfs_iread_extents(tp, ip, whichfork); if (error) @@ -5850,7 +5849,7 @@ xfs_bmap_can_insert_extents( int is_empty; int error = 0; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL); if (xfs_is_shutdown(ip->i_mount)) return -EIO; @@ -5892,7 +5891,7 @@ xfs_bmap_insert_extents( if (xfs_is_shutdown(mp)) return -EIO; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_ILOCK_EXCL); error = xfs_iread_extents(tp, ip, whichfork); if (error) diff --git a/fs/xfs/libxfs/xfs_defer.c b/fs/xfs/libxfs/xfs_defer.c index bcfb6a4203cd..7927a721dc86 100644 --- a/fs/xfs/libxfs/xfs_defer.c +++ b/fs/xfs/libxfs/xfs_defer.c @@ -744,7 +744,7 @@ xfs_defer_ops_capture( * transaction. */ for (i = 0; i < dfc->dfc_held.dr_inos; i++) { - ASSERT(xfs_isilocked(dfc->dfc_held.dr_ip[i], XFS_ILOCK_EXCL)); + xfs_assert_ilocked(dfc->dfc_held.dr_ip[i], XFS_ILOCK_EXCL); ihold(VFS_I(dfc->dfc_held.dr_ip[i])); } diff --git a/fs/xfs/libxfs/xfs_inode_fork.c b/fs/xfs/libxfs/xfs_inode_fork.c index 5a2e7ddfa76d..14193e044f3d 100644 --- a/fs/xfs/libxfs/xfs_inode_fork.c +++ b/fs/xfs/libxfs/xfs_inode_fork.c @@ -563,7 +563,7 @@ xfs_iextents_copy( struct xfs_bmbt_irec rec; int64_t copied = 0; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL | XFS_ILOCK_SHARED)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL | XFS_ILOCK_SHARED); ASSERT(ifp->if_bytes > 0); for_each_xfs_iext(ifp, &icur, &rec) { diff --git a/fs/xfs/libxfs/xfs_rtbitmap.c b/fs/xfs/libxfs/xfs_rtbitmap.c index fa180ab66b73..4146b8697c89 100644 --- a/fs/xfs/libxfs/xfs_rtbitmap.c +++ b/fs/xfs/libxfs/xfs_rtbitmap.c @@ -974,7 +974,7 @@ xfs_rtfree_extent( mp = tp->t_mountp; ASSERT(mp->m_rbmip->i_itemp != NULL); - ASSERT(xfs_isilocked(mp->m_rbmip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(mp->m_rbmip, XFS_ILOCK_EXCL); error = xfs_rtcheck_alloc_range(mp, tp, bno, len); if (error) diff --git a/fs/xfs/libxfs/xfs_trans_inode.c b/fs/xfs/libxfs/xfs_trans_inode.c index 6b2296ff248a..05229fdef36b 100644 --- a/fs/xfs/libxfs/xfs_trans_inode.c +++ b/fs/xfs/libxfs/xfs_trans_inode.c @@ -31,7 +31,7 @@ xfs_trans_ijoin( { struct xfs_inode_log_item *iip; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (ip->i_itemp == NULL) xfs_inode_item_init(ip, ip->i_mount); iip = ip->i_itemp; @@ -60,7 +60,7 @@ xfs_trans_ichgtime( struct timespec64 tv; ASSERT(tp); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); tv = current_time(inode); @@ -90,7 +90,7 @@ xfs_trans_log_inode( struct inode *inode = VFS_I(ip); ASSERT(iip); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(!xfs_iflags_test(ip, XFS_ISTALE)); tp->t_flags |= XFS_TRANS_DIRTY; diff --git a/fs/xfs/scrub/readdir.c b/fs/xfs/scrub/readdir.c index e51c1544be63..0b200bab04c8 100644 --- a/fs/xfs/scrub/readdir.c +++ b/fs/xfs/scrub/readdir.c @@ -283,7 +283,7 @@ xchk_dir_walk( return -EIO; ASSERT(S_ISDIR(VFS_I(dp)->i_mode)); - ASSERT(xfs_isilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL); if (dp->i_df.if_format == XFS_DINODE_FMT_LOCAL) return xchk_dir_walk_sf(sc, dp, dirent_fn, priv); @@ -334,7 +334,7 @@ xchk_dir_lookup( return -EIO; ASSERT(S_ISDIR(VFS_I(dp)->i_mode)); - ASSERT(xfs_isilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL); if (dp->i_df.if_format == XFS_DINODE_FMT_LOCAL) { error = xfs_dir2_sf_lookup(&args); diff --git a/fs/xfs/xfs_attr_list.c b/fs/xfs/xfs_attr_list.c index 99bbbe1a0e44..235dd125c92f 100644 --- a/fs/xfs/xfs_attr_list.c +++ b/fs/xfs/xfs_attr_list.c @@ -505,7 +505,7 @@ xfs_attr_list_ilocked( { struct xfs_inode *dp = context->dp; - ASSERT(xfs_isilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(dp, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL); /* * Decide on what work routines to call based on the inode size. diff --git a/fs/xfs/xfs_bmap_util.c b/fs/xfs/xfs_bmap_util.c index fcefab687285..1bd430f69d44 100644 --- a/fs/xfs/xfs_bmap_util.c +++ b/fs/xfs/xfs_bmap_util.c @@ -655,8 +655,8 @@ xfs_can_free_eofblocks( * Caller must either hold the exclusive io lock; or be inactivating * the inode, which guarantees there are no other users of the inode. */ - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL) || - (VFS_I(ip)->i_state & I_FREEING)); + if (!(VFS_I(ip)->i_state & I_FREEING)) + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL); /* prealloc/delalloc exists only on regular files */ if (!S_ISREG(VFS_I(ip)->i_mode)) @@ -1112,8 +1112,7 @@ xfs_collapse_file_space( xfs_fileoff_t shift_fsb = XFS_B_TO_FSB(mp, len); bool done = false; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL)); - ASSERT(xfs_isilocked(ip, XFS_MMAPLOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_MMAPLOCK_EXCL); trace_xfs_collapse_file_space(ip); @@ -1182,8 +1181,7 @@ xfs_insert_file_space( xfs_fileoff_t shift_fsb = XFS_B_TO_FSB(mp, len); bool done = false; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL)); - ASSERT(xfs_isilocked(ip, XFS_MMAPLOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_MMAPLOCK_EXCL); trace_xfs_insert_file_space(ip); diff --git a/fs/xfs/xfs_dir2_readdir.c b/fs/xfs/xfs_dir2_readdir.c index 9f3ceb461515..95cd8b9cf3dc 100644 --- a/fs/xfs/xfs_dir2_readdir.c +++ b/fs/xfs/xfs_dir2_readdir.c @@ -521,7 +521,7 @@ xfs_readdir( return -EIO; ASSERT(S_ISDIR(VFS_I(dp)->i_mode)); - ASSERT(xfs_isilocked(dp, XFS_IOLOCK_SHARED | XFS_IOLOCK_EXCL)); + xfs_assert_ilocked(dp, XFS_IOLOCK_SHARED | XFS_IOLOCK_EXCL); XFS_STATS_INC(dp->i_mount, xs_dir_getdents); args.dp = dp; diff --git a/fs/xfs/xfs_dquot.c b/fs/xfs/xfs_dquot.c index ac6ba646624d..4b2f1b82badc 100644 --- a/fs/xfs/xfs_dquot.c +++ b/fs/xfs/xfs_dquot.c @@ -949,7 +949,7 @@ xfs_qm_dqget_inode( if (error) return error; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(xfs_inode_dquot(ip, type) == NULL); id = xfs_qm_id_for_quotatype(ip, type); @@ -1006,7 +1006,7 @@ xfs_qm_dqget_inode( } dqret: - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); trace_xfs_dqget_miss(dqp); *O_dqpp = dqp; return 0; diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index 203700278ddb..9ba2b89b3862 100644 --- a/fs/xfs/xfs_file.c +++ b/fs/xfs/xfs_file.c @@ -842,7 +842,7 @@ xfs_break_dax_layouts( { struct page *page; - ASSERT(xfs_isilocked(XFS_I(inode), XFS_MMAPLOCK_EXCL)); + xfs_assert_ilocked(XFS_I(inode), XFS_MMAPLOCK_EXCL); page = dax_layout_busy_page(inode->i_mapping); if (!page) @@ -863,7 +863,7 @@ xfs_break_layouts( bool retry; int error; - ASSERT(xfs_isilocked(XFS_I(inode), XFS_IOLOCK_SHARED|XFS_IOLOCK_EXCL)); + xfs_assert_ilocked(XFS_I(inode), XFS_IOLOCK_SHARED | XFS_IOLOCK_EXCL); do { retry = false; diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index 4d55f58d99b7..812d6f255d84 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -333,52 +333,26 @@ xfs_ilock_demote( trace_xfs_ilock_demote(ip, lock_flags, _RET_IP_); } -#if defined(DEBUG) || defined(XFS_WARN) -static inline bool -__xfs_rwsem_islocked( - struct rw_semaphore *rwsem, - bool shared) -{ - if (!debug_locks) - return rwsem_is_locked(rwsem); - - if (!shared) - return lockdep_is_held_type(rwsem, 0); - - /* - * We are checking that the lock is held at least in shared - * mode but don't care that it might be held exclusively - * (i.e. shared | excl). Hence we check if the lock is held - * in any mode rather than an explicit shared mode. - */ - return lockdep_is_held_type(rwsem, -1); -} - -bool -xfs_isilocked( +void +xfs_assert_ilocked( struct xfs_inode *ip, uint lock_flags) { - if (lock_flags & (XFS_ILOCK_EXCL|XFS_ILOCK_SHARED)) { - if (!(lock_flags & XFS_ILOCK_SHARED)) - return !!ip->i_lock.mr_writer; - return rwsem_is_locked(&ip->i_lock.mr_lock); - } - - if (lock_flags & (XFS_MMAPLOCK_EXCL|XFS_MMAPLOCK_SHARED)) { - return __xfs_rwsem_islocked(&VFS_I(ip)->i_mapping->invalidate_lock, - (lock_flags & XFS_MMAPLOCK_SHARED)); - } - - if (lock_flags & (XFS_IOLOCK_EXCL | XFS_IOLOCK_SHARED)) { - return __xfs_rwsem_islocked(&VFS_I(ip)->i_rwsem, - (lock_flags & XFS_IOLOCK_SHARED)); - } - - ASSERT(0); - return false; + if (lock_flags & XFS_ILOCK_SHARED) + rwsem_assert_held(&ip->i_lock.mr_lock); + else if (lock_flags & XFS_ILOCK_EXCL) + BUG_ON(!ip->i_lock.mr_writer); + + if (lock_flags & XFS_MMAPLOCK_SHARED) + rwsem_assert_held(&VFS_I(ip)->i_mapping->invalidate_lock); + else if (lock_flags & XFS_MMAPLOCK_EXCL) + rwsem_assert_held_write(&VFS_I(ip)->i_mapping->invalidate_lock); + + if (lock_flags & XFS_IOLOCK_SHARED) + rwsem_assert_held(&VFS_I(ip)->i_rwsem); + else if (lock_flags & XFS_IOLOCK_EXCL) + rwsem_assert_held_write(&VFS_I(ip)->i_rwsem); } -#endif /* * xfs_lockdep_subclass_ok() is only used in an ASSERT, so is only called when @@ -1335,9 +1309,9 @@ xfs_itruncate_extents_flags( xfs_filblks_t unmap_len; int error = 0; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); - ASSERT(!atomic_read(&VFS_I(ip)->i_count) || - xfs_isilocked(ip, XFS_IOLOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); + if (atomic_read(&VFS_I(ip)->i_count)) + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL); ASSERT(new_size <= XFS_ISIZE(ip)); ASSERT(tp->t_flags & XFS_TRANS_PERM_LOG_RES); ASSERT(ip->i_itemp != NULL); @@ -1598,7 +1572,7 @@ xfs_inactive_ifree( xfs_trans_ijoin(tp, ip, XFS_ILOCK_EXCL); error = xfs_ifree(tp, ip); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (error) { /* * If we fail to free the inode, shut down. The cancel @@ -2352,7 +2326,7 @@ xfs_ifree( struct xfs_inode_log_item *iip = ip->i_itemp; int error; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(VFS_I(ip)->i_nlink == 0); ASSERT(ip->i_df.if_nextents == 0); ASSERT(ip->i_disk_size == 0 || !S_ISREG(VFS_I(ip)->i_mode)); @@ -2421,7 +2395,7 @@ static void xfs_iunpin( struct xfs_inode *ip) { - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL|XFS_ILOCK_SHARED)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL | XFS_ILOCK_SHARED); trace_xfs_inode_unpin_nowait(ip, _RET_IP_); @@ -3182,7 +3156,7 @@ xfs_iflush( struct xfs_mount *mp = ip->i_mount; int error; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL|XFS_ILOCK_SHARED)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL|XFS_ILOCK_SHARED); ASSERT(xfs_iflags_test(ip, XFS_IFLUSHING)); ASSERT(ip->i_df.if_format != XFS_DINODE_FMT_BTREE || ip->i_df.if_nextents > XFS_IFORK_MAXEXT(ip, XFS_DATA_FORK)); diff --git a/fs/xfs/xfs_inode.h b/fs/xfs/xfs_inode.h index 0c5bdb91152e..b70f3e8e4525 100644 --- a/fs/xfs/xfs_inode.h +++ b/fs/xfs/xfs_inode.h @@ -515,7 +515,7 @@ void xfs_ilock(xfs_inode_t *, uint); int xfs_ilock_nowait(xfs_inode_t *, uint); void xfs_iunlock(xfs_inode_t *, uint); void xfs_ilock_demote(xfs_inode_t *, uint); -bool xfs_isilocked(struct xfs_inode *, uint); +void xfs_assert_ilocked(struct xfs_inode *, uint); uint xfs_ilock_data_map_shared(struct xfs_inode *); uint xfs_ilock_attr_map_shared(struct xfs_inode *); diff --git a/fs/xfs/xfs_inode_item.c b/fs/xfs/xfs_inode_item.c index 127b2410eb20..02427b53328f 100644 --- a/fs/xfs/xfs_inode_item.c +++ b/fs/xfs/xfs_inode_item.c @@ -648,7 +648,7 @@ xfs_inode_item_pin( { struct xfs_inode *ip = INODE_ITEM(lip)->ili_inode; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(lip->li_buf); trace_xfs_inode_pin(ip, _RET_IP_); @@ -754,7 +754,7 @@ xfs_inode_item_release( unsigned short lock_flags; ASSERT(ip->i_itemp != NULL); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); lock_flags = iip->ili_lock_flags; iip->ili_lock_flags = 0; diff --git a/fs/xfs/xfs_iops.c b/fs/xfs/xfs_iops.c index 1c1e6171209d..2492b746912e 100644 --- a/fs/xfs/xfs_iops.c +++ b/fs/xfs/xfs_iops.c @@ -791,8 +791,7 @@ xfs_setattr_size( uint lock_flags = 0; bool did_zeroing = false; - ASSERT(xfs_isilocked(ip, XFS_IOLOCK_EXCL)); - ASSERT(xfs_isilocked(ip, XFS_MMAPLOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_IOLOCK_EXCL | XFS_MMAPLOCK_EXCL); ASSERT(S_ISREG(inode->i_mode)); ASSERT((iattr->ia_valid & (ATTR_UID|ATTR_GID|ATTR_ATIME|ATTR_ATIME_SET| ATTR_MTIME_SET|ATTR_TIMES_SET)) == 0); diff --git a/fs/xfs/xfs_qm.c b/fs/xfs/xfs_qm.c index 086e78a6143a..660e7878db0d 100644 --- a/fs/xfs/xfs_qm.c +++ b/fs/xfs/xfs_qm.c @@ -254,7 +254,7 @@ xfs_qm_dqattach_one( struct xfs_dquot *dqp; int error; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); error = 0; /* @@ -322,7 +322,7 @@ xfs_qm_dqattach_locked( if (!xfs_qm_need_dqattach(ip)) return 0; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (XFS_IS_UQUOTA_ON(mp) && !ip->i_udquot) { error = xfs_qm_dqattach_one(ip, XFS_DQTYPE_USER, @@ -353,7 +353,7 @@ xfs_qm_dqattach_locked( * Don't worry about the dquots that we may have attached before any * error - they'll get detached later if it has not already been done. */ - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); return error; } @@ -1808,7 +1808,7 @@ xfs_qm_vop_chown( XFS_TRANS_DQ_RTBCOUNT : XFS_TRANS_DQ_BCOUNT; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); ASSERT(XFS_IS_QUOTA_ON(ip->i_mount)); /* old dquot */ @@ -1896,7 +1896,7 @@ xfs_qm_vop_create_dqattach( if (!XFS_IS_QUOTA_ON(mp)) return; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (udqp && XFS_IS_UQUOTA_ON(mp)) { ASSERT(ip->i_udquot == NULL); diff --git a/fs/xfs/xfs_reflink.c b/fs/xfs/xfs_reflink.c index eb9102453aff..1267e008014e 100644 --- a/fs/xfs/xfs_reflink.c +++ b/fs/xfs/xfs_reflink.c @@ -527,7 +527,7 @@ xfs_reflink_allocate_cow( int error; bool found; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (!ip->i_cowfp) { ASSERT(!xfs_is_reflink_inode(ip)); xfs_ifork_init_cow(ip); diff --git a/fs/xfs/xfs_rtalloc.c b/fs/xfs/xfs_rtalloc.c index 16534e9873f6..29382bc9d373 100644 --- a/fs/xfs/xfs_rtalloc.c +++ b/fs/xfs/xfs_rtalloc.c @@ -1188,7 +1188,7 @@ xfs_rtallocate_extent( xfs_fsblock_t sb; /* summary file block number */ struct xfs_buf *sumbp; /* summary file block buffer */ - ASSERT(xfs_isilocked(mp->m_rbmip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(mp->m_rbmip, XFS_ILOCK_EXCL); ASSERT(minlen > 0 && minlen <= maxlen); /* @@ -1431,7 +1431,7 @@ xfs_rtpick_extent( uint64_t seq; /* sequence number of file creation */ uint64_t *seqp; /* pointer to seqno in inode */ - ASSERT(xfs_isilocked(mp->m_rbmip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(mp->m_rbmip, XFS_ILOCK_EXCL); seqp = (uint64_t *)&VFS_I(mp->m_rbmip)->i_atime; if (!(mp->m_rbmip->i_diflags & XFS_DIFLAG_NEWRTBM)) { diff --git a/fs/xfs/xfs_symlink.c b/fs/xfs/xfs_symlink.c index 85e433df6a3f..2ca157de4a73 100644 --- a/fs/xfs/xfs_symlink.c +++ b/fs/xfs/xfs_symlink.c @@ -43,7 +43,7 @@ xfs_readlink_bmap_ilocked( int fsblocks = 0; int offset; - ASSERT(xfs_isilocked(ip, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_SHARED | XFS_ILOCK_EXCL); fsblocks = xfs_symlink_blocks(mp, pathlen); error = xfs_bmapi_read(ip, 0, fsblocks, mval, &nmaps, 0); diff --git a/fs/xfs/xfs_trans_dquot.c b/fs/xfs/xfs_trans_dquot.c index aa00cf67ad72..9c159d016ecf 100644 --- a/fs/xfs/xfs_trans_dquot.c +++ b/fs/xfs/xfs_trans_dquot.c @@ -796,7 +796,7 @@ xfs_trans_reserve_quota_nblks( return 0; ASSERT(!xfs_is_quota_inode(&mp->m_sb, ip->i_ino)); - ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL)); + xfs_assert_ilocked(ip, XFS_ILOCK_EXCL); if (force) qflags |= XFS_QMOPT_FORCE_RES; From patchwork Sat Oct 7 20:35:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13412475 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE175E95A62 for ; Sat, 7 Oct 2023 20:36:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 413998D000E; Sat, 7 Oct 2023 16:36:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3C3118D0001; Sat, 7 Oct 2023 16:36:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2B2638D000E; Sat, 7 Oct 2023 16:36:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1BE368D0001 for ; Sat, 7 Oct 2023 16:36:02 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D8AE7C02AF for ; Sat, 7 Oct 2023 20:36:01 +0000 (UTC) X-FDA: 81319822122.24.CC07704 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf27.hostedemail.com (Postfix) with ESMTP id A91C04000E for ; Sat, 7 Oct 2023 20:35:59 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=vvK0TzTj; dmarc=none; spf=none (imf27.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696710960; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XSq7hOCZtUTD3xmIh2+j7L/eFgD05O4FG3L1sm/q0vU=; b=OL1AS5fOtmdaBc1VhqEnJcOFfQIaI2U8LMu4ynb2HIGTwicvFC2ORgdQVP9MsFBJgXPBm/ Gvma8z6a2x8O3FbL9f8tf9jUnMHYnu9xRSAeVqSSvbVD+GFJau9t3C0lVhGn7gXfAmV/V3 /v5tcqlVF4JwFhciqgEVrJH2YsBTqB0= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=vvK0TzTj; dmarc=none; spf=none (imf27.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696710960; a=rsa-sha256; cv=none; b=7GhpeUZZTP9aTLI6H/Bpu7uP41U5qXRi9jiBbxFOY3lXyFylxKvzUN60NCwHq3iiwz45VX 9HPXKLE9tSbyTj9DgnlFQCBNs0OjCR4wCqmBN4/ib+rD9BvTM417QSY/O/P/jyf+rUIeRw nPL0U6H0BXLgiusbbPwUp8OaHCC8zuY= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=XSq7hOCZtUTD3xmIh2+j7L/eFgD05O4FG3L1sm/q0vU=; b=vvK0TzTj5czxdfAs0vX3rONj3R CyOinIlNZZFgSisIAlafVf+YcJotrT54o8RfwWrEfl4Mjhkq/6gRHoXp/GWLfEav5h+9d27Lfgrue kZGkrdGXIjvuB+ies2741IrTpLI7ISbl8uaY0x0VP2r5Pf0ADQGLNcRjFiZe9AfHj8rWjKI1s1zW2 rycySDH7CUC80HfTQA6VuVMJJrM/ktxfimoYiC6Cj75IE8oXjIPrQX1+xTMSdswir1pIJ8ynDLu0R gABbXZ8H2vDfxTFUt/pzfNKXILKBZcVquhCtWVbqfSey1hKSSUCcZNtNXBSnKHmdRldFXYMUtIeW+ 2J0CfQBw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qpE1Y-005mLG-Rn; Sat, 07 Oct 2023 20:35:44 +0000 From: "Matthew Wilcox (Oracle)" To: Peter Zijlstra , Ingo Molnar , Will Deacon , Waiman Long Cc: "Matthew Wilcox (Oracle)" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Chandan Babu R , "Darrick J . Wong" , linux-xfs@vger.kernel.org, Mateusz Guzik Subject: [PATCH v2 4/5] xfs: Remove mrlock wrapper Date: Sat, 7 Oct 2023 21:35:42 +0100 Message-Id: <20231007203543.1377452-5-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231007203543.1377452-1-willy@infradead.org> References: <20231007203543.1377452-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: yuuduozeisrhjz4mmqsn36nk3hp8qr36 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: A91C04000E X-HE-Tag: 1696710959-802853 X-HE-Meta: U2FsdGVkX18YwUGaENPjf+3wH3cZQYBDd+qPqwU3HUEtRc5hHLMrUJhs2kj5a5vsVsKrO28WeJcbOT+QgnHk6/hbjgIaGpMRvC/maI0VNVgtdrfk/C/2Ux884RVNZ7P86vdAT2fMhxQWFziV03YNzOLo00XSv8COBnwmoAcno1PWk1ZWj2s0bS8bwc1bk0GCqI0yrsO3GjTRuQy28drLLqqdR/g2R+kYTmbMyBnQx1VqIYwvvd5kw8KEAGQldNLq86cEM6RRC+xNMZjXlHELhILpzPGSL/4/gKvKompJ86AiN/2mGjzELiUjX9jy3KUAx3k5nLcVyEzpjY25UR7X05qs7/uZs5AteHXQpZ5dHUHhv+IYch9Vre26TIov2KxemFP8iq347D0eg17AwwbjT+jkR0y6SiNK+Eyr2cTDgpCqc+rJl7yPOsNGNtWkSFLAnkZgnYE3cclhTCeNmSnRBUGRpGf+YLH4vLgdnnPTrZagGhygvVx8nLpbpGbZLpaInNd+OTuRL2L60TeULCxLF90QlagoMkw8299wDezsqe0xnYmbzg0Mfg/rjQ4XCDv/MDPG6qNN3YZ/WE+nTqb7U2/6E8HWE1JgdXP8jueGCld+DWX0P5DPCgYx83ECNZdQccSqayJnXmzLaoEf1kvC7tGeSLILR4zom7XOEtDZeLZvSW0dls/z0+k3cWMN3fRpjPhPYoCDx7i+HrV39r9Tv1EcmQ4O/ZiXo0PvsSEet05u9NJcNnbruw8kUG3Zq4mj+W1X9hBUEClJqm6n1jUyb/xpJp9qEUMP6m7B6xHNcKJRjxkMzZKixHKYSQgrWtreLcvJt2V7O9bpQ7Ue/UelJHBdIEwRLRkGWv3fx9/Y1ZbGuxaNBlBjWKHyCrHd9dd4Xzu7za/oDnBI08a2sjd3FHKsbFaa52Zxw0BwqTEt1ZMCq+jeuhluheQMMY+bNf3ksJOQlU0QfwOqv7MduYr 2XbXPZjX bGp8qSJ5IPH3rxGzFPj/hYv8NzWK56/o7+CRlVyBOVb0syszEfIDnAW8L/oOTqDtNwRf1V/n8szB4ZF2zUBOwP6Yar6ir8DJ5W2xlajDSPq7U/Nxls18xTCWOVmCLXE2r+M8qQILtw7JYilvmc82jj/q8GasRk6s9S47/4Hvy0czlp0dEcVv7K38a6nHSOUoiaM6uwEGm88h5Ejc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: mrlock was an rwsem wrapper that also recorded whether the lock was held for read or write. Now that we can ask the generic code whether the lock is held for read or write, we can remove this wrapper and use an rwsem directly. As the comment says, we can't use lockdep to assert that the ILOCK is held for write, because we might be in a workqueue, and we aren't able to tell lockdep that we do in fact own the lock. Signed-off-by: Matthew Wilcox (Oracle) --- fs/xfs/mrlock.h | 78 ---------------------------------------------- fs/xfs/xfs_inode.c | 22 +++++++------ fs/xfs/xfs_inode.h | 2 +- fs/xfs/xfs_iops.c | 4 +-- fs/xfs/xfs_linux.h | 2 +- fs/xfs/xfs_super.c | 4 +-- 6 files changed, 18 insertions(+), 94 deletions(-) delete mode 100644 fs/xfs/mrlock.h diff --git a/fs/xfs/mrlock.h b/fs/xfs/mrlock.h deleted file mode 100644 index 79155eec341b..000000000000 --- a/fs/xfs/mrlock.h +++ /dev/null @@ -1,78 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Copyright (c) 2000-2006 Silicon Graphics, Inc. - * All Rights Reserved. - */ -#ifndef __XFS_SUPPORT_MRLOCK_H__ -#define __XFS_SUPPORT_MRLOCK_H__ - -#include - -typedef struct { - struct rw_semaphore mr_lock; -#if defined(DEBUG) || defined(XFS_WARN) - int mr_writer; -#endif -} mrlock_t; - -#if defined(DEBUG) || defined(XFS_WARN) -#define mrinit(mrp, name) \ - do { (mrp)->mr_writer = 0; init_rwsem(&(mrp)->mr_lock); } while (0) -#else -#define mrinit(mrp, name) \ - do { init_rwsem(&(mrp)->mr_lock); } while (0) -#endif - -#define mrlock_init(mrp, t,n,s) mrinit(mrp, n) -#define mrfree(mrp) do { } while (0) - -static inline void mraccess_nested(mrlock_t *mrp, int subclass) -{ - down_read_nested(&mrp->mr_lock, subclass); -} - -static inline void mrupdate_nested(mrlock_t *mrp, int subclass) -{ - down_write_nested(&mrp->mr_lock, subclass); -#if defined(DEBUG) || defined(XFS_WARN) - mrp->mr_writer = 1; -#endif -} - -static inline int mrtryaccess(mrlock_t *mrp) -{ - return down_read_trylock(&mrp->mr_lock); -} - -static inline int mrtryupdate(mrlock_t *mrp) -{ - if (!down_write_trylock(&mrp->mr_lock)) - return 0; -#if defined(DEBUG) || defined(XFS_WARN) - mrp->mr_writer = 1; -#endif - return 1; -} - -static inline void mrunlock_excl(mrlock_t *mrp) -{ -#if defined(DEBUG) || defined(XFS_WARN) - mrp->mr_writer = 0; -#endif - up_write(&mrp->mr_lock); -} - -static inline void mrunlock_shared(mrlock_t *mrp) -{ - up_read(&mrp->mr_lock); -} - -static inline void mrdemote(mrlock_t *mrp) -{ -#if defined(DEBUG) || defined(XFS_WARN) - mrp->mr_writer = 0; -#endif - downgrade_write(&mrp->mr_lock); -} - -#endif /* __XFS_SUPPORT_MRLOCK_H__ */ diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index 812d6f255d84..a6d9a834d869 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -208,9 +208,9 @@ xfs_ilock( } if (lock_flags & XFS_ILOCK_EXCL) - mrupdate_nested(&ip->i_lock, XFS_ILOCK_DEP(lock_flags)); + down_write_nested(&ip->i_lock, XFS_ILOCK_DEP(lock_flags)); else if (lock_flags & XFS_ILOCK_SHARED) - mraccess_nested(&ip->i_lock, XFS_ILOCK_DEP(lock_flags)); + down_read_nested(&ip->i_lock, XFS_ILOCK_DEP(lock_flags)); } /* @@ -251,10 +251,10 @@ xfs_ilock_nowait( } if (lock_flags & XFS_ILOCK_EXCL) { - if (!mrtryupdate(&ip->i_lock)) + if (!down_write_trylock(&ip->i_lock)) goto out_undo_mmaplock; } else if (lock_flags & XFS_ILOCK_SHARED) { - if (!mrtryaccess(&ip->i_lock)) + if (!down_read_trylock(&ip->i_lock)) goto out_undo_mmaplock; } return 1; @@ -303,9 +303,9 @@ xfs_iunlock( up_read(&VFS_I(ip)->i_mapping->invalidate_lock); if (lock_flags & XFS_ILOCK_EXCL) - mrunlock_excl(&ip->i_lock); + up_write(&ip->i_lock); else if (lock_flags & XFS_ILOCK_SHARED) - mrunlock_shared(&ip->i_lock); + up_read(&ip->i_lock); trace_xfs_iunlock(ip, lock_flags, _RET_IP_); } @@ -324,7 +324,7 @@ xfs_ilock_demote( ~(XFS_IOLOCK_EXCL|XFS_MMAPLOCK_EXCL|XFS_ILOCK_EXCL)) == 0); if (lock_flags & XFS_ILOCK_EXCL) - mrdemote(&ip->i_lock); + downgrade_write(&ip->i_lock); if (lock_flags & XFS_MMAPLOCK_EXCL) downgrade_write(&VFS_I(ip)->i_mapping->invalidate_lock); if (lock_flags & XFS_IOLOCK_EXCL) @@ -338,10 +338,14 @@ xfs_assert_ilocked( struct xfs_inode *ip, uint lock_flags) { + /* + * Sometimes we assert the ILOCK is held exclusively, but we're in + * a workqueue, so lockdep doesn't know we're the owner. + */ if (lock_flags & XFS_ILOCK_SHARED) - rwsem_assert_held(&ip->i_lock.mr_lock); + rwsem_assert_held(&ip->i_lock); else if (lock_flags & XFS_ILOCK_EXCL) - BUG_ON(!ip->i_lock.mr_writer); + __rwsem_assert_held_write(&ip->i_lock); if (lock_flags & XFS_MMAPLOCK_SHARED) rwsem_assert_held(&VFS_I(ip)->i_mapping->invalidate_lock); diff --git a/fs/xfs/xfs_inode.h b/fs/xfs/xfs_inode.h index b70f3e8e4525..1c2484b6afff 100644 --- a/fs/xfs/xfs_inode.h +++ b/fs/xfs/xfs_inode.h @@ -39,7 +39,7 @@ typedef struct xfs_inode { /* Transaction and locking information. */ struct xfs_inode_log_item *i_itemp; /* logging information */ - mrlock_t i_lock; /* inode lock */ + struct rw_semaphore i_lock; /* inode lock */ atomic_t i_pincount; /* inode pin count */ struct llist_node i_gclist; /* deferred inactivation list */ diff --git a/fs/xfs/xfs_iops.c b/fs/xfs/xfs_iops.c index 2492b746912e..fcceb2b4e40d 100644 --- a/fs/xfs/xfs_iops.c +++ b/fs/xfs/xfs_iops.c @@ -1279,9 +1279,9 @@ xfs_setup_inode( */ lockdep_set_class(&inode->i_rwsem, &inode->i_sb->s_type->i_mutex_dir_key); - lockdep_set_class(&ip->i_lock.mr_lock, &xfs_dir_ilock_class); + lockdep_set_class(&ip->i_lock, &xfs_dir_ilock_class); } else { - lockdep_set_class(&ip->i_lock.mr_lock, &xfs_nondir_ilock_class); + lockdep_set_class(&ip->i_lock, &xfs_nondir_ilock_class); } /* diff --git a/fs/xfs/xfs_linux.h b/fs/xfs/xfs_linux.h index e9d317a3dafe..15fdaef578fe 100644 --- a/fs/xfs/xfs_linux.h +++ b/fs/xfs/xfs_linux.h @@ -22,7 +22,6 @@ typedef __u32 xfs_nlink_t; #include "xfs_types.h" #include "kmem.h" -#include "mrlock.h" #include #include @@ -51,6 +50,7 @@ typedef __u32 xfs_nlink_t; #include #include #include +#include #include #include #include diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c index 819a3568b28f..19435fa4c6b3 100644 --- a/fs/xfs/xfs_super.c +++ b/fs/xfs/xfs_super.c @@ -717,9 +717,7 @@ xfs_fs_inode_init_once( /* xfs inode */ atomic_set(&ip->i_pincount, 0); spin_lock_init(&ip->i_flags_lock); - - mrlock_init(&ip->i_lock, MRLOCK_ALLOW_EQUAL_PRI|MRLOCK_BARRIER, - "xfsino", ip->i_ino); + init_rwsem(&ip->i_lock); } /* From patchwork Sat Oct 7 20:35:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13412478 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B10C9E95A71 for ; Sat, 7 Oct 2023 20:36:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 302C18D0014; Sat, 7 Oct 2023 16:36:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2B2638D0013; Sat, 7 Oct 2023 16:36:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 153D78D0014; Sat, 7 Oct 2023 16:36:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id F02C98D0013 for ; Sat, 7 Oct 2023 16:36:06 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id CBCE2140170 for ; Sat, 7 Oct 2023 20:36:06 +0000 (UTC) X-FDA: 81319822332.20.3434F7B Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf08.hostedemail.com (Postfix) with ESMTP id 2DFCA16000A for ; Sat, 7 Oct 2023 20:36:04 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=lQcmH1QC; dmarc=none; spf=none (imf08.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696710965; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=M8hQeLdTKHD1dkcSFiFXLuX/qDihXIc+aJqbr2IAWzY=; b=cuV8MLpT9uFidewr6H4mQOtyOKfrIzAnWCYMVafG7Lw7PATibNy1qmL2DbOktf18xfbH0t gANsjvonxhXGTOr4KXhV3jbo6SIYelzUjQ1GXvHzgsedr6yMZvcmKBM50OOAIsSQSThAIv GzXnmZNqwxnKoNtMJ3RruagPLUSvSO0= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=lQcmH1QC; dmarc=none; spf=none (imf08.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696710965; a=rsa-sha256; cv=none; b=SpNeRxpmJWRPJwCkB6AnLo3wy1lHO9bWC7KizhHY3REiMqqntIQk1rbS966aiVwTrwvBeY Qj3Py90BYwJ4k1q950ujTLZia1/CQiI0BlhwqnZpd0DVWlXZlP17Ly1mKSgc2jvY2l71kj eV3zGSAEvxdZXmYoRYB3gO9kcajO5DU= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=M8hQeLdTKHD1dkcSFiFXLuX/qDihXIc+aJqbr2IAWzY=; b=lQcmH1QCtfWiamaUkdggT/KxOo OK0QlZrv3IuBk3rgCNbSb2qFSC5vOBeAFcN8mrzvdhDtou8RV6BMahNOsV1MSS3hJGeedDAplZ/cV YyZccOspWO/IbpA8IroZfVlWGHQo5O+RCZo0GoNt5Yux+UC2DMSWlPddd+fjtxxMg5V5rPKo917iV WjmPfeKI9oYkPHpC3LSi9i3iUyWzvGuW88sPcLpQLhA1QXCCqoAa2MqrVH0X0URx34oIh7eWvoHi7 JFUsXkX6bwQdA1WfweORl4aGPqfBoGr6P0DIIn4hQWkVuXzWWIWuUVR7MIui0t5jKGZAvoGc/z6VP kst7qR8w==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qpE1Y-005mLI-Ux; Sat, 07 Oct 2023 20:35:44 +0000 From: "Matthew Wilcox (Oracle)" To: Peter Zijlstra , Ingo Molnar , Will Deacon , Waiman Long Cc: "Matthew Wilcox (Oracle)" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Chandan Babu R , "Darrick J . Wong" , linux-xfs@vger.kernel.org, Mateusz Guzik Subject: [PATCH v2 5/5] fs: Add inode_assert_locked() and inode_assert_locked_excl() Date: Sat, 7 Oct 2023 21:35:43 +0100 Message-Id: <20231007203543.1377452-6-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20231007203543.1377452-1-willy@infradead.org> References: <20231007203543.1377452-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: y4nzd755ge77h1q1sig7i783rkm61k4d X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 2DFCA16000A X-HE-Tag: 1696710964-582276 X-HE-Meta: U2FsdGVkX1/RlP4qZx3hRCx+5/3EnpSYxoLolRmKPYh1t1kHiMnBhK6n/1Hl8JhAylFR3Jlh2OmGm6LsX6qnuPQ1ApmtFrNpDPVxJHok7Ih00Ra/3I1u21oXCVN1kpBOfDW8FlA6oHPcxf8fiAx3KE3hFeHDZBqnIUy9+mz9Ce3RdMrokmfoku21avxw9jRPQiNMxuYrvHJWayGd9yl6mR5XibENRFC+C5NS2rOFiUUhrtCSSwddMJwMunIRpUe4TsxdKF3QmkVGR0Rf8WuJr5dxdG72vlErwzYCBtqbsAq6NdkkdlIllRCLV3neg/plnZAZNHMsUDNxVPzbPYu3MthUUkZz6fDoXAlyb85xleSZIIyEUrC4iUHk0xfaNQpEFserd+LFCy4I6E4EXzKmFnyiSbSM0+jLYvdWuyLOBIPlzqhR4zWNlt2g6HXTldzrUb5VAjpXVIOfY+4AbNbez2zcXInNmMUCdmyJ4ysXRyH1ISkR29N4AtZo3QFl2ti8AI4DY/n1cnHgLIgwM0UgZiwLs/LLStLui89XOdS8f4f7mYL38O1LouanLM1BnQKnvt7TGKEdm/kLvZamhvLVLlF2SxZ5TTgL3+xr/RuGWn29ShvklMqSG4xxI884mnJ6zCejdcS+qgrqy89ifPidWKgEPRmMVIzO85btn9cmhfmL+3EWTPulhEIyp2J2sElJ1dKkzdNkbShjkdRNHTOnh3ndY2ivBZdQVycsJQ1lYNXBUhPtIgTXrzmTHD2TB5Oky6A2ls1ymnRKFKxNVePu/xVsY+sHvNk06knmDmJY7SS9typXqJ3ulA5vzvwR2FBDMF2oP8uTes/1l32XP3ovBSDHxvsoCPSN+9nEWI/g3eTPdbYXMTiQkRD+vdqK3+OahuBMjj9N12AcjhEfmc3rF7FXjPOYsWpEFhbcMx0t6itcPyH6Gzds/u3sxKkEgvpcx0si1zw3ocGU8TSKSJO TRrKiP5O M+RWhBLVteF3ldE91BOo89oGzCzUrevDGOQMa7RuGPKLrMvVpp3tvU+xkqurotDMaGsEuaN+DCJe03j4T7AjZZQJUNF/gNbRhHgrodoeJkRASm4w0VjJj4A7TG46z8MVlVRZ4McakrHgiR7Gk7uXb6V2eIglGATZmUHAC72WtDTZmJx2p/Wt7VO2zQqrqriHlqBOSDbVaviZgpZAhE/6eGzvTJZry9S737WIl X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Use the new rwsem_assert_held functions to implement these new assertions. Convert the inode_is_locked() callers in the VFS to use them. Signed-off-by: Matthew Wilcox (Oracle) --- fs/attr.c | 2 +- fs/namei.c | 6 +++--- include/linux/fs.h | 10 ++++++++++ 3 files changed, 14 insertions(+), 4 deletions(-) diff --git a/fs/attr.c b/fs/attr.c index a8ae5f6d9b16..5e32b0a4f8c2 100644 --- a/fs/attr.c +++ b/fs/attr.c @@ -387,7 +387,7 @@ int notify_change(struct mnt_idmap *idmap, struct dentry *dentry, struct timespec64 now; unsigned int ia_valid = attr->ia_valid; - WARN_ON_ONCE(!inode_is_locked(inode)); + inode_assert_locked_excl(inode); error = may_setattr(idmap, inode, ia_valid); if (error) diff --git a/fs/namei.c b/fs/namei.c index 567ee547492b..6b595ad4318d 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -2708,7 +2708,7 @@ struct dentry *try_lookup_one_len(const char *name, struct dentry *base, int len struct qstr this; int err; - WARN_ON_ONCE(!inode_is_locked(base->d_inode)); + inode_assert_locked(base->d_inode); err = lookup_one_common(&nop_mnt_idmap, name, base, len, &this); if (err) @@ -2735,7 +2735,7 @@ struct dentry *lookup_one_len(const char *name, struct dentry *base, int len) struct qstr this; int err; - WARN_ON_ONCE(!inode_is_locked(base->d_inode)); + inode_assert_locked(base->d_inode); err = lookup_one_common(&nop_mnt_idmap, name, base, len, &this); if (err) @@ -2765,7 +2765,7 @@ struct dentry *lookup_one(struct mnt_idmap *idmap, const char *name, struct qstr this; int err; - WARN_ON_ONCE(!inode_is_locked(base->d_inode)); + inode_assert_locked(base->d_inode); err = lookup_one_common(idmap, name, base, len, &this); if (err) diff --git a/include/linux/fs.h b/include/linux/fs.h index b528f063e8ff..e01e041c102b 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -832,6 +832,16 @@ static inline int inode_is_locked(struct inode *inode) return rwsem_is_locked(&inode->i_rwsem); } +static inline void inode_assert_locked(const struct inode *inode) +{ + rwsem_assert_held(&inode->i_rwsem); +} + +static inline void inode_assert_locked_excl(const struct inode *inode) +{ + rwsem_assert_held_write(&inode->i_rwsem); +} + static inline void inode_lock_nested(struct inode *inode, unsigned subclass) { down_write_nested(&inode->i_rwsem, subclass);