From patchwork Tue Oct 19 12:50:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12569833 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E91DC433F5 for ; Tue, 19 Oct 2021 12:53:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A38646135E for ; Tue, 19 Oct 2021 12:53:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A38646135E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id F1F866B006C; Tue, 19 Oct 2021 08:53:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EA8396B0071; Tue, 19 Oct 2021 08:53:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2B1C6B0072; Tue, 19 Oct 2021 08:53:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id C15F76B006C for ; Tue, 19 Oct 2021 08:53:44 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 84EEE18088931 for ; Tue, 19 Oct 2021 12:53:44 +0000 (UTC) X-FDA: 78713178768.24.C33AA67 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf22.hostedemail.com (Postfix) with ESMTP id B5D2B1905 for ; Tue, 19 Oct 2021 12:53:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:In-Reply-To:References; bh=7aO2sJT2+GeR/XWdjn3j7bZAxj20ZX+OpXfpp2I0p3c=; b=A5Z69eHo60AE/Y+fUvWRs3rX/S KU3N3oKw3HP3UKYKv1IqWM2ZNQV2srcgmKQ2f136Rezf/xDNVXhn55AHyx84o31YVgG9t/umvB85I 5RpOYvtaY/deOSZIkx1GntuDAW8zN1OI/t0TCiG9kt5pdHvcHQrwsD/Ssg2D4pqg8GAr9btSTrkDG p38+6ActQirxy08PKt9fOsg1JJZAvTvbbaiByqhR1jwYrt0uPZPaeZTpWqv7tCGgycuRFVAm9cwAU B4HIVBDihJBXqEKyfDh4Nwe0vVQDds8uITprmSYEovoSJN5tt7EYIdhW+TQVhRPouoBEwjxEBTvOH vpH8DLcw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1mcoZg-00BkH0-G7; Tue, 19 Oct 2021 12:51:22 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , Alexander Duyck , Alex Shi , Hugh Dickins , Johannes Weiner , Vlastimil Babka Subject: [PATCH] memcg: Optimise relock_page_lruvec functions Date: Tue, 19 Oct 2021 13:50:34 +0100 Message-Id: <20211019125034.2799438-1-willy@infradead.org> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=A5Z69eHo; dmarc=none; spf=none (imf22.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: B5D2B1905 X-Stat-Signature: 1ndmycohji1ad5jsm7uq8r7pq9o9u1br X-HE-Tag: 1634648021-334723 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Leave interrupts disabled when we change which lru lock is held. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Vlastimil Babka --- include/linux/memcontrol.h | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 3096c9a0ee01..a6a90b00a22b 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1524,16 +1524,22 @@ static inline bool page_matches_lruvec(struct page *page, struct lruvec *lruvec) } /* Don't lock again iff page's lruvec locked */ -static inline struct lruvec *relock_page_lruvec_irq(struct page *page, +static inline struct lruvec *relock_page_lruvec(struct page *page, struct lruvec *locked_lruvec) { - if (locked_lruvec) { - if (page_matches_lruvec(page, locked_lruvec)) - return locked_lruvec; + if (page_matches_lruvec(page, locked_lruvec)) + return locked_lruvec; - unlock_page_lruvec_irq(locked_lruvec); - } + unlock_page_lruvec(locked_lruvec); + return lock_page_lruvec(page); +} +/* Don't lock again iff page's lruvec locked */ +static inline struct lruvec *relock_page_lruvec_irq(struct page *page, + struct lruvec *locked_lruvec) +{ + if (locked_lruvec) + return relock_page_lruvec(page, locked_lruvec); return lock_page_lruvec_irq(page); } @@ -1541,13 +1547,8 @@ static inline struct lruvec *relock_page_lruvec_irq(struct page *page, static inline struct lruvec *relock_page_lruvec_irqsave(struct page *page, struct lruvec *locked_lruvec, unsigned long *flags) { - if (locked_lruvec) { - if (page_matches_lruvec(page, locked_lruvec)) - return locked_lruvec; - - unlock_page_lruvec_irqrestore(locked_lruvec, *flags); - } - + if (locked_lruvec) + return relock_page_lruvec(page, locked_lruvec); return lock_page_lruvec_irqsave(page, flags); }