From patchwork Thu May 27 09:33:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12283853 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59A0FC4708A for ; Thu, 27 May 2021 09:34:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 09B3F613DE for ; Thu, 27 May 2021 09:34:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 09B3F613DE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1F6436B007D; Thu, 27 May 2021 05:34:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 19FCB6B007E; Thu, 27 May 2021 05:34:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BB1ED6B0080; Thu, 27 May 2021 05:34:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0159.hostedemail.com [216.40.44.159]) by kanga.kvack.org (Postfix) with ESMTP id 76BCE6B007D for ; Thu, 27 May 2021 05:34:03 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 172AB181AF5C7 for ; Thu, 27 May 2021 09:34:03 +0000 (UTC) X-FDA: 78186499566.24.C28C70F Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by imf12.hostedemail.com (Postfix) with ESMTP id 2E857E4 for ; Thu, 27 May 2021 09:33:52 +0000 (UTC) Received: by mail-pf1-f176.google.com with SMTP id 22so118021pfv.11 for ; Thu, 27 May 2021 02:34:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=U0YhrymrmSUcJ+T+JtK0OJA6wg0f0ZwEXSDhmmxyIN4=; b=FyIy/JTLAy/uShk1TncAyQKrw8G/f5Ojqc5RG7p6MZ8FM2+BnLHKRWiHC3Se1mckDY kfP8joLFacRWyM9yy/djdd5EtN2bPH6kFUlNBs7iBN+rdkGxryYDobs0zs82yE1fz5ls s/olT1vG+Y8b8XGFPWejsZIYp92Wfc3XeVVAcxWsSqlfaNoZpv0d46b7YgQMErFYkggF DRuX1YtSotV0yzKhsZr4gStcfYJ0VwnEObHJe4cuRAWadjjWllRc2m11JmLWcq4l/0lQ MP5iQsLy2fClryCvNtJmY2/06BWnSl9+egyt9/I3z6hPYkLuCeVAbo/d0ZqBJ+9S4w5Y t/fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=U0YhrymrmSUcJ+T+JtK0OJA6wg0f0ZwEXSDhmmxyIN4=; b=qY7iKwsFyvf4af6c3Eo6jGeQ5eWA4yMhVSPx5d/zCoVI4DPyO/DdORpnCHqmVwP2D/ hMRD5ILDIWiL742vxuHgyyG8flPf1x+EyBl2JAbhlBNnyY6F6SmXlRzMZTekjuRTRIpP Lp3G0Ovnm5yH1WeJovfCBLDbuRFoPFwK9DnVDJbUmit1lu7z1myQJbp2QtOh5ymJVfvY tdYqTQU1Gpj43uGJ58EdZLpsdm01Zt3qqHTL5mTsCNdK64Ku7gW4TjihZr7JLEnC+89t Eskp9DKAs7l0ma+pMbGuK7TWVSK7jIIR9qhxWEF2CRE8JfHLb3527Ybsqum14XFT5Rf5 e1Qw== X-Gm-Message-State: AOAM532U4ycuun6iPf89+7jNNTLd706yiihslMrYlF3sffttX7KLHxra sQX2fUmv0GdF2oTfq+OVRlxMpw== X-Google-Smtp-Source: ABdhPJxWkKaforjSgY5McTXQGVxZMMc2zHnli+MFqpGqnaszhkiedFoYqoejpPl4F1STf6pgjEREow== X-Received: by 2002:aa7:8159:0:b029:2c5:dfd8:3ac4 with SMTP id d25-20020aa781590000b02902c5dfd83ac4mr2508426pfn.16.1622108041980; Thu, 27 May 2021 02:34:01 -0700 (PDT) Received: from localhost.bytedance.net ([139.177.225.254]) by smtp.gmail.com with ESMTPSA id a9sm1418917pfl.57.2021.05.27.02.33.56 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 27 May 2021 02:34:01 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, bsingharora@gmail.com, shy828301@gmail.com, alexs@kernel.org, smuchun@gmail.com, zhengqi.arch@bytedance.com, Muchun Song Subject: [RFC PATCH v4 02/12] mm: memcontrol: introduce compact_lock_page_irqsave Date: Thu, 27 May 2021 17:33:26 +0800 Message-Id: <20210527093336.14895-3-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210527093336.14895-1-songmuchun@bytedance.com> References: <20210527093336.14895-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 2E857E4 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=bytedance-com.20150623.gappssmtp.com header.s=20150623 header.b="FyIy/JTL"; spf=pass (imf12.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.210.176 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-Rspamd-Server: rspam04 X-Stat-Signature: cjp6z4n7c1s8gr4kmqppbczts1s87mpy X-HE-Tag: 1622108032-362353 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If we reuse the objcg APIs to charge LRU pages, the page_memcg() can be changed when the LRU pages reparented. In this case, we need to acquire the new lruvec lock. lruvec = mem_cgroup_page_lruvec(page); // The page is reparented. compact_lock_irqsave(&lruvec->lru_lock, &flags, cc); // Acquired the wrong lruvec lock and need to retry. But compact_lock_irqsave() only take lruvec lock as the parameter, we cannot aware this change. If it can take the page as parameter to acquire the lruvec lock. When the page memcg is changed, we can use the page_memcg() detect whether we need to reacquire the new lruvec lock. So compact_lock_irqsave() is not suitable for us. Similar to lock_page_lruvec_irqsave(), introduce compact_lock_page_irqsave() to acquire the lruvec lock in the compaction routine. Signed-off-by: Muchun Song Acked-by: Roman Gushchin --- mm/compaction.c | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index 7d41b58fb17c..851fd8f62695 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -511,6 +511,29 @@ static bool compact_lock_irqsave(spinlock_t *lock, unsigned long *flags, return true; } +static struct lruvec *compact_lock_page_irqsave(struct page *page, + unsigned long *flags, + struct compact_control *cc) +{ + struct lruvec *lruvec; + + lruvec = mem_cgroup_page_lruvec(page); + + /* Track if the lock is contended in async mode */ + if (cc->mode == MIGRATE_ASYNC && !cc->contended) { + if (spin_trylock_irqsave(&lruvec->lru_lock, *flags)) + goto out; + + cc->contended = true; + } + + spin_lock_irqsave(&lruvec->lru_lock, *flags); +out: + lruvec_memcg_debug(lruvec, page); + + return lruvec; +} + /* * Compaction requires the taking of some coarse locks that are potentially * very heavily contended. The lock should be periodically unlocked to avoid @@ -1035,11 +1058,9 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, if (locked) unlock_page_lruvec_irqrestore(locked, flags); - compact_lock_irqsave(&lruvec->lru_lock, &flags, cc); + lruvec = compact_lock_page_irqsave(page, &flags, cc); locked = lruvec; - lruvec_memcg_debug(lruvec, page); - /* Try get exclusive access under lock */ if (!skip_updated) { skip_updated = true;