From patchwork Tue Mar 30 10:15:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12172207 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC727C433DB for ; Tue, 30 Mar 2021 10:21:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6DB00619AE for ; Tue, 30 Mar 2021 10:21:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6DB00619AE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 009146B0095; Tue, 30 Mar 2021 06:21:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EFB976B0096; Tue, 30 Mar 2021 06:21:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D748F6B0098; Tue, 30 Mar 2021 06:21:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0073.hostedemail.com [216.40.44.73]) by kanga.kvack.org (Postfix) with ESMTP id B69926B0095 for ; Tue, 30 Mar 2021 06:21:56 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 7B046180AD81A for ; Tue, 30 Mar 2021 10:21:56 +0000 (UTC) X-FDA: 77976149832.27.177153C Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) by imf21.hostedemail.com (Postfix) with ESMTP id 1C576E0011C5 for ; Tue, 30 Mar 2021 10:21:53 +0000 (UTC) Received: by mail-pl1-f182.google.com with SMTP id y2so5913941plg.5 for ; Tue, 30 Mar 2021 03:21:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/bUVH6CYiEDE2OojiUwNW4YxUovgMumcyycakaekQOs=; b=rc7vxMSybQciv9/5dtRhvyJjfFL8C1QprmT4SwgF2AknzGU48EXo4+csaZ88DVCOXN HLlyMSTI83QInTdlLsM8wBPyhykOQhUA/6woR2kxUWN+MmoJVmdL1vS71CF8KVeKTdTJ PKc4xisUKrZmhoRDxyT2rRl3UDmkf6Lh7h8b9EYup+jepXkv/1CLnlPJ9UxibpwX715z Q+gfpkC4dMvYufZcIUNgZ6i/Z7ZQ3FLiey5sq6MfWLruvg6IU3TW2N+UxDLIGk6HM0eF wUMId628x41BQmLxwu9CJoCIODDjm7BlALZL17b24aZbF+qb/aSW2b9vu4AAJRFuAY28 xwQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/bUVH6CYiEDE2OojiUwNW4YxUovgMumcyycakaekQOs=; b=FWAiLA4MvNgw/tMw3kgEj67UlcOKaTrUx0UYIZ3fSZK8YJOamJ6p1npVezkZtzq3+d v/Nf3vDbzWAODfV61pXphrnf7q/AJ01/jpZKJJpFbqTWQo0/qUIj4hN2F/UfDuXQyJqs w6CjtPqNr8mwaxfr31eWrUo93I1szihIVnubtxMtTaEAMcRdYiY4NWTN7TLau/YOZWkh cTFWW/SVCOCauDEBDU/AN+UI2UpQhVVH1aiB/RGRQ9k1JZZmnlpmsdcFiCMQwot1wZie eviRoT3Ugp6Ar1KfLdRYxUL8vs8W+utEETQ+PmUojEcjqnW2YFsgMlMi5gW9Yb1g+L1F v28w== X-Gm-Message-State: AOAM532mlGBsmMBzXBfRxVGJenA1m+1sSjmJrg0BCCFwQN2kt94vcb1k i1aYYFA92gdcUB/8tfo5W8jLOA== X-Google-Smtp-Source: ABdhPJwUaQ5H5WNHLQADgPnCOwmyrS9JVPM2onvAWbxdHSJgQZXin+hFD74iiQOLABz8q4nyQCLqCQ== X-Received: by 2002:a17:902:c101:b029:e7:3268:6fed with SMTP id 1-20020a170902c101b02900e732686fedmr19897636pli.79.1617099715239; Tue, 30 Mar 2021 03:21:55 -0700 (PDT) Received: from localhost.localdomain ([2408:8445:ad30:68d8:c87f:ca1b:dc00:4730]) by smtp.gmail.com with ESMTPSA id k10sm202259pfk.205.2021.03.30.03.21.42 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 30 Mar 2021 03:21:54 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, Muchun Song Subject: [RFC PATCH 07/15] mm: memcontrol: introduce compact_lock_page_lruvec_irqsave Date: Tue, 30 Mar 2021 18:15:23 +0800 Message-Id: <20210330101531.82752-8-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210330101531.82752-1-songmuchun@bytedance.com> References: <20210330101531.82752-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Stat-Signature: qyn3mnootq9zs3xd41u9m9kfcej3nwna X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 1C576E0011C5 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf21; identity=mailfrom; envelope-from=""; helo=mail-pl1-f182.google.com; client-ip=209.85.214.182 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617099713-558630 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If we reuse the objcg APIs to charge LRU pages, the page_memcg() can be changed when the LRU pages reparented. In this case, we need to acquire the new lruvec lock. lruvec = mem_cgroup_page_lruvec(page); // The page is reparented. compact_lock_irqsave(&lruvec->lru_lock, &flags, cc); // Acquired the wrong lruvec lock and need to retry. But compact_lock_irqsave() only take lruvec lock as the parameter, we cannot aware this change. If it can take the page as parameter to acquire the lruvec lock. When the page memcg is changed, we can use the page_memcg() detect whether we need to reacquire the new lruvec lock. So compact_lock_irqsave() is not suitable for us. Similar to lock_page_lruvec_irqsave(), introduce compact_lock_page_lruvec_irqsave() to acquire the lruvec lock in the compaction routine. Signed-off-by: Muchun Song --- mm/compaction.c | 29 +++++++++++++++++++++++++---- 1 file changed, 25 insertions(+), 4 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index 8b8fc279766e..d6b7d5f90fce 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -511,6 +511,29 @@ static bool compact_lock_irqsave(spinlock_t *lock, unsigned long *flags, return true; } +static struct lruvec * +compact_lock_page_lruvec_irqsave(struct page *page, unsigned long *flags, + struct compact_control *cc) +{ + struct lruvec *lruvec; + + lruvec = mem_cgroup_page_lruvec(page); + + /* Track if the lock is contended in async mode */ + if (cc->mode == MIGRATE_ASYNC && !cc->contended) { + if (spin_trylock_irqsave(&lruvec->lru_lock, *flags)) + goto out; + + cc->contended = true; + } + + spin_lock_irqsave(&lruvec->lru_lock, *flags); +out: + lruvec_memcg_debug(lruvec, page); + + return lruvec; +} + /* * Compaction requires the taking of some coarse locks that are potentially * very heavily contended. The lock should be periodically unlocked to avoid @@ -1001,10 +1024,8 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, if (locked) unlock_page_lruvec_irqrestore(locked, flags); - compact_lock_irqsave(&lruvec->lru_lock, &flags, cc); - locked = lruvec; - - lruvec_memcg_debug(lruvec, page); + locked = compact_lock_page_lruvec_irqsave(page, &flags, cc); + lruvec = locked; /* Try get exclusive access under lock */ if (!skip_updated) {