From patchwork Fri Apr 9 12:29:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 12194047 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56543C433ED for ; Fri, 9 Apr 2021 12:32:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EED7761165 for ; Fri, 9 Apr 2021 12:32:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EED7761165 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8A7D76B007B; Fri, 9 Apr 2021 08:32:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 87E276B007D; Fri, 9 Apr 2021 08:32:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7465E6B007E; Fri, 9 Apr 2021 08:32:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0007.hostedemail.com [216.40.44.7]) by kanga.kvack.org (Postfix) with ESMTP id 59E676B007B for ; Fri, 9 Apr 2021 08:32:35 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 150E88248047 for ; Fri, 9 Apr 2021 12:32:35 +0000 (UTC) X-FDA: 78012767070.25.3317D1F Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf25.hostedemail.com (Postfix) with ESMTP id EF6C46000108 for ; Fri, 9 Apr 2021 12:32:32 +0000 (UTC) Received: by mail-pj1-f52.google.com with SMTP id ep1-20020a17090ae641b029014d48811e37so3110615pjb.4 for ; Fri, 09 Apr 2021 05:32:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zEHZ5qG2Y9NIbhhZ3B17EZ5gBaH7ne5FjVmUdr7WNhw=; b=HhI9y7ykMy8qmjJd8T6jqQ4HxvvjCcyyeW1xVnQfYtZzlES1KSGQ85Q+DfM9GA7CQi SSIlTfC85Xfx/2gq/5GAnU3p9Foxs6ZApRZK2Ijcw2U8vzpqhy8cjpb9EaZgJEJf2Prs eX/U+LX5IkGaJlbTsgcf5zxP26U7l2ydVfxd865qVJYe7cdjeRVszFRwhdq/gNIwuJgm pCBHrk6ItplIEzUe8TbAm6EW+anIgZWthNuQ6hpKDJaf+0kyBN/LHB268iJFmdRMG1nE Dfi9W6U6lCMtkd5hUkMVeLoQUN1Tnl7ze1uAPrmGFcMe5HzgDSYaSCvoyA91sxWYLpid 3lwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zEHZ5qG2Y9NIbhhZ3B17EZ5gBaH7ne5FjVmUdr7WNhw=; b=EudclIuWuwVGMGWMPPhdhhwhJLhyVbMSWre+G/t1dTfjdE+y7pqGLKqfPXj9dDGnRB Z2Mb5LNYounlf9BQxuIysvWJWUUJcizY9efVnm57Rz7T/ooxlbpxwkXYWfqz23Weg8h7 6DcdtfkJoybPB51JTV+Or0Gd2B8XLvWiIYqtv7XkDZs9Wlmh5I0hHK2UF4pJsK7VhhpJ qeZEPFxwcTmgZlDI18g9zheMsYH3/tL0bBKkGA9HhRrVJWwts1/3TcGwymJBcuIyApni YaKjqQvp/xVnPFIAa+WZHTUrk7NFayXGT/ec5zF10JH3bgvaqHM4wbXjkSEChHl/Dj8W fSmQ== X-Gm-Message-State: AOAM531iUwqjiAZMSm2fPdxQbJgwjB3BbOCT1KtKZSWgAtPArViop8rW fpPydPESH788RIVOXIelC9eFcZIzeuGPGEfk X-Google-Smtp-Source: ABdhPJy7MgN+wnbbgO/53f8oazsZWPqfnfHqoxI3nVximXuOLtS2UPc/V+JY5nOP8tMuCP3A+H3F5g== X-Received: by 2002:a17:90a:156:: with SMTP id z22mr14391859pje.181.1617971553903; Fri, 09 Apr 2021 05:32:33 -0700 (PDT) Received: from localhost.localdomain ([139.177.225.243]) by smtp.gmail.com with ESMTPSA id t3sm2167785pfg.176.2021.04.09.05.32.26 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 09 Apr 2021 05:32:33 -0700 (PDT) From: Muchun Song To: guro@fb.com, hannes@cmpxchg.org, mhocko@kernel.org, akpm@linux-foundation.org, shakeelb@google.com, vdavydov.dev@gmail.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, bsingharora@gmail.com, shy828301@gmail.com, alex.shi@linux.alibaba.com, Muchun Song Subject: [RFC PATCH v2 07/18] mm: memcontrol: introduce compact_lock_page_lruvec_irqsave Date: Fri, 9 Apr 2021 20:29:48 +0800 Message-Id: <20210409122959.82264-8-songmuchun@bytedance.com> X-Mailer: git-send-email 2.21.0 (Apple Git-122) In-Reply-To: <20210409122959.82264-1-songmuchun@bytedance.com> References: <20210409122959.82264-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: EF6C46000108 X-Stat-Signature: d3o7jj4q1i7ah76bj6w93be99b61jbe1 X-Rspamd-Server: rspam02 Received-SPF: none (bytedance.com>: No applicable sender policy available) receiver=imf25; identity=mailfrom; envelope-from=""; helo=mail-pj1-f52.google.com; client-ip=209.85.216.52 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617971552-863397 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If we reuse the objcg APIs to charge LRU pages, the page_memcg() can be changed when the LRU pages reparented. In this case, we need to acquire the new lruvec lock. lruvec = mem_cgroup_page_lruvec(page); // The page is reparented. compact_lock_irqsave(&lruvec->lru_lock, &flags, cc); // Acquired the wrong lruvec lock and need to retry. But compact_lock_irqsave() only take lruvec lock as the parameter, we cannot aware this change. If it can take the page as parameter to acquire the lruvec lock. When the page memcg is changed, we can use the page_memcg() detect whether we need to reacquire the new lruvec lock. So compact_lock_irqsave() is not suitable for us. Similar to lock_page_lruvec_irqsave(), introduce compact_lock_page_lruvec_irqsave() to acquire the lruvec lock in the compaction routine. Signed-off-by: Muchun Song --- mm/compaction.c | 29 +++++++++++++++++++++++++---- 1 file changed, 25 insertions(+), 4 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index e7da342003dd..c9efe3542b0a 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -511,6 +511,29 @@ static bool compact_lock_irqsave(spinlock_t *lock, unsigned long *flags, return true; } +static struct lruvec * +compact_lock_page_lruvec_irqsave(struct page *page, unsigned long *flags, + struct compact_control *cc) +{ + struct lruvec *lruvec; + + lruvec = mem_cgroup_page_lruvec(page); + + /* Track if the lock is contended in async mode */ + if (cc->mode == MIGRATE_ASYNC && !cc->contended) { + if (spin_trylock_irqsave(&lruvec->lru_lock, *flags)) + goto out; + + cc->contended = true; + } + + spin_lock_irqsave(&lruvec->lru_lock, *flags); +out: + lruvec_memcg_debug(lruvec, page); + + return lruvec; +} + /* * Compaction requires the taking of some coarse locks that are potentially * very heavily contended. The lock should be periodically unlocked to avoid @@ -1040,10 +1063,8 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, if (locked) unlock_page_lruvec_irqrestore(locked, flags); - compact_lock_irqsave(&lruvec->lru_lock, &flags, cc); - locked = lruvec; - - lruvec_memcg_debug(lruvec, page); + locked = compact_lock_page_lruvec_irqsave(page, &flags, cc); + lruvec = locked; /* Try get exclusive access under lock */ if (!skip_updated) {