From patchwork Tue May 24 10:36:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zhongkun He X-Patchwork-Id: 12859953 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDDB6C433EF for ; Tue, 24 May 2022 10:36:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 178608D0003; Tue, 24 May 2022 06:36:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 128258D0002; Tue, 24 May 2022 06:36:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 014708D0003; Tue, 24 May 2022 06:36:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id E80788D0002 for ; Tue, 24 May 2022 06:36:50 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id BA72134CB4 for ; Tue, 24 May 2022 10:36:50 +0000 (UTC) X-FDA: 79500283380.14.9A50010 Received: from mail-pg1-f170.google.com (mail-pg1-f170.google.com [209.85.215.170]) by imf18.hostedemail.com (Postfix) with ESMTP id BF8E11C002C for ; Tue, 24 May 2022 10:36:32 +0000 (UTC) Received: by mail-pg1-f170.google.com with SMTP id t28so8391944pga.6 for ; Tue, 24 May 2022 03:36:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=Mv0h03Fi/OtxPp/Z+fPoItRmuRgY3cwaQPU9funRX9c=; b=XSkF/bmPvGekW75aBNlxtn4mVjVNuZkS8BypyCMs1G5idGF4Ty3XbFkec8vlHBZMaj zZwD2LVcBdYOOHEDFY9XAoWlv78XpR6SeHp/n21hGalRs7qtlO8pDMhAQzN6/Sm7kUfI aiBO9h/VWkiDCNG8a5iZDWtmuNe/raMvjMRrCW9B4u8ZWES3W7Fdd2QnMvwP02HDxhRl yR+gXercw+VgwdzFpA171XoYDBJ4URP3bB0ZcxwNifVs6zwanO703O+XCetzwpoBPHl5 1sxVk7U579O7sNBv8XzyWH3IHvyjibFJeXc8o4euBduT94QIJ6oZN+m0+ssKCU/X9GmZ j1jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=Mv0h03Fi/OtxPp/Z+fPoItRmuRgY3cwaQPU9funRX9c=; b=PsBRN9Yinhy9QwqC8qqQ6r4bM+o6l0KzP7lb/XwKV2U6rilbRf6DcFKS7p38J3a2QP GVRrxmSKCgThxCtz+Gio0AaIDmR7rsl0mqKTiP4Bhd4tym7WLqiZ1dOH3j9LQPrUbRrR ZDBX17APt54TEp06Fw7WomB9IN3/MFwaw639iGaOJlNidrrYGg/nb3QbJOEdtjPyqd3H 51Y6LAoOVhqTkOMxeaStjrPLuOK3HQcVBVCpxtbu8D0dQsM4+kWYjJiBm8v1Dqs2eZxt 6BkJzsERehlbBScv1ZW3QtII0G0XQ3UG5NGDPW0bJrbTQNWT15BVDLZDd4UU9FMmAV3D dGFw== X-Gm-Message-State: AOAM530MHuf+1jq93KAJLCkBoH0uLjrMRfRkw7bpbbrRJZLpmZR25jNr W4C2PMqgEJGTb5hdTiQRJuUFcQ== X-Google-Smtp-Source: ABdhPJzU0XpiMdHt9AZg3iIPm1ofoXx4H5Mz44HKUELtKJUbPAnWeRAYPyeFGF6qI+8tBZMmdL6dOA== X-Received: by 2002:a63:fc08:0:b0:3f9:e159:b114 with SMTP id j8-20020a63fc08000000b003f9e159b114mr14550018pgi.526.1653388607832; Tue, 24 May 2022 03:36:47 -0700 (PDT) Received: from R911R1VA-1UT.bytedance.net ([139.177.225.245]) by smtp.gmail.com with ESMTPSA id j12-20020a170903024c00b00161823de53csm6913010plh.282.2022.05.24.03.36.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 May 2022 03:36:47 -0700 (PDT) From: hezhongkun To: hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, lizefan.x@bytedance.com, Hezhongkun Subject: [PATCH] mm: memcontrol: add the mempolicy interface for cgroup v2. Date: Tue, 24 May 2022 18:36:38 +0800 Message-Id: <20220524103638.473-1-hezhongkun.hzk@bytedance.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b="XSkF/bmP"; dmarc=pass (policy=none) header.from=bytedance.com; spf=pass (imf18.hostedemail.com: domain of hezhongkun.hzk@bytedance.com designates 209.85.215.170 as permitted sender) smtp.mailfrom=hezhongkun.hzk@bytedance.com X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: BF8E11C002C X-Stat-Signature: puti4j4ki7rpibacyskhgq7k1qys8ep7 X-HE-Tag: 1653388592-82204 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Hezhongkun Mempolicy is difficult to use because it is set in-process via a system call. We want to make it easier to use mempolicy in cgroups, so that we can control low-priority cgroups to allocate memory in specified nodes. So this patch want to adds the mempolicy interface. the mempolicy priority of memcgroup is higher than the priority of task. The order of getting the policy is, memcgroup->policy,task->policy or vma policy, default policy. memcgroup's policy is owned by itself, so descendants will not inherit it. Signed-off-by: Hezhongkun Reported-by: kernel test robot Reported-by: kernel test robot Reported-by: kernel test robot Reported-by: kernel test robot --- include/linux/memcontrol.h | 1 + mm/memcontrol.c | 42 ++++++++++++++++++++++++++++++++++++++ mm/mempolicy.c | 30 ++++++++++++++++++++++----- 3 files changed, 68 insertions(+), 5 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 89b14729d59f..2261eeb6100c 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -343,6 +343,7 @@ struct mem_cgroup { #ifdef CONFIG_TRANSPARENT_HUGEPAGE struct deferred_split deferred_split_queue; #endif + struct mempolicy *mempolicy; struct mem_cgroup_per_node *nodeinfo[]; }; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 598fece89e2b..38108fd4df64 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -6332,6 +6332,42 @@ static int memory_numa_stat_show(struct seq_file *m, void *v) return 0; } + +static int memory_policy_show(struct seq_file *m, void *v) +{ + char buffer[64]; + struct mempolicy *mpol = mem_cgroup_from_seq(m)->mempolicy; + + memset(buffer, 0, sizeof(buffer)); + + if (!mpol || mpol->mode == MPOL_DEFAULT) + return 0; + + mpol_to_str(buffer, sizeof(buffer), mpol); + seq_printf(m, buffer); + seq_putc(m, '\n'); + return 0; +} + +static ssize_t memory_policy_write(struct kernfs_open_file *of, + char *buf, size_t nbytes, loff_t off) +{ + int err = 1; + struct mempolicy *mpol, *old; + struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of)); + + old = memcg->mempolicy; + buf = strstrip(buf); + err = mpol_parse_str(buf, &mpol); + + if (err) + goto out; + mpol_put(old); + memcg->mempolicy = mpol; +out: + return nbytes; +} + #endif static int memory_oom_group_show(struct seq_file *m, void *v) @@ -6416,6 +6452,12 @@ static struct cftype memory_files[] = { .name = "numa_stat", .seq_show = memory_numa_stat_show, }, + { + .name = "policy", + .flags = CFTYPE_NOT_ON_ROOT, + .seq_show = memory_policy_show, + .write = memory_policy_write, + }, #endif { .name = "oom.group", diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 8c74107a2b15..5153b046f8c3 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -176,6 +176,16 @@ struct mempolicy *get_task_policy(struct task_struct *p) return &default_policy; } +struct mempolicy *get_cgrp_or_task_policy(struct task_struct *p) +{ + struct mempolicy *pol; + struct mem_cgroup *memcg = mem_cgroup_from_task(p); + + pol = (memcg && memcg->mempolicy) ? memcg->mempolicy : get_task_policy(p); + return pol; +} + + static const struct mempolicy_operations { int (*create)(struct mempolicy *pol, const nodemask_t *nodes); void (*rebind)(struct mempolicy *pol, const nodemask_t *nodes); @@ -1782,6 +1792,16 @@ static struct mempolicy *get_vma_policy(struct vm_area_struct *vma, return pol; } +static struct mempolicy *get_cgrp_or_vma_policy(struct vm_area_struct *vma, + unsigned long addr) +{ + struct mempolicy *pol; + struct mem_cgroup *memcg = mem_cgroup_from_task(current); + + pol = (memcg && memcg->mempolicy) ? memcg->mempolicy : get_vma_policy(vma, addr); + return pol; +} + bool vma_policy_mof(struct vm_area_struct *vma) { struct mempolicy *pol; @@ -1896,7 +1916,7 @@ unsigned int mempolicy_slab_node(void) if (!in_task()) return node; - policy = current->mempolicy; + policy = get_cgrp_or_task_policy(current); if (!policy) return node; @@ -2005,7 +2025,7 @@ int huge_node(struct vm_area_struct *vma, unsigned long addr, gfp_t gfp_flags, int nid; int mode; - *mpol = get_vma_policy(vma, addr); + *mpol = get_cgrp_or_vma_policy(vma, addr); *nodemask = NULL; mode = (*mpol)->mode; @@ -2158,7 +2178,7 @@ struct page *alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma, int preferred_nid; nodemask_t *nmask; - pol = get_vma_policy(vma, addr); + pol = get_cgrp_or_vma_policy(vma, addr); if (pol->mode == MPOL_INTERLEAVE) { unsigned nid; @@ -2257,7 +2277,7 @@ struct page *alloc_pages(gfp_t gfp, unsigned order) struct page *page; if (!in_interrupt() && !(gfp & __GFP_THISNODE)) - pol = get_task_policy(current); + pol = get_cgrp_or_task_policy(current); /* * No reference counting needed for current->mempolicy @@ -2562,7 +2582,7 @@ int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned long int polnid = NUMA_NO_NODE; int ret = NUMA_NO_NODE; - pol = get_vma_policy(vma, addr); + pol = get_cgrp_or_vma_policy(vma, addr); if (!(pol->flags & MPOL_F_MOF)) goto out;