From patchwork Mon Jan 27 17:34:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roman Gushchin X-Patchwork-Id: 11353005 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CAFE6924 for ; Mon, 27 Jan 2020 17:43:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8A62A24658 for ; Mon, 27 Jan 2020 17:43:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=fb.com header.i=@fb.com header.b="lIgHKuh2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8A62A24658 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=fb.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B20766B0272; Mon, 27 Jan 2020 12:43:04 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AD2086B0003; Mon, 27 Jan 2020 12:43:04 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9E7156B0274; Mon, 27 Jan 2020 12:43:04 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0203.hostedemail.com [216.40.44.203]) by kanga.kvack.org (Postfix) with ESMTP id 8768E6B0003 for ; Mon, 27 Jan 2020 12:43:04 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 45763824805A for ; Mon, 27 Jan 2020 17:43:04 +0000 (UTC) X-FDA: 76424135088.19.week46_8f2179c83845f X-Spam-Summary: 2,0,0,4eed370c58d312d8,d41d8cd98f00b204,prvs=829571e488=guro@fb.com,::akpm@linux-foundation.org:mhocko@kernel.org:hannes@cmpxchg.org:shakeelb@google.com:vdavydov.dev@gmail.com:linux-kernel@vger.kernel.org:kernel-team@fb.com:bharata@linux.ibm.com:laoar.shao@gmail.com:guro@fb.com,RULES_HIT:41:69:355:379:541:800:960:966:973:988:989:1260:1261:1277:1313:1314:1345:1359:1437:1516:1518:1535:1544:1605:1711:1730:1747:1777:1792:2196:2199:2393:2559:2562:2693:2895:3138:3139:3140:3141:3142:3865:3866:3867:3868:3871:3872:3874:4118:4250:4321:4385:4605:5007:6261:6653:7875:7903:9592:10004:11026:11658:11914:12043:12114:12291:12296:12297:12438:12555:12683:12895:12986:13227:13229:14096:14097:14181:14394:14721:21080:21450:21627:21966:21990:30045:30054:30064:30070,0,RBL:67.231.145.42:@fb.com:.lbl8.mailshell.net-62.12.0.100 64.201.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: week46_8f2179c83845f X-Filterd-Recvd-Size: 7697 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf24.hostedemail.com (Postfix) with ESMTP for ; Mon, 27 Jan 2020 17:43:03 +0000 (UTC) Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 00RHes1O030391 for ; Mon, 27 Jan 2020 09:43:02 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=4XpYpv6LK8FngCG+qbORk66V4qNLpc1mD1i+i5N5KPs=; b=lIgHKuh2xN+Bh6cgcWz5xnNrr82Aj4PxRkU74TGGYeT7g5LRA83xzwXQ2HZfQU/zscFa wEh8Eon87RDmMxe7OpQr+MYfGOy8v3AO2paXHA2YFoUFXLSR9QecW56z6e8SNa1UNIgd MVezPCdDit95rHNoM7WXZeue70Ywri5AFUY= Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com with ESMTP id 2xs6qme8gn-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 27 Jan 2020 09:43:02 -0800 Received: from intmgw002.41.prn1.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c085:21d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1779.2; Mon, 27 Jan 2020 09:43:01 -0800 Received: by devvm2643.prn2.facebook.com (Postfix, from userid 111017) id A54F61DFEFC83; Mon, 27 Jan 2020 09:35:07 -0800 (PST) Smtp-Origin-Hostprefix: devvm From: Roman Gushchin Smtp-Origin-Hostname: devvm2643.prn2.facebook.com To: , Andrew Morton CC: Michal Hocko , Johannes Weiner , Shakeel Butt , Vladimir Davydov , , , Bharata B Rao , Yafang Shao , Roman Gushchin Smtp-Origin-Cluster: prn2c23 Subject: [PATCH v2 10/28] mm: memcg: introduce mod_lruvec_memcg_state() Date: Mon, 27 Jan 2020 09:34:35 -0800 Message-ID: <20200127173453.2089565-11-guro@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200127173453.2089565-1-guro@fb.com> References: <20200127173453.2089565-1-guro@fb.com> X-FB-Internal: Safe MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138,18.0.572 definitions=2020-01-27_06:2020-01-24,2020-01-27 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 bulkscore=0 mlxlogscore=823 impostorscore=0 priorityscore=1501 adultscore=0 spamscore=0 phishscore=0 suspectscore=0 mlxscore=0 clxscore=1015 lowpriorityscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1911200001 definitions=main-2001270143 X-FB-Internal: deliver X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: To prepare for per-object accounting of slab objects, let's introduce __mod_lruvec_memcg_state() and mod_lruvec_memcg_state() helpers, which are similar to mod_lruvec_state(), but do not update global node counters, only lruvec and per-cgroup. It's necessary because soon node slab counters will be used for accounting of all memory used by slab pages, however on memcg level only the actually used memory will be counted. The free space will be shared between all cgroups, so it can't be accounted to any specific cgroup. Signed-off-by: Roman Gushchin --- include/linux/memcontrol.h | 22 ++++++++++++++++++++++ mm/memcontrol.c | 37 +++++++++++++++++++++++++++---------- 2 files changed, 49 insertions(+), 10 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 37d4f418e336..73c2a7d32862 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -694,6 +694,8 @@ static inline unsigned long lruvec_page_state_local(struct lruvec *lruvec, void __mod_lruvec_state(struct lruvec *lruvec, enum node_stat_item idx, int val); +void __mod_lruvec_memcg_state(struct lruvec *lruvec, enum node_stat_item idx, + int val); void __mod_lruvec_obj_state(void *p, enum node_stat_item idx, int val); void mod_memcg_obj_state(void *p, int idx, int val); @@ -707,6 +709,16 @@ static inline void mod_lruvec_state(struct lruvec *lruvec, local_irq_restore(flags); } +static inline void mod_lruvec_memcg_state(struct lruvec *lruvec, + enum node_stat_item idx, int val) +{ + unsigned long flags; + + local_irq_save(flags); + __mod_lruvec_memcg_state(lruvec, idx, val); + local_irq_restore(flags); +} + static inline void __mod_lruvec_page_state(struct page *page, enum node_stat_item idx, int val) { @@ -1104,6 +1116,16 @@ static inline void mod_lruvec_state(struct lruvec *lruvec, mod_node_page_state(lruvec_pgdat(lruvec), idx, val); } +static inline void __mod_lruvec_memcg_state(struct lruvec *lruvec, + enum node_stat_item idx, int val) +{ +} + +static inline void mod_lruvec_memcg_state(struct lruvec *lruvec, + enum node_stat_item idx, int val) +{ +} + static inline void __mod_lruvec_page_state(struct page *page, enum node_stat_item idx, int val) { diff --git a/mm/memcontrol.c b/mm/memcontrol.c index cbf01cc0cbac..730f230cee6a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -712,16 +712,16 @@ parent_nodeinfo(struct mem_cgroup_per_node *pn, int nid) } /** - * __mod_lruvec_state - update lruvec memory statistics + * __mod_lruvec_memcg_state - update lruvec memory statistics * @lruvec: the lruvec * @idx: the stat item * @val: delta to add to the counter, can be negative * * The lruvec is the intersection of the NUMA node and a cgroup. This - * function updates the all three counters that are affected by a - * change of state at this level: per-node, per-cgroup, per-lruvec. + * function updates the two of three counters that are affected by a + * change of state at this level: per-cgroup and per-lruvec. */ -void __mod_lruvec_state(struct lruvec *lruvec, enum node_stat_item idx, +void __mod_lruvec_memcg_state(struct lruvec *lruvec, enum node_stat_item idx, int val) { pg_data_t *pgdat = lruvec_pgdat(lruvec); @@ -729,12 +729,6 @@ void __mod_lruvec_state(struct lruvec *lruvec, enum node_stat_item idx, struct mem_cgroup *memcg; long x; - /* Update node */ - __mod_node_page_state(pgdat, idx, val); - - if (mem_cgroup_disabled()) - return; - pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); memcg = pn->memcg; @@ -755,6 +749,29 @@ void __mod_lruvec_state(struct lruvec *lruvec, enum node_stat_item idx, __this_cpu_write(pn->lruvec_stat_cpu->count[idx], x); } +/** + * __mod_lruvec_state - update lruvec memory statistics + * @lruvec: the lruvec + * @idx: the stat item + * @val: delta to add to the counter, can be negative + * + * The lruvec is the intersection of the NUMA node and a cgroup. This + * function updates the all three counters that are affected by a + * change of state at this level: per-node, per-cgroup, per-lruvec. + */ +void __mod_lruvec_state(struct lruvec *lruvec, enum node_stat_item idx, + int val) +{ + pg_data_t *pgdat = lruvec_pgdat(lruvec); + + /* Update node */ + __mod_node_page_state(pgdat, idx, val); + + /* Update per-cgroup and per-lruvec stats */ + if (!mem_cgroup_disabled()) + __mod_lruvec_memcg_state(lruvec, idx, val); +} + void __mod_lruvec_obj_state(void *p, enum node_stat_item idx, int val) { pg_data_t *pgdat = page_pgdat(virt_to_page(p));