From patchwork Tue Aug 21 21:35:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roman Gushchin X-Patchwork-Id: 10572373 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D91311390 for ; Tue, 21 Aug 2018 21:36:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DA9322AAB0 for ; Tue, 21 Aug 2018 21:36:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CC98F2AE21; Tue, 21 Aug 2018 21:36:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 198502AAB0 for ; Tue, 21 Aug 2018 21:36:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 364606B20AF; Tue, 21 Aug 2018 17:36:37 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2E8426B20B3; Tue, 21 Aug 2018 17:36:37 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 077986B20BB; Tue, 21 Aug 2018 17:36:37 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id 9237A6B20AF for ; Tue, 21 Aug 2018 17:36:36 -0400 (EDT) Received: by mail-ed1-f72.google.com with SMTP id x20-v6so61711eda.22 for ; Tue, 21 Aug 2018 14:36:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:dkim-signature:from:to:cc:subject :date:message-id:mime-version:spamdiagnosticoutput :spamdiagnosticmetadata; bh=sjNTnG9VNjXa3jh+emY2eBDGTtdLI0WfMhnhtZC4Fks=; b=KayPRebI+wvq21DSjJpqj3Ve6ORR5GYihvKKkX7zJenJKAAQ6sAPJapn9Zg19IzTeS QvTyJcaVn2XxV9I7gUVDnC/G5A3COX3LvLk1umINqqetsqE4sFI8LAdN4L9VVYqKxip1 4In1S7VUXy7g2aVv6/T2v3YRT/NDUjhYobBQJmdYIv0VL54fFxGhN0vTQvGkqxfkqCLt fGFYCqHecy4m+oVRBVZXhbaXFGDhguIhk4g/tpBqFutdh2kHhPHVrmLATuFyA9wBJ4AI hWMdi1P53HeWq0S1fjwfGKjh0yvx10sBWDZVDACtMFVvmQXUyry2KhLuMcjKHoKB8hpn 0Wsg== X-Gm-Message-State: AOUpUlG2MS/72LhwR0Frp2Xc4gWKW+4YvaVzHt0VEf582+sV6IcYyXvz cMTqJ0r8hRo+LRXthszFFoDCDzp3R6HHByJAbSJ1dmXd+600AzAuzsEWEN2nJEWmxrbCB8vnv+O 7GNarMOZRQ5Gm6kjYjeT0xNehC7eU0eWqmiwdx4ZEpnh1pCfQVWc3VvITUc4F4hIWqQ== X-Received: by 2002:a50:fd0f:: with SMTP id i15-v6mr65108869eds.83.1534887396124; Tue, 21 Aug 2018 14:36:36 -0700 (PDT) X-Google-Smtp-Source: AA+uWPyeR1Jq6R/jEpOYajvIyqnFZBus1HBxvUbFuefzZCwNVbsrFVe6DR8J3cWRJ6bXnIFhAL3v X-Received: by 2002:a50:fd0f:: with SMTP id i15-v6mr65108840eds.83.1534887395368; Tue, 21 Aug 2018 14:36:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534887395; cv=none; d=google.com; s=arc-20160816; b=xLYJzvrgxQ/MSu65X4YwX5Rz3lWMeVLcjKFTvz3Io7tfOrzDcCvuwwMmwoh5l0Q3AF k0L7ttiTvlSC0h4utE1ZfzjNNFcBGpylBYn2IMxPM69wRWuJDIbX2owWEUXnXlFhfySk SM7PuUrMNsrUXEiISQbXJqy3IiOTmeTMKdb+dLL+suTAfyekGI1h48FUSmKhlRjbJBO6 9A+TXXCF5e3JI6IyJ46TVv0bbel1wEHIPIeBVi7QwTzu7pvr4qe1XZiHJwso/+AYAK6+ zmUCl7a96GrZKIhHBxtZp08M1U1S+cDmrGC+ijhhXIDf51nAQOW951m7hK4dzDSYnR2A L+5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=spamdiagnosticmetadata:spamdiagnosticoutput:mime-version:message-id :date:subject:cc:to:from:dkim-signature:dkim-signature :arc-authentication-results; bh=sjNTnG9VNjXa3jh+emY2eBDGTtdLI0WfMhnhtZC4Fks=; b=kUiL5LontNUekxgReGsicMsi29mYPsXKc9uHr+3ZN28nGSDTWaFG2w+zDg+9Vh4JL1 W+K1/Eb8cGowafeQwkVnLvLgzcDG7RS8rbRQOne+cOaPI4tntZclx6Ornzl9Okgex1M0 rc1U4ig8xd9I3fe5CXTgjNyVwbf3BJVSaLBb0A4M5MHQXBjuv7doMRNa8oQ1xVKvIG31 8SeIL/Q7jiXq2ECgcn9lSBEf9rH2Uqo6GSKMNvSiG84Rsn6I1OifLAFJ1S9FBYNxHb77 uAsbrVA48ADaswv7Tkp+nRyCrojSt/CYIJKvcBIfgow7qvxV7jpLZlnSJzwvuYsdsKf3 lCBA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=Z1Wf7pXM; dkim=pass header.i=@fb.onmicrosoft.com header.s=selector1-fb-com header.b=CVTtJZ5y; spf=pass (google.com: domain of prvs=177171ea8f=guro@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=177171ea8f=guro@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id q3-v6si182833edl.373.2018.08.21.14.36.34 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 21 Aug 2018 14:36:35 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=177171ea8f=guro@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=Z1Wf7pXM; dkim=pass header.i=@fb.onmicrosoft.com header.s=selector1-fb-com header.b=CVTtJZ5y; spf=pass (google.com: domain of prvs=177171ea8f=guro@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=177171ea8f=guro@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0148460.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w7LLYGVk010571; Tue, 21 Aug 2018 14:36:27 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : mime-version : content-type; s=facebook; bh=sjNTnG9VNjXa3jh+emY2eBDGTtdLI0WfMhnhtZC4Fks=; b=Z1Wf7pXMvzSjdOdEMiT/E/d+VCZfQJqx+y4yvt/YLA7EIbiHxLy9A98GKQyZuLXsxcEZ Up7397CAwRgr1OtmNQexdC4KT9wZq+qPUPrTerGWchtXE/+JZFgZKapq8VQ/1rWI709h V/g57SHJIP/9qLKTpkQUQYhFrvWLvJuKZ3Q= Received: from mail.thefacebook.com ([199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 2m0ssj05p6-12 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 21 Aug 2018 14:36:27 -0700 Received: from NAM03-CO1-obe.outbound.protection.outlook.com (192.168.54.28) by o365-in.thefacebook.com (192.168.16.14) with Microsoft SMTP Server (TLS) id 14.3.361.1; Tue, 21 Aug 2018 14:36:20 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sjNTnG9VNjXa3jh+emY2eBDGTtdLI0WfMhnhtZC4Fks=; b=CVTtJZ5yVnNinEavJgLGDmsBy6S7D3WqNVe6n3w2ClRL2BWAMvwBp4y+iMx63Szc92Np8EgbduooNtwP6TvKjur5e9HuOEaHeBOMRDIDBIYlGXR9J+F1Oz2WUZx9ujBls40J4U6lLFLY6y/GA8VDADydIpW5ZY9jVsnL4eXQJnE= Received: from tower.thefacebook.com (2620:10d:c090:200::7:b0bf) by BY2PR15MB0166.namprd15.prod.outlook.com (2a01:111:e400:58e0::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1038.23; Tue, 21 Aug 2018 21:36:15 +0000 From: Roman Gushchin To: CC: , , Roman Gushchin , Johannes Weiner , Michal Hocko , Andy Lutomirski , Konstantin Khlebnikov , Tejun Heo , Shakeel Butt Subject: [PATCH v2 1/3] mm: rework memcg kernel stack accounting Date: Tue, 21 Aug 2018 14:35:57 -0700 Message-ID: <20180821213559.14694-1-guro@fb.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-Originating-IP: [2620:10d:c090:200::7:b0bf] X-ClientProxiedBy: MWHPR1301CA0004.namprd13.prod.outlook.com (2603:10b6:301:29::17) To BY2PR15MB0166.namprd15.prod.outlook.com (2a01:111:e400:58e0::12) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 32584ca1-365a-477b-5fc0-08d607ae2003 X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(2017052603328)(7153060)(7193020);SRVR:BY2PR15MB0166; X-Microsoft-Exchange-Diagnostics: 1;BY2PR15MB0166;3:tgCPoof/SYN9Sk12hWJ/AB4u2UEj+8JsA0iXHsxOWQS7fI6jM5H+osnpNr6BujSUd9nk/JNYK1Uxj9k19DW1FljmLjU8R8inRnKdOhHI9gFJEbr7UDwWneUNmzPRNN/SDNFraTeNqHHwRtzudnZPnRftQp8nCQ6fbPPbKYMk2wKbMbWQDDtVM7fnKtq8xs0JiIN78+HAlR1xK3IsorTHCgD/0+NtCpJY+vKuYLHNIzlwqx/7t3DsZYbkrCCF0lmP;25:Gra0fB8batYaxEjUY9/vMlu//vz7FNn/8Fh7IMJHRBgtdxUrrV1w8MhdMeMurUi3gm8qlWAE3WEJStgXTyb/csWrDq1draEFQeTGk9Ai1GaWIvTZuC1fKTlW17XUE+6PJjvpBSxcnDfjHiKEYkormV6J1S/TwF/CtsJ1IS0v8Gl+dxrWVM53oNATOxuRn66E27ipjev8urqfYsWKTF87Kdf+4x+J1wpv7PFjL+pUzLZ2PUo+u4J3T/4ng7OecrUOlNoh+6LbczsxNqiGU1Sbth491Aynkt3p5hrWZIhu0VuDnTNm4DgyBsnvQ0SM14lVVQBw/nPkVabLEdIYD4KfCA==;31:IEtIgeOmtuqNdLvQtaxwl1K0mbDK6TI11RGxq3nZV1pwG3vVwG3XDWFZ78gLw87Of4G5opXblF6sefDNey8RAumWByHSmqdAiagywyi5ObyDgGVMApi+IYk6fCp0fSTmuV2fVF53EWIXKp7ckhe4AndKSI4SBYMI43u9LXpdLaAQ/9fEXh1albNYKtemjOCJhbcAbupK6Vtgm7dS1qK8nF+Euz0BtmbCipwLurdM3DQ= X-MS-TrafficTypeDiagnostic: BY2PR15MB0166: X-Microsoft-Exchange-Diagnostics: 1;BY2PR15MB0166;20:m8uRvxeukJFtxyc96wzVrIStgz4dCprSBl1pfs5/x56lbP+/FQI7euC4j8HIbTFlxRQ6rnFAWqEocIbzGFQDp3dRazpXEr8I6H7JsN++q8xDxmIxgZjMOWQUKRNQrIiujt+ohVDnGwy41G43lkoGWwvggo5wvdKnxp3nSexPqk4mLSrhL30jwXb/ZHS+/43K1EgBOVqOkhF/tuZajLD7U6fpUi1G95i6pmbAfUWJN6VxvkO8AJuVI0hOBonD9Sxyd1KjO9t0UV0jWqbleZdxTNHd9DZSgUy38oni61d0X6KLjrX6UlAQdppWV3ZHXcCg/OOqWGcUYONb0p9wCJGwQiAqtgZt43nHV1S1QyGBYu9izN2LdZ+vWBmw0/ScGDxEk82YOppHFnX6SYjRlsqMMgFLou3IbXzyRUqiLUBVwdgwyrIBsQSvw+4AkMVDbkAZlEiinMLBTg4hnY0shVmAbtsg+VlLTAFwexBbMlZTZyMNru4nr+4U4txneVHTdn0r;4:55stgnN2YYskx0nvMTwXkBmuTEd56MPytlaRhuL0gEval+frIrrc7GVKsZCU52o9OjMg7KZ55oZYtf+EFbSpwtP29QwIiGknaIKMyicaR1+Gb3M8myVZMxGwgK/0r2HJf1iwFz71aCvT7270L5dZ6p3eKzsSfluEQIR8W/UxUEYJZWhNj7By/EnLL4ZKZMGpu0Vbh8wJEIsTmiN4suXg171QXdiDfSkrQPVxyVX83ZLrOYsa18fBMv6W79hAH4XfjDwZqlJPv6Ja9+kX9eXBCHXBpz1tY5EyqE9+QPcsriImaDC6VzpTEYYXULuKs4P6g9KNIho/0f0SbXNlXCy/yP+4SFvLEqpEaeAnaQZEeWMZGYwVA1MUzBVmxTSa4CE+EiFWYjgc0T8iVnY1ENr2BY/KyDPH8gD108z6qt lJxts= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(85827821059158)(67672495146484)(211936372134217)(153496737603132); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(823301075)(93006095)(93001095)(3002001)(10201501046)(3231311)(11241501184)(944501410)(52105095)(149027)(150027)(6041310)(20161123562045)(20161123564045)(20161123560045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699016);SRVR:BY2PR15MB0166;BCL:0;PCL:0;RULEID:;SRVR:BY2PR15MB0166; X-Forefront-PRVS: 0771670921 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(346002)(136003)(376002)(39860400002)(396003)(366004)(199004)(189003)(53936002)(68736007)(39060400002)(50226002)(8936002)(15650500001)(25786009)(1076002)(2906002)(6116002)(4326008)(476003)(486006)(86362001)(16526019)(14444005)(97736004)(6506007)(186003)(386003)(46003)(2361001)(478600001)(105586002)(48376002)(50466002)(54906003)(106356001)(6486002)(2351001)(2616005)(6916009)(52396003)(305945005)(6512007)(8676002)(53416004)(69596002)(36756003)(16586007)(81156014)(47776003)(81166006)(316002)(52116002)(5660300001)(6666003)(7736002)(51416003)(42262002);DIR:OUT;SFP:1102;SCL:1;SRVR:BY2PR15MB0166;H:tower.thefacebook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; Received-SPF: None (protection.outlook.com: fb.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: 1;BY2PR15MB0166;23:xDn2hDUrIy8rIoN/UWzwm8K419ygmjqNCFVbgjoHm5HBDwfasRgeyTqZGZ+ForvgS+RUHUhrslyeG03Hhs64G7Y98EY9xdHVMLOqDsS71J79FzHdpcX+tHMqre8cvAqsZcNRzU1W1DRRvKJmDNxE3XJs6AF7pX9bIIUitySJAbPE7fSETBIU8NIPgKNm/DUOG4kyqN+ZvTRVkQ2//UIp95jLHlYnkCfS7AZFBMv1dMyFoxBH21shM1kANSNIFJRZJiEK1XVvWfCclKOl54NSe7B9ZM4urWf8NM/51P7uxDoiYxLhtp1z1B5RRwyLRcvEAcOEpU0yhtkP6KlzZyPhFk4NIdJw9drZCSvQDyNr1+nrjXy0lBm4QM6XKcNf3V1hkwq1ewr9epnDl01lgsW2I+EltkZYK3AjtREEZGnEDg/Zz7HEYXOYvxlQZgAsHMsfHIBOU11QH6w6pchk7fM/fG0NePrttZmb4HS5QDX7BGsF+EMn7+uuS/TEJMC3P1CMNfAVPULNUvW/crT3ezFCyf5kZgw+oHLyW5eZaJ7F3uROpqLlIT3fgCPo7KZBvfXmk52BY1ZBh1jDHfXXCqELRnPHTyappKjJ6MdHKSpmMt0xxgZg73/UPggB3XQUCfQLrLGb/M96bvGFtGxB31eicvzoTSWvKNIhaCqk3EMEEjgKZTvSoW7aACLumTIgHGUX7DRiGbaMbErS2V5MEc0eEerBrZ+pEpXR0VGI/0ghJKzx9pFkoDrZsvK15KmZaMzA4m/FhwMzNlO8rRpg2jQG+xcK8fJSM3ioJi3FLuOOKhHu1ZCnkDTrMzTwoagTvdmFgJFVvcFNdhIa2yeryOY7T8PTPEQd2fFRFQdghMi7m8/ZmB71/QcUT8+uaPRlxxZB4Yx8XjnjS+91lqZHcXrXhP25tPNLFRotfX8Swaq6X5PDk5Jz8YHJ8zV7v5+0mCsiJLCrbgNOyAyhN/p3HLTYLx8DBSLOKqbXoN/7Qn1YlZaqFizR0yoNqvPyTfc5ht8Jroqwskuth7xuu03B0Nv+j52gTzOrB/OL8IVLn2mhbT1JyAk+RZ5R1b4EFKBzY9jVlvVroSByBVTbGTFVecdyWsB1iF25rLY8IzXHDV4Qj2NNIBnWDECtj+9XSpf2VvXLyqTfHzw4HanENl+XTns4M8GX8j7HDwgF0ERd4lpaiDcwGTWfbDyph1hjiBVceqxjEomSJxgXE37xCU4TXdRV8+/Hrvgsg4d+WUQlsTCcLbfqLUHS13ukYQcTsdcUYSutiRvDKIGJ8aftHlYECjZH+g== X-Microsoft-Antispam-Message-Info: VofNFfZc6Jw7ubo7iHtrzdrYe8mHfx6XsmwlrjvXh0DW1jIN4iRbXJtNQXPogvRW0+QlKMl96jJk2LpfOgCTR7wRbhWjEadUGifLdhJ/dLbct/Q3y6vgqIXHvnreD/W4BmBzDQBDM5Te/NilpbNvxS6/slpX9caLWNYIFhbFV0dnNA00zqzjdeEqtAPmPIEZhKA4JdJ2jzR7Yl/btj/WG/5/oSRT414cGhh6KGV0RfzYS3dDCkTWO4PY0B/6D3GfQ5YIL9W7KfVzlOpdeLriBfKmJwMpwu0/mW5J/cjHa1nFJ6CijJKqk7I6AZ0cpbpk2fauoV1ZIsuG74i6wobUdVzv07aXfz6mJ/IB/fbJKqg= X-Microsoft-Exchange-Diagnostics: 1;BY2PR15MB0166;6:Gy0Gf653+FwrZlXeljEJFfWrSfimS9bckK1nCoGLW6fi03OhHqHf+GmKFa+242Cx6pKKvmToUldC8VLczApDCShfXj5i7c4RCLY2xC2cjkpzYtsh7+z3U5+XbF3wq7T3nEoRhTrTzpcwnUmEkDtbuAkELjniyaVJWYRf+KVwk+OCHFlbMO4CTrtp6cSvR5I7jiswFMEnAcNJxx4t3DuGcK3RmmepzQqOCJ3bRDp7flKd6EJefmGBqj9jNSG9LwxiOza10Q5xVuxzWXNKCMrP9mBvusBkLy14+tIbl/l1Cw1iEIfgU9MiCoeAICxQxmT/dzorcr4nL1QG0T1Ntxa/z1wreGVGep2Ux2C8aRFb2V1K2VFv1XVTXxkl+xYaS62LrDMCt3NrzyL7gsR8IPwfuG+J87+vxdM4feK2UB4Jjcf25Z/tyX9kcDqOCdbiS1mE1rxKONeEakRLSYJaeeASFw==;5:mpLMpwPCXauoiV4/erNnVBHXurphdUdL9knHpJNBvXFI2jT+JnXMH4V7Iu6/unTyOFGFqcp6CkHlhb8wvGMNe0ktPV9prEbWYorjCaGKQmh2G720AedmgjhDPeqh4QfmUpVl+q2CuNQBLzCn+C2ai/2F1bEqCsW2dTlccFIiqRQ=;7:uqeX9r5JeCdC2+/eXel3slh4FaACnvnl4fWdc2xg1AgCzbxZvYsuRXiohR/qAH1AWrGlfXffu4l3BFsHnd6YyInlecm2ciicx3eaCCG3ZwjirckR+kXLHrFuCoAuIyPBVjLi82o8RcRhvVjzxmKe3dIy2sQPnJlaFLjg8wIEwQOgtIl2M8mhUlmT1V00xDGaxxoOlbATJ9w+IdW+BNbGOEuSabx5lAQ3CRC3eh6Vbt8bn6w6DZNUfYgZrjhZ+48k SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;BY2PR15MB0166;20:58lLBuf8Ra/4oL0auhGtXhyjuvdYWX4SHXZ8o6ZcvIWIplQikhX2bCA7miFKo6elnSBQp2AyqHezxsI63btJSesds9eF5OPDDyZrdzfuKkrsiDWsT2m3FwReasUlp2GHwTS61pyon/jn+y0UhOV6L5PcP0uK0shvDZGSf+PJDu4= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2018 21:36:15.5424 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 32584ca1-365a-477b-5fc0-08d607ae2003 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR15MB0166 X-OriginatorOrg: fb.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-08-21_09:,, signatures=0 X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP If CONFIG_VMAP_STACK is set, kernel stacks are allocated using __vmalloc_node_range() with __GFP_ACCOUNT. So kernel stack pages are charged against corresponding memory cgroups on allocation and uncharged on releasing them. The problem is that we do cache kernel stacks in small per-cpu caches and do reuse them for new tasks, which can belong to different memory cgroups. Each stack page still holds a reference to the original cgroup, so the cgroup can't be released until the vmap area is released. To make this happen we need more than two subsequent exits without forks in between on the current cpu, which makes it very unlikely to happen. As a result, I saw a significant number of dying cgroups (in theory, up to 2 * number_of_cpu + number_of_tasks), which can't be released even by significant memory pressure. As a cgroup structure can take a significant amount of memory (first of all, per-cpu data like memcg statistics), it leads to a noticeable waste of memory. Signed-off-by: Roman Gushchin Cc: Johannes Weiner Cc: Michal Hocko Cc: Andy Lutomirski Cc: Konstantin Khlebnikov Cc: Tejun Heo Cc: Shakeel Butt Reviewed-by: Shakeel Butt Acked-by: Michal Hocko --- include/linux/memcontrol.h | 13 +++++++++- kernel/fork.c | 51 +++++++++++++++++++++++++++++++++----- 2 files changed, 57 insertions(+), 7 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 0e6c515fb698..b12a553048e2 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1250,10 +1250,11 @@ struct kmem_cache *memcg_kmem_get_cache(struct kmem_cache *cachep); void memcg_kmem_put_cache(struct kmem_cache *cachep); int memcg_kmem_charge_memcg(struct page *page, gfp_t gfp, int order, struct mem_cgroup *memcg); + +#ifdef CONFIG_MEMCG_KMEM int memcg_kmem_charge(struct page *page, gfp_t gfp, int order); void memcg_kmem_uncharge(struct page *page, int order); -#ifdef CONFIG_MEMCG_KMEM extern struct static_key_false memcg_kmem_enabled_key; extern struct workqueue_struct *memcg_kmem_cache_wq; @@ -1289,6 +1290,16 @@ extern int memcg_expand_shrinker_maps(int new_id); extern void memcg_set_shrinker_bit(struct mem_cgroup *memcg, int nid, int shrinker_id); #else + +static inline int memcg_kmem_charge(struct page *page, gfp_t gfp, int order) +{ + return 0; +} + +static inline void memcg_kmem_uncharge(struct page *page, int order) +{ +} + #define for_each_memcg_cache_index(_idx) \ for (; NULL; ) diff --git a/kernel/fork.c b/kernel/fork.c index 5ee74c113381..09b5b9a40166 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -223,9 +223,14 @@ static unsigned long *alloc_thread_stack_node(struct task_struct *tsk, int node) return s->addr; } + /* + * Allocated stacks are cached and later reused by new threads, + * so memcg accounting is performed manually on assigning/releasing + * stacks to tasks. Drop __GFP_ACCOUNT. + */ stack = __vmalloc_node_range(THREAD_SIZE, THREAD_ALIGN, VMALLOC_START, VMALLOC_END, - THREADINFO_GFP, + THREADINFO_GFP & ~__GFP_ACCOUNT, PAGE_KERNEL, 0, node, __builtin_return_address(0)); @@ -248,9 +253,20 @@ static unsigned long *alloc_thread_stack_node(struct task_struct *tsk, int node) static inline void free_thread_stack(struct task_struct *tsk) { #ifdef CONFIG_VMAP_STACK - if (task_stack_vm_area(tsk)) { + struct vm_struct *vm = task_stack_vm_area(tsk); + + if (vm) { int i; + for (i = 0; i < THREAD_SIZE / PAGE_SIZE; i++) { + mod_memcg_page_state(vm->pages[i], + MEMCG_KERNEL_STACK_KB, + -(int)(PAGE_SIZE / 1024)); + + memcg_kmem_uncharge(vm->pages[i], + compound_order(vm->pages[i])); + } + for (i = 0; i < NR_CACHED_STACKS; i++) { if (this_cpu_cmpxchg(cached_stacks[i], NULL, tsk->stack_vm_area) != NULL) @@ -350,10 +366,6 @@ static void account_kernel_stack(struct task_struct *tsk, int account) NR_KERNEL_STACK_KB, PAGE_SIZE / 1024 * account); } - - /* All stack pages belong to the same memcg. */ - mod_memcg_page_state(vm->pages[0], MEMCG_KERNEL_STACK_KB, - account * (THREAD_SIZE / 1024)); } else { /* * All stack pages are in the same zone and belong to the @@ -369,6 +381,30 @@ static void account_kernel_stack(struct task_struct *tsk, int account) } } +static int memcg_charge_kernel_stack(struct task_struct *tsk) +{ +#ifdef CONFIG_VMAP_STACK + struct vm_struct *vm = task_stack_vm_area(tsk); + int ret; + + if (vm) { + int i; + + for (i = 0; i < THREAD_SIZE / PAGE_SIZE; i++) { + ret = memcg_kmem_charge(vm->pages[i], GFP_KERNEL, + compound_order(vm->pages[i])); + if (ret) + return ret; + + mod_memcg_page_state(vm->pages[i], + MEMCG_KERNEL_STACK_KB, + PAGE_SIZE / 1024); + } + } +#endif + return 0; +} + static void release_task_stack(struct task_struct *tsk) { if (WARN_ON(tsk->state != TASK_DEAD)) @@ -807,6 +843,9 @@ static struct task_struct *dup_task_struct(struct task_struct *orig, int node) if (!stack) goto free_tsk; + if (memcg_charge_kernel_stack(tsk)) + goto free_stack; + stack_vm_area = task_stack_vm_area(tsk); err = arch_dup_task_struct(tsk, orig); From patchwork Tue Aug 21 21:35:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roman Gushchin X-Patchwork-Id: 10572371 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DF0581390 for ; Tue, 21 Aug 2018 21:36:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E01542AAB0 for ; Tue, 21 Aug 2018 21:36:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D2AEF2AE21; Tue, 21 Aug 2018 21:36:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D1DF52AAB0 for ; Tue, 21 Aug 2018 21:36:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C8366B20AE; Tue, 21 Aug 2018 17:36:34 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 950306B20AF; Tue, 21 Aug 2018 17:36:34 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A35A6B20B2; Tue, 21 Aug 2018 17:36:34 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by kanga.kvack.org (Postfix) with ESMTP id 146FC6B20AE for ; Tue, 21 Aug 2018 17:36:34 -0400 (EDT) Received: by mail-ed1-f69.google.com with SMTP id g15-v6so71405edm.11 for ; Tue, 21 Aug 2018 14:36:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:dkim-signature:from:to:cc:subject :date:message-id:in-reply-to:references:mime-version :spamdiagnosticoutput:spamdiagnosticmetadata; bh=zxLfJma5W6+QeF7kJqnEsF/bgjjJ2NJLkMmgfAY0Ecw=; b=VB9lr+b9aZxmZ5EV0LnPUXR2kisZMVFzCJNAPi3ZSgv6gAuEqK9WEWRBBC44Q4BLJY 43LOmpSKsbkTi8bUgL6IYwU+BQ7PCuL0YRAvjm0b3JU3fR3oA1WcsADFYo4b4K71sKZR KeojcP3rot1TbUmwOmw2r5Rm4asaEzWLNtlrDTqCiErk7uN1MFhYoLtjCc/VBYzzGj9g ehBhUbI9o/ghToPKT5qfAmDMYS3/t+3hKKL8Ba0MwASgWVRZGclrrDXwu1Xg4cLuj4O9 SllmAJyw3kJggfuTojkpSaaZ9X1KhgtdDKgoRI8Z8qN5BsY2CUkyX967MgeZ7enkrx6g sxcA== X-Gm-Message-State: APzg51CPpJYpn0RsmAzzIJ/ZXNIC7aQFQyBNbxaeXWucGDDWDhyCCIkQ LwXQnO0zgfVUkIJfK4koqixgI3OnpW028tozNakRmVRCa1WPBLJNxbcaM5VNAn2iXt9rUqnJB5i 5NOylJM6nQMqnUohKBWMYnjMg/DPpVcvaIxbdeGXP3SbcqxVw4NLgcZ5Kk9wMAQMXEw== X-Received: by 2002:a50:e79c:: with SMTP id b28-v6mr2921238edn.246.1534887393478; Tue, 21 Aug 2018 14:36:33 -0700 (PDT) X-Google-Smtp-Source: ANB0VdbW8XUUTXiX7JDCy9nQzDtSzAkW7fcoiaThMydIpYPw2BS9UiSbd1wVVxRrEOa8yzeD2c1U X-Received: by 2002:a50:e79c:: with SMTP id b28-v6mr2921196edn.246.1534887392323; Tue, 21 Aug 2018 14:36:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534887392; cv=none; d=google.com; s=arc-20160816; b=tAI+hboMnk9P+/LO57T7jV7EEveh+aNgWOTHzZ6+MjkJDiW2dufhsWWPYXQU8YaXgd cJwOiEgfWMR22zNu9g8tYtkQSBCFfqxhEtFTpu5/ru+7pWk2SA3OYrBdl4ytqlPud0Rm 4/jvv1aUhUfxJ+svtc0ihmzpn4ov1bgH2cWV68Zg/Nqyj2jpwP/5V9NrjXsp3nKmgwmS i+zqdxgrSxUaUsSRVMgmV8RXeFpjpht57GPDjHbre2WUm30XusbucM1H/yjPqJPjAngP FMt2kjIejokBHVcbJHeo8x8M5Eg7juUNq0U3hAb9Jfz+h12iyD+A+VvPR1reC8bf2oso t7EA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=spamdiagnosticmetadata:spamdiagnosticoutput:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-signature:arc-authentication-results; bh=zxLfJma5W6+QeF7kJqnEsF/bgjjJ2NJLkMmgfAY0Ecw=; b=mgIUQiNN+GeiUsKEpRFoE5IRvL2aaeYabQh0GBc1YLl9IAqejmdSr6wfzAr7Hi1UCc bIMFZxY1ulC92G/vEjnpX6OsOT8DneslbAo/ZAXORcFgQ5nllBx3XDPlGTBz78nKReOC k+Ro8+TfDWAFKjHHbCMYRQtnAq9FRQTXAi/lulsZy0N47vkpiXL4zPGZaIiXN3aeFkom ORUvGpyaxuUmnTow23i+2XPgNSAt06E3Or2mAIVNyBQnTnVPqm40yzC+jqEDjsSCx/u1 R6NxH2n1p71Vt9H5ODrcr7FmBQmrQn9KaQkb3Ej6jvQWHfQICbKaAeo3oTN9GqigsDRK fkTQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=Oi7kisRU; dkim=pass header.i=@fb.onmicrosoft.com header.s=selector1-fb-com header.b=PQBAuvZE; spf=pass (google.com: domain of prvs=177171ea8f=guro@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=177171ea8f=guro@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id t16-v6si198265edb.65.2018.08.21.14.36.31 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 21 Aug 2018 14:36:32 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=177171ea8f=guro@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=Oi7kisRU; dkim=pass header.i=@fb.onmicrosoft.com header.s=selector1-fb-com header.b=PQBAuvZE; spf=pass (google.com: domain of prvs=177171ea8f=guro@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=177171ea8f=guro@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0148460.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w7LLYGVi010571; Tue, 21 Aug 2018 14:36:26 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=zxLfJma5W6+QeF7kJqnEsF/bgjjJ2NJLkMmgfAY0Ecw=; b=Oi7kisRU44EBMcHZZOg9Q+dAI7wT9U+eT1qWmZJzYw6PaBShjtyb+3Sk6MjzCqAql0Xo IT4QiY6MDQv1s3yMFN7VAtT2sffIN/LQ4Z8rbW5/6gZ6yxjPB4Zd66TnpR1crpjCz2Wl iqvESvmmbkVGDjQ/Qdbzp8SwUUVoh29whmw= Received: from mail.thefacebook.com ([199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 2m0ssj05p6-10 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 21 Aug 2018 14:36:26 -0700 Received: from NAM03-CO1-obe.outbound.protection.outlook.com (192.168.54.28) by o365-in.thefacebook.com (192.168.16.14) with Microsoft SMTP Server (TLS) id 14.3.361.1; Tue, 21 Aug 2018 14:36:20 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zxLfJma5W6+QeF7kJqnEsF/bgjjJ2NJLkMmgfAY0Ecw=; b=PQBAuvZE11i21vm1cwg9E4vHPd93gMtSnq4ssVqc/rDUC2qNvHtgxd7oBAiGyTwJ3MtuizTkQhJ5+/K85Akon5W3WFA4Er4Y7jyFrlZwn17sm42jGKt+L02u5U4ti49U+xzltsPaK7kPqVWGaGNsdoTn2UJNyvS5qxG6Jl5bTHU= Received: from tower.thefacebook.com (2620:10d:c090:200::7:b0bf) by BY2PR15MB0166.namprd15.prod.outlook.com (2a01:111:e400:58e0::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1038.23; Tue, 21 Aug 2018 21:36:16 +0000 From: Roman Gushchin To: CC: , , Roman Gushchin , Johannes Weiner , Konstantin Khlebnikov , Tejun Heo Subject: [PATCH v2 2/3] mm: drain memcg stocks on css offlining Date: Tue, 21 Aug 2018 14:35:58 -0700 Message-ID: <20180821213559.14694-2-guro@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180821213559.14694-1-guro@fb.com> References: <20180821213559.14694-1-guro@fb.com> MIME-Version: 1.0 X-Originating-IP: [2620:10d:c090:200::7:b0bf] X-ClientProxiedBy: MWHPR1301CA0004.namprd13.prod.outlook.com (2603:10b6:301:29::17) To BY2PR15MB0166.namprd15.prod.outlook.com (2a01:111:e400:58e0::12) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 40419c43-56d2-49ee-f824-08d607ae207c X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(2017052603328)(7153060)(7193020);SRVR:BY2PR15MB0166; X-Microsoft-Exchange-Diagnostics: 1;BY2PR15MB0166;3:TBL0qJTkjZiWFKU7w565iphd9wDD/jpRMM8XT/T5YoDaOLV2j0Jwm2uvpaXazPwhh0EAiM7TYJ8K5h+82/B49QzyZAf4dukl6hzNpiTk5Ic5n47GNG1b/fBY1nlmqlESSlOx5+A0jg9js0eoCklIJTLQEYF6Ty0gluDqHTzaJ8TKO5SzK+1qXdZvkvBjcVjW3WKIzOo5jTQIs7kxThQMrm5Kx4Ku6YztmCVkbd1gr1oU0jNBt5e1bi9oeUfZwFDK;25:2Oj7blzgBNaTQz3a472uDfIr4SuDLuGU7waAnlewMh0NniB+DEU1CVj6rR039ndi7KvHG81YL2yTOXaA+X9gBUYFQH3IuYzgWVjPrq+aE2MsKOQwz4/PVRpGJAm2zOoIpyCINiwgy59bJZNKkWFZ6s9Mqs42WAcWFxZWR2NZkOWV3wE4DvaKw9YwHnqFbNQyWlkaL+sFnBBH2O0GaZTqs/2VtqyvCxuDed6Y+p5ZS4cSfTYq/XtALWaV3DaLokXoGp1jVsphQUx9HykYXreG042z/SDdgtmLZk1nQNkCjXxX625DSV2lN0Ww9vMoPrI8r9J4Imair+XAyVtitIr9FA==;31:1ssX5LFspXHxLr1ukFUTti2dGkExnsm6F8E/6JT+7zDZ3FFBTgQj6r4Fsso8FjWjxQz/bUhg0ZQB1PMhxZnSPUetLSZ8Y0JQVzPWVm5pUXBbU7wZp/aC7DS7suA7xB/FCL7JlS4dHWa9eWE3ppKP9I106O3bdDmdqNZSH/4oGutSyzHYY9xAb14I6YRqsSSEDi0p1qZmvr40Vr+n4xHSrH0btKmaSh0GE/xudgPpYhY= X-MS-TrafficTypeDiagnostic: BY2PR15MB0166: X-Microsoft-Exchange-Diagnostics: 1;BY2PR15MB0166;20:KYBrQAu+OxkB/UuxpqUeKoI+deFee/3XzZpCO+NGYupvn8mNnZBoMku3ECyUJtx7bUQ8+pi1pVziDdbg40PDP2JMkKVjzr11eLVK/OBXA5gWMtjJjjPUQUodjtLVdAChHvWwbh4gDDxcsldzSaQul/47JSnKjFur8ec9/E3c0wPOyxjX/wuCE/urvOgeoCnAKIJhQD93cvh/wiOYklO5yttxFOfEKF4iC2xDUFH1SxKZwKDHOeyDlxvoFO7TWHD4Rhg2yOJQoXT+n3aTKznMD3SUkMUhDkyVscBFzDg2HUDf4cph1LP1t0tW9BWz55imq3Q0EOZFFeYfUFHJitGiAxL2dInEPfBt0kAN3y5ALu5deTdrcx+CN0ufkUKuvhF7NxiSm1dK2QZOlAV5C5dV19JtZvb5NwflLNnF8XIZq14aNM2ANzPhFcYwbuhQB5HEOXlKg/fOm01IRID4axgow3cFM30p9o93+IJ94qqFnTGuF3Hyp+ubEMrov/lYmIIG;4:q3X4FjkMQHdHiSqeUb91Gi66R1VZaNAAhFobAZMMksmeqzJzv4qVIiIp9gJ2NnOqpxj5v4bCTRrNjHPOYwXxISSnoNFsKxOsD7zVbVtrN5BLYbsMIFH1hvD7rknUrfmieXOQASV7bpgp0vZSg3Atk8V9KtzgMedft8vCMYoh9wYn1oCVP25x3QAlNQmqeN8zckR6yiHxGpTflIGSyMKbw92J044nOYvkKA846buQOB94Eup6nSGZr9k9O564sflXEOSmeWI0I1hfKl4e+d3Z48nAnqXaMj1ZxlFYVjWlPuRPcXxz9Z81lSdjz3zjuklkSpnttGLLRIsBERAVsjOI1sv5nQvZmzl/UKvGdecvBGoy0gCjwZinbNPN06TPsBpWGKmWMW7i47CT597o2jhBYxFp1ZOAk8lWGS8BOw beJ4g= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(85827821059158)(67672495146484)(211936372134217)(153496737603132); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(823301075)(93006095)(93001095)(3002001)(10201501046)(3231311)(11241501184)(944501410)(52105095)(149027)(150027)(6041310)(20161123562045)(20161123564045)(20161123560045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699016);SRVR:BY2PR15MB0166;BCL:0;PCL:0;RULEID:;SRVR:BY2PR15MB0166; X-Forefront-PRVS: 0771670921 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(346002)(136003)(376002)(39860400002)(396003)(366004)(199004)(189003)(53936002)(68736007)(39060400002)(50226002)(8936002)(25786009)(1076002)(2906002)(6116002)(4326008)(476003)(486006)(86362001)(16526019)(97736004)(6506007)(186003)(386003)(46003)(2361001)(478600001)(105586002)(48376002)(50466002)(54906003)(446003)(106356001)(6486002)(2351001)(2616005)(11346002)(6916009)(76176011)(52396003)(305945005)(6512007)(8676002)(53416004)(69596002)(36756003)(16586007)(81156014)(47776003)(81166006)(316002)(52116002)(5660300001)(6666003)(7736002)(51416003)(42262002);DIR:OUT;SFP:1102;SCL:1;SRVR:BY2PR15MB0166;H:tower.thefacebook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; Received-SPF: None (protection.outlook.com: fb.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: 1;BY2PR15MB0166;23:rFusLMPsguwOOA2ZTbxEEV+B6UIijFRTnsNwcZnMR/5dL9RFF/cax+ECy3ZmWEQmxmZqznJIlUdEVZ/0eCi8WqjNiLWR8aRZ6AvjP1pMjwXg+qNiBK5e/PeULaNSnKqHTIGGdbaJqGlqDgimfF84qTKUqMjdaui2yhqdxNBF0IZBa2S2KGmLh9x0FwfPgk/FctI0GuSoHfgP05bJofN5p/kSHW6K9ezawnf9/fj6nW7Sj4FSp6b12RMF74bIGefoTYDbd1Thszc18h7HpETHB17KBJb4tZWhbGFuWkHmakNzTArpX01Xun2YGcjjwQ7mj/LNBLPgo7JA59Tafx5qowXENIzsoulAEASdrcO0aZUcRiAYIomZS4MZaDhIvlPYd6UZDytnJGEwa9JbseX9zszuB1cyljHvCaEeBbXP9xivcKQPRK2orEcgJgM30BC97+/6Y0IaHqupOXmBGlf2gWXFy92k7TsUwI8hje5CqGj7AkdDPkW8oT0vncRPaZkSpvfd+8dMjOQZ++WkRKqGtlmPwLxVxJgttE2IqTh1TbGZOs2Uee5KymiyhmZjJ8GCe23yeyiD9Pin60pTH/Kheck4MGh0UL/pW7+Wl4WQa+9/YN+lzgc8LaAq3UyrvI7UD3/cv+H1o/qhEKDCyEbOJtVBahU/AuKfta2ncPMSuIhNGZy071kjIqUQERb1lSeqEFz+pnfqccYIw5se/hlXeUB8z0V8AuCS9pEjYn0w1iXygW2lpxbwL7xGyZbZwszZSfWbe/WhiUdQMsSt9ovyYlLuEq04YNeXY5i3/UpuygWVWOZ1ANMwg+Hc3OROnzVVFZaISoWPS89uhyidz29yw0i3QqD5DExOYRhFgxwKmxfbEgT0iwFJeC2xj6LHBC5VSKOgIb4WBDLZPMosUMOtolewLBT7F7MbXOph8ZH+TKZHTsqZFLagcUrBBA6XnCZke6VrB4+sryDiMRY5sy9zuTWOv3fn/WT8GSgIg12aw0HJFKgQsOt/Qs5/Kr9RCpG7ZAJxek+XhYE10ttZjMBTCh6N0FXFhwYC3IKW9UH03sjtHLWFwL2E6nR3mvgW0QabKdYF+eCzRrSha3x0XVm6pjcO324ap5dDZEBSG1DdYjC4p+RTNvuyCJjTYRUFl4BQuMPUzV28A5jtWC336WpxFpitPJ6uontTRnwtgSJMd42EojiO2BW9O7GllA9C4irsnx+HfUjalD720czDGTd6KYtM+a0e97/xF27mfNs99tq/FnJX1HG5k2DxC+QIM16++acpl3COeCfA3X8VUaC1Qw== X-Microsoft-Antispam-Message-Info: kr8NafbeI82blmvsxo/yRtCuWkiSdENdFZcJBrhKQ493F4+PZ2VF1t0WqFPdk6/TxXxTU8rerXcJ/0t171e70/4vo/qNGimLJxq0I3oSM5M5//ADC2LbSG7TgSHRGNeOdqIE6EQ/YkO52olhe7wzueAlxUECD3Kgu3lONQYiutQ1mexq0zeyKlg4boICIwflN19S2FD+9mwUBgsQ/Eis8sOvFI5CQkdeqa/iqQOi6Tnd6mDJTqw0O0gJnHiUDs6O7EPrUw1uNASET/a/WTdmOpLBkkL/TBIdVi+BrWFlgUsFT6GnkOI30UV+Jeo+Sew2qbzYojDaSKg66QvA474e7teO0B3O08YB7uSj+hT42J8= X-Microsoft-Exchange-Diagnostics: 1;BY2PR15MB0166;6:15A4IiM76ksmxBdDnlyEcGhxpyqeMgtMcSiCyiOErhw1PsjIU2fJu8GaDUR4e7JDzp+aQXJdi2RFiKpk/nT3fdmvQeWhymzhWs5ojasZcWXJUKTXK/bHphAW8H4o/A6YwdnWdWcgQopdP0hyDUuyJDOZT/0yQ7qA8YCjfALqkoJsEx4wqJtDm/sk1QBkWgA3rD9FVoOj2N6/jEzOKCPkNa1KskaWAeiYvdXI5JD7g4z4d8UElt6HqPVXHtmNm5mYg6PeBMVcjjj7AHUw2eS7gf9Rmhtwn3dHppv/eJBqmndDl5VVvHZmSzTJz7avPnL1ngp5FAxxwaDoK+ACSdJEATsV+IvqsRO+jCp0LHRnJztAvqJKfowQnXJbSmROEw9NUpBy7f6upAo0GHyk2cu9XIfrdL0kjSz9LtB6EG+E0kQ+x00clCgzTYYJG2Rh4mPyVe/dMo/b7U831S3zQYhfQQ==;5:9P+acqHu8LuXWzM44mYKAVusAcsdSCtSMJ/1BEeSYAXiV1AwhNxMOMTfuJpJmv2xxiKIu8gEletzAITSisZdAZ+Q2iPUhiqwqaYWDW9lkq8WgcPCTERWxsQZPrU8FJ9M7hx61p3evs0eZXrxqKKKWag5sz+DAgHKVht00yAABXY=;7:LFz0z1301GGpmKoFKBP1b9P0Izx8PkV46mQtBlMGSo/HhzClJFGghtOxV2BUmT3jA1LPlzB/RqWC4NNmymIHPXRqEY6urad0PMIHvV2lN7i/WMoemYPSceYiTT6gOZ93jXDK8UtIobAo10Ph+cBRYm2gY70zUYHqlVMEvfJfKhVjvV/BPSfrCgl1JZrQC+2McpZrTGVSitjijOnbNG1Dv44cLiC0kVoGSxuXx/HtgKokGfMIm5+1WUmHKiF0QqY4 SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;BY2PR15MB0166;20:AWf5XKKK8ByplrzGOml1j5BzxpEV44GJgySP+SLB+dslmMNrFl2/MesYuMcXT6d+DMoW1Go/Gqe3E3GOywHPNyLFoiqoKkI8NZMELMhPRcgWWqK3GQ+ssTCUX2wo2kMe1qDBGCxs+PHOCwXYnLH5g4FL+TRBWqiUAbVgFsvOyhE= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2018 21:36:16.5581 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 40419c43-56d2-49ee-f824-08d607ae207c X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR15MB0166 X-OriginatorOrg: fb.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-08-21_09:,, signatures=0 X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Memcg charge is batched using per-cpu stocks, so an offline memcg can be pinned by a cached charge up to a moment, when a process belonging to some other cgroup will charge some memory on the same cpu. In other words, cached charges can prevent a memory cgroup from being reclaimed for some time, without any clear need. Let's optimize it by explicit draining of all stocks on css offlining. As draining is performed asynchronously, and is skipped if any parallel draining is happening, it's cheap. Signed-off-by: Roman Gushchin Reviewed-by: Shakeel Butt Acked-by: Michal Hocko Cc: Johannes Weiner Cc: Konstantin Khlebnikov Cc: Tejun Heo --- mm/memcontrol.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 6a921890739f..c2a254f74f30 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -4518,6 +4518,8 @@ static void mem_cgroup_css_offline(struct cgroup_subsys_state *css) memcg_offline_kmem(memcg); wb_memcg_offline(memcg); + drain_all_stock(memcg); + mem_cgroup_id_put(memcg); } From patchwork Tue Aug 21 21:35:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roman Gushchin X-Patchwork-Id: 10572375 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D61E0112E for ; Tue, 21 Aug 2018 21:36:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D803A2AE2C for ; Tue, 21 Aug 2018 21:36:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BE52F2AE3C; Tue, 21 Aug 2018 21:36:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6DCF52AE69 for ; Tue, 21 Aug 2018 21:36:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 50EEF6B20BB; Tue, 21 Aug 2018 17:36:53 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 495696B20BC; Tue, 21 Aug 2018 17:36:53 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 311296B20BD; Tue, 21 Aug 2018 17:36:53 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-lf1-f71.google.com (mail-lf1-f71.google.com [209.85.167.71]) by kanga.kvack.org (Postfix) with ESMTP id AE0E46B20BB for ; Tue, 21 Aug 2018 17:36:52 -0400 (EDT) Received: by mail-lf1-f71.google.com with SMTP id q35-v6so1412031lfi.1 for ; Tue, 21 Aug 2018 14:36:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:dkim-signature:from:to:cc:subject :date:message-id:in-reply-to:references:mime-version :spamdiagnosticoutput:spamdiagnosticmetadata; bh=AtNuxCAd7TECN28SamN4dQlQrOCShOwPTHfKehhUkiY=; b=bseLalj50KifSTkdY9ukqnXWEHW2/YDPKeVesuuBrG3HN9gGjDkVJf0P5qzOkmQ1j1 3HQ2LIbrGZD5q78kVJNusIK1mjEDeCmV7NJCq7vxyYj2o1FpIanQMbd8bH6bWwBlAPp6 hF7qsSXJvM0ruRUlXBB2N+KmDZzs/x6ZFKs4ULX2SSHjYq0iDm44d189x5l091p3K/U8 Z4EyUaHtGLj+tYf/YDTZwhao5C7L4rBh38HuA+gCL6+/pj54tP+z2ELzt5CdRxcdodd4 CszFUns0n2PhGSHVFvy4/O2ex6MOJsARq4NPXoJD7Wdy9yEdEVeK1GwSeSWnHjpOnx28 o0vw== X-Gm-Message-State: AOUpUlHH4OL3u31TPW8ua2pz8janj1Ea3nvN8XmKcyjt8P9kxg5BBFQ4 FlIpodW4STVwVPBctMLoylg4m8YVB6NNXUvMDZtiS9EB8sO/AXRyUpVjKPfOGkXv7X1tXTsDqKy EvTw3mpTcTCtvT4qhd9ap04RCJV6VnA1P4iRi2LyP39nPxmBpVp1j6NJfwP2vHMRkYQ== X-Received: by 2002:a2e:4619:: with SMTP id t25-v6mr36959655lja.96.1534887412033; Tue, 21 Aug 2018 14:36:52 -0700 (PDT) X-Google-Smtp-Source: AA+uWPzyp11KRg2KUq5dt9GUWPkCkhtLAmmUft7bGKcUeXOeONpyRk8ezgBRodJVPwcjHQpK/Y7v X-Received: by 2002:a2e:4619:: with SMTP id t25-v6mr36959631lja.96.1534887411121; Tue, 21 Aug 2018 14:36:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534887411; cv=none; d=google.com; s=arc-20160816; b=P0+MJOAequ4Y3XjKjrNnAdd7pSx1fG7TlZ/lR3XCRIBk6I/FoJbHyPN3iJE5Lq+gR1 4CczJtSmvBZ/nRIcNSepQsereEBJrVe3CzQboRc5XPQo2mcE6/N22u65ASUdgltoS5qn z0ZlF+8jnKwcJZNjY83DkkXsVzvIPk2Hp+W5NY0KXrYB24y10xNaIK4QhRVO0DTHbDDh 2HgJ+Yi/0+N2ob371Ss7Jae5hVOK+RQeW0IUFc3dFtBep+giR6lEVMU5l6tpQVm6+hfP DoLXQg5pfudpxwT0/twwXg/tb5E5HZzUgw5G5ZVuiXVROaUMhNJ4cqXSSu3m6tYDIzLT MLTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=spamdiagnosticmetadata:spamdiagnosticoutput:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-signature:arc-authentication-results; bh=AtNuxCAd7TECN28SamN4dQlQrOCShOwPTHfKehhUkiY=; b=nRpNkeKj6jZ743PYzNRxPOGy8uQ5F9JZlIIkn3eYYqa89Mp+jFyunjHZuC3xOEiq7w v7s1d1+/25sUBczsCnMaFNTWaK8AOnPdalXGy0ddV/mXyWOem5/OwCty2NkyC8pry6tF bDzAunrTD3eCoMkTD06fBMGCV70vxHNJ8TYYfa7Nu9qib0QaPlZgO3VDIQLbmIMEemxm uOKKOY2DI2mcPcBYsg8ipESisnInH7A0E9ghafaG8/IKIuvpZmNCTFr7SHHGu3BlDZS3 SM7ull2wb2q3PgICyGgmqM1sT3C/n5acqVMenrS+LX/64gYeMSuRqL8flQkv4b5wMEDO jQOA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=DGg+vclb; dkim=pass header.i=@fb.onmicrosoft.com header.s=selector1-fb-com header.b=Lgq7d7uW; spf=pass (google.com: domain of prvs=177171ea8f=guro@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=177171ea8f=guro@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0b-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id k16-v6si6243740ljh.387.2018.08.21.14.36.50 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 21 Aug 2018 14:36:51 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=177171ea8f=guro@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=DGg+vclb; dkim=pass header.i=@fb.onmicrosoft.com header.s=selector1-fb-com header.b=Lgq7d7uW; spf=pass (google.com: domain of prvs=177171ea8f=guro@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=177171ea8f=guro@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0148460.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w7LLYGVf010571; Tue, 21 Aug 2018 14:36:25 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=AtNuxCAd7TECN28SamN4dQlQrOCShOwPTHfKehhUkiY=; b=DGg+vclb8Efw0NXz5xBuHbfrNiteeGUPQL7H8PG02tFFC5w/1by8xyc5mw2N7GhzJvVH LCe3Q4Q6cHDf/4ll3BlSd37urE/oJBQzSZHgg8UeCSCFmTnCurCcvLcGfHCLsB2g/gPw WUU0xDVFfxZJTLRFJBJZck5ZvSg+qnnH7J8= Received: from maileast.thefacebook.com ([199.201.65.23]) by mx0a-00082601.pphosted.com with ESMTP id 2m0ssj05sg-1 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 21 Aug 2018 14:36:24 -0700 Received: from NAM03-CO1-obe.outbound.protection.outlook.com (192.168.183.28) by o365-in.thefacebook.com (192.168.177.29) with Microsoft SMTP Server (TLS) id 14.3.361.1; Tue, 21 Aug 2018 17:36:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AtNuxCAd7TECN28SamN4dQlQrOCShOwPTHfKehhUkiY=; b=Lgq7d7uWgarELyI34TnUPnI2stRSMm4W9mKIh4PRGBJjiLf5hLuMGIgPc6Yer3kPKdDSLXc5ZI6f5KqFh3BKdWEcRbTaCHvPi21Hkj8Euu3i3qLNe1tWtVGBLa9xSadZMAYh+wCEGjIFCE97roxL3bJR9gF08WXNtdNKpTwHBq0= Received: from tower.thefacebook.com (2620:10d:c090:200::7:b0bf) by BY2PR15MB0166.namprd15.prod.outlook.com (2a01:111:e400:58e0::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1038.23; Tue, 21 Aug 2018 21:36:17 +0000 From: Roman Gushchin To: CC: , , Roman Gushchin , Andrew Morton , Johannes Weiner , Michal Hocko , Tejun Heo , Rik van Riel , Konstantin Khlebnikov , Matthew Wilcox Subject: [PATCH v2 3/3] mm: don't miss the last page because of round-off error Date: Tue, 21 Aug 2018 14:35:59 -0700 Message-ID: <20180821213559.14694-3-guro@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180821213559.14694-1-guro@fb.com> References: <20180821213559.14694-1-guro@fb.com> MIME-Version: 1.0 X-Originating-IP: [2620:10d:c090:200::7:b0bf] X-ClientProxiedBy: MWHPR1301CA0004.namprd13.prod.outlook.com (2603:10b6:301:29::17) To BY2PR15MB0166.namprd15.prod.outlook.com (2a01:111:e400:58e0::12) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d642fca3-da94-4a09-ca7f-08d607ae2110 X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(2017052603328)(7153060)(7193020);SRVR:BY2PR15MB0166; X-Microsoft-Exchange-Diagnostics: 1;BY2PR15MB0166;3:LIBr+b21RooN9jRroTXs615jOiIuFKiW+AnvWZcTWQk1CZM8WocHJde+ADyH39Mm7+bFfWZmo7gcY3VfEy6yacHVHLh0U3zmQNDOYvpHAmNC20TYufAFxo9rRAFxKSYsx5PShqRaIVOwOZ9Rkc1vzLuyyizFzH9IMJ/g9IH3B2okbITxvzeYeak22o4myQAObGjlRWss6CUubQNkXA+oNgRzFsFpGyHuUL/bDtCJctwOfpHWQaG2J9Svf8MRyWQy;25:XWtKi1pOzsSvDscwOwVf21Ar/6gs1YH6QC9s6Dc2PWr2n1tqS0czPmaGtmcGqlELgcFY7i95lEdrrghgY6L3JysrtVcKDUJu2Mpe/DJjGGMmLXeUSBVALAuAje08A0suCXaKmUPW+DNZWp6vDlulVn/sSOqfeA5Uww8kX1Tyav1zwTXouCc0yqq1nJt5C5AVkT5y5xmIlZmkBtdbZwx+Q7TgKAu9L46FIGTmJjm44mFCfw0sz8lYGYxr1KSkksP+wXvtklUucscgxvYoX64yT3CZST52dloDdmYeSx1JuVRyInSu/OGc+qigAXdJD9b8/foKmJYmQWtWT+NEIusBag==;31:ZVL8h2o8SHYz40lFhG6k7z29gRsezh9UhvaVA940Bhi7aE/VIFdicJf8JSHXB490f9uFdKccTrC6rPxs+g4fxryZrcvnNWytrO9h5NTw0KAxg+Q8rMOpTkhUBL4BBr6yW17grtx9H0s+2OoIRYaURHalYA1a4SW5NGETqkPHl37/fMp2xK0tSb42i663p+5BVT8EqsSEy8ak+E+SacGZZr+YAe6gUdbYDRje8l1L9p8= X-MS-TrafficTypeDiagnostic: BY2PR15MB0166: X-Microsoft-Exchange-Diagnostics: 1;BY2PR15MB0166;20:GOgscW/zu+6S9gsbsw+9Hy61JZ50qxlxt6bYSIWgSf94dxTUWx4tj1IX3I+xPRonC4P5441QR5ed8rea/Orq5ormLWU19+iZQlR0bRi9CQt48EfnuuSLJVsS8EKtAQgWJ+It+aTEEE7jyYcD+0qP88EE8LfSQU4kzN+nrZ5WNr2XQ+dl2dUtUNMSAyCDp3sfuLx0+q/K0zbKYh+AXVF2sIJd4QYyE/Py52Zn4eOPuU84nHvMkn3H6rjt9pKPwthhK3sPR2rGd4lJDgtw6RnqlBP9ZqtgVaeW4jzToXQv3w9Q34U/PicCykyqyOIp5x3YkS6XHRAVh6ddutvx571ZikgDBESRcMkcdK530U3yRmnpRaDujyLNICg/LG8hFLhbtuVnetNL+mql2/tILn4DBC/mfmcSwqZNB/Gb6IPUptTEfJTvtPawJcpgfOwtOWuXZqZpeGbdL2Cms2FEhEiduY2cDTZMP9n8t+pG11BY3w8bO0lbDg0Abne10nSkCKwH;4:EOjS/smeJd1k98sbq9ecI8oer0bxN4DadOT2HlQtXGoF14dKeDa+x8scsGC/BAhhug8uQWycw3nm96imIg6pgBwYO/iebaF6u48u2L6LJpGd3+Gjazl4dGIxt2R2YozfeYzQ5/zR7G/dexQQ/XhJ4aklRoCBdHDq6VHLHz2Gda3hJ/YScm34cKr89LjZs4iZuA07mZMdIAU+zfIxGHg9Gul8d3mjjlodhI7pmfbJFL3eMvpD/9/daNox8K+YzeZFsMS/eYALywaaF8zjNAoF0SllGk9TuZ5VNHnXttkqh8nUIOxM8c0UenuwOJgkKRgPlyadQCpez5Mh33px6K7OHuEbjOTi+E+CEvYTHjMHDSo= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(85827821059158)(67672495146484); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(823301075)(93006095)(93001095)(3002001)(10201501046)(3231311)(11241501184)(944501410)(52105095)(149027)(150027)(6041310)(20161123562045)(20161123564045)(20161123560045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699016);SRVR:BY2PR15MB0166;BCL:0;PCL:0;RULEID:;SRVR:BY2PR15MB0166; X-Forefront-PRVS: 0771670921 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(346002)(136003)(376002)(39860400002)(396003)(366004)(199004)(189003)(53936002)(68736007)(39060400002)(50226002)(8936002)(25786009)(1076002)(2906002)(6116002)(4326008)(476003)(486006)(86362001)(16526019)(97736004)(6506007)(186003)(386003)(46003)(2361001)(478600001)(105586002)(48376002)(50466002)(54906003)(446003)(106356001)(6486002)(2351001)(2616005)(11346002)(6916009)(76176011)(52396003)(305945005)(6512007)(8676002)(53416004)(69596002)(36756003)(16586007)(81156014)(47776003)(81166006)(316002)(52116002)(5660300001)(6666003)(7736002)(51416003)(32033001)(42262002);DIR:OUT;SFP:1102;SCL:1;SRVR:BY2PR15MB0166;H:tower.thefacebook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; Received-SPF: None (protection.outlook.com: fb.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: 1;BY2PR15MB0166;23:y0hBRbMJzP+aNudL97O3T5kyvLzHJjmpSEXCS3DcZALUZJgzhSCgQg9j1mA7BlGBEngDRouKde43EdX8k4oRLvCk3yY02i+4HrXwj471NszSe/1X4hT2hkSMM+WdaHBG1zqYUi4uMsZLyNeeu2is7w/ZWMfQN8bCvaLHJZSayEdxt5YwiEDNLUp4V5Ieg3xZqMKXJVQcpX8tz0zZYx9Yqct2QF5u/GdtbSgQpKffccad+uZfC6omSv7rkreWBcfKdTE4tjgKBffR5x00PJXS5WvhbisSw89aVLdjdjqzovMXdE0Pu6QJ8jALtBHBDqlc0WEob87G7shprWHiJUFtMA81Pg8VcDETtYps+uJNnu5G1bfGNrG5wqnxRCSKyvXyvsfat2YPYoRlfYfrMFDjcUPYYdC0bWoEFs+e3YhQWIe/Es44B6bGy4/dlLaIK6U0vaj3IwaCwtGarEbbmXh4qnOOISJcxDLtDN3NY8EXR+4q2/HO0pFviiqbNRWg+6KtkBxnILN9JUdm7utGuhuANvM8Bur4Ppp9dBzyFlzUNjjq2doXyOi13tdCBAk6jfw2/g/tszozbLpf3rfDRabZ5Av49pI8v+c3e1M7h69P1SpxC7i2G8Md/s1IHwmavC1gibCN79nISe3Iy6WXZGnEem7O/PbCyjjuTt4AD0yPYAI5rih3tlqfifIvJ4K6Cb1fPYD/KmnGv+7y+ienOHlPFGc+bU+d+sRPvrt8I0jathsE0xwGxJ2cbobnhmZ3j6bfxLm4r6d2uEvHJ2UtMPRkv8DuFiAQ4MNEC9Zb9Qj+MfSS54HYRd7vSnQWfePP0/G09wOS//BgWbGkrPsTJIdscHGUFNhZSksQnRQQEVB+xVcQ5l9cKPQctAwmK93kKFjes2lQQF9i4xjFxSTuaU6JHOkKsrVAVz86jrny9lfvMVPI8AbwqgNzB325JWOyVQpjtv1dF13yw3OKWDQjnRwYJtVbTrJ2p1czC5nj8bEsuusxru3OuboQWN5iJjRHAQBmi9Db/RF023H7Pw4rsrmVHOc1abQZ3wjnolJ9Wj7SrcKTefPg4U9ytMGwMzGj96rZ1fkIyE1vBDAueQG4sPKHrbIL1MKbT0dZSm88rvzbrH/1DxygudM3sTDXESXofPFyCxkp0RIW+Drd9MeWMkrqpVCYElBfjHDFOVXZ5iL5ZVL3TJIp5r1loJG7nPEufYIY6XRhIIJUYD+OxkwMlHZIsxConST7iOSZ6KcEqk5E6f00ofIiKBPiD5588I/eqmf+hABEJXetN/az0G5jwmnHZHhiZYn+OCdCMSlIuH2D5uc= X-Microsoft-Antispam-Message-Info: lLZoO6CUS+IvXEWqzE2a0QtnqRqrK/nGorfQ5fou4s26qViR96FmMqLF637os8uC19GKDDCkzy7MrHE+ngFIvTGFlOdT7XlErSBT1QDHWJSfjzAGPhg92GeGG6IAoUb6Z5NLGCx2qxFR+adnmsVbyRej6j5CAyj6riCE+YIo7bbpdd030e3n84umvSO1aXku90LKLpLVO90b6I6sqjXilxZv3LnzIwn/RLN/2W+NeS5aszqCuQZ5Oa3vY0BFKCexTZWUaB0qNzaTfw2sAk/PLNd7ZHczahZh8RMf229RcXV71flOhRauDRb/1w+R/VLwzcPWlzP//5Yddh3IkHGwsiyQnom9WcpX0PdlRt3EbKc= X-Microsoft-Exchange-Diagnostics: 1;BY2PR15MB0166;6:Pja/BRbYDsCuHlrh3yJ1itmlY4CoLbSganc4Cx7ckF5U0hlqoFG5PbP802ndAJmUmvAiirCrLXL9JpK3WGWN6zx01tcgy9mxYvdoDx+gmpZ8J5FkP/wBy1kSEbBmgdsAg73Q+LQ6aaBoxdA7aS9u3ziHF7qfRfHNPDlH863xm3nix4ohS6IZymJ/izp/u4tBjkPX3h8tBg0XryCJd+ggb9Ex4wv9hbJmR0lVhBY98Ab/zOs7y6vS8K1Zopcvb2S/YlAIU+mx333SE455BUvx4tKTOWYmiTR+rxw/Km0SnJPOPMAj4RFZtdrZpdOtA+be7y6PVMBUMkWxamDtScWk9XjcPhqOiMYZCukzmhHE64TCD7x7RpHc3d8eVj9SNtvsdd5avZTY56+UgdtHdhT7BQnT7OrZKbW7wV8iSS7v0lkck3AT5Yo3aNN0+un5dLRMCBYN4MVAVcWclMQieWYkZA==;5:/KAa5zBKS9g8U/2n0OB8vVd2y0iQnsZhzY6ooXa9HBafJJQOQUPRZue9SCzjJIFuYYhs06EB32EYuu989QHUvRIa3JB402RzNC3k74M4MF3RPFXO9U/sOEA9zspqqVFk5M06W8nLc+l5l77ms0L6LR3ulEO/0vFA2biSssEVu7M=;7:mrS+4dl9ehYoUU77B5pmLc3SbxSj6zHC72hcf8tGifsM9da6LMAW0kD1o7hbXXfVHHxvOOpkHTocnBRcAFquO9UssvFaO4KYBibDWY+1ciKGnPvAvkjmZQj8O+6oGPxz/pzMoD1kUd8IbLyXZmJI6ieDZFs/qF3BXyXMmthkSJHZ3UfzI3uh3jUYtqI4GEER+bgi+VfZU5VwqfenZKCmfxRZnCv5Sev9uv6qb/DA596IRR9LJHUunRhuvuemVTwx SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;BY2PR15MB0166;20:EWokAItNE4ut026bbDHsbyZ5DRk6mePeiyLx/68KdmyPckPmFrCGm6Gzk+un71kunFzR48SxJSU5hEkjyOPhVf164cPbhKuNJtMRIQfqRf6jAoidczWnNproN0ITuqlyn19+WvS44sd+bnLyFmOkYo2cH/Ryzs1/5vgLMUD31BQ= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2018 21:36:17.3081 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d642fca3-da94-4a09-ca7f-08d607ae2110 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR15MB0166 X-OriginatorOrg: fb.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-08-21_09:,, signatures=0 X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP I've noticed, that dying memory cgroups are often pinned in memory by a single pagecache page. Even under moderate memory pressure they sometimes stayed in such state for a long time. That looked strange. My investigation showed that the problem is caused by applying the LRU pressure balancing math: scan = div64_u64(scan * fraction[lru], denominator), where denominator = fraction[anon] + fraction[file] + 1. Because fraction[lru] is always less than denominator, if the initial scan size is 1, the result is always 0. This means the last page is not scanned and has no chances to be reclaimed. Fix this by rounding up the result of the division. In practice this change significantly improves the speed of dying cgroups reclaim. Signed-off-by: Roman Gushchin Cc: Andrew Morton Cc: Johannes Weiner Cc: Michal Hocko Cc: Tejun Heo Cc: Rik van Riel Cc: Konstantin Khlebnikov Cc: Matthew Wilcox --- include/linux/math64.h | 2 ++ mm/vmscan.c | 6 ++++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/include/linux/math64.h b/include/linux/math64.h index 837f2f2d1d34..94af3d9c73e7 100644 --- a/include/linux/math64.h +++ b/include/linux/math64.h @@ -281,4 +281,6 @@ static inline u64 mul_u64_u32_div(u64 a, u32 mul, u32 divisor) } #endif /* mul_u64_u32_div */ +#define DIV64_U64_ROUND_UP(ll, d) div64_u64((ll) + (d) - 1, (d)) + #endif /* _LINUX_MATH64_H */ diff --git a/mm/vmscan.c b/mm/vmscan.c index 4375b1e9bd56..2d78c5cff15e 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2445,9 +2445,11 @@ static void get_scan_count(struct lruvec *lruvec, struct mem_cgroup *memcg, /* * Scan types proportional to swappiness and * their relative recent reclaim efficiency. + * Make sure we don't miss the last page + * because of a round-off error. */ - scan = div64_u64(scan * fraction[file], - denominator); + scan = DIV64_U64_ROUND_UP(scan * fraction[file], + denominator); break; case SCAN_FILE: case SCAN_ANON: