From patchwork Mon Aug 27 16:26:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roman Gushchin X-Patchwork-Id: 10577417 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 81B2B13B8 for ; Mon, 27 Aug 2018 16:27:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6B6FB29A62 for ; Mon, 27 Aug 2018 16:27:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5DD4629DCC; Mon, 27 Aug 2018 16:27:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 207C329DB9 for ; Mon, 27 Aug 2018 16:27:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D72F46B4166; Mon, 27 Aug 2018 12:27:17 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D239D6B4168; Mon, 27 Aug 2018 12:27:17 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BEA4A6B4169; Mon, 27 Aug 2018 12:27:17 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id 59B386B4166 for ; Mon, 27 Aug 2018 12:27:17 -0400 (EDT) Received: by mail-ed1-f72.google.com with SMTP id m9-v6so1456725eds.17 for ; Mon, 27 Aug 2018 09:27:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:dkim-signature:from:to:cc:subject :date:message-id:mime-version:spamdiagnosticoutput :spamdiagnosticmetadata; bh=J1cj7ZXg2ZGUw9fXBt8+jsCqFzx1mcYvf7SzwHTz9nk=; b=PWlTVUhG32DFwA75RGRoNwsFHTrh5R5KLncKOeiULvoBQsN7ytaS7323Ie7hW5ZA8K s9KzEMBdAJfPIP8QZrjvGfl+Ii3/zQho7zcQ1pHGNukPverQ/JCXvm/WbPElKcnPsm8I qRNwok8hsXc+5rK5GT1YQ+p74ev2epXHFi3bGcCELC+30Yf3Mlv9tGHZfuQGDh2ddGy0 TA8WVpogLvAlKghBK+CeX5h3YTNU6dFoTEoLa+vtj++4hTXhpy9YrUTjXA18m44lEoMp BcLcPumFggdi2wyT565tgg5IThhDounTEtQQ6Gnh8jSC+z7fZS96GVb18dSQgDNWLHaM qdbA== X-Gm-Message-State: APzg51Dza6QbPVi5a1wJjki9ECecpr22qewiru+xdESdJOzzVMuOMVSH KcCukhL92umC44VQYoiNkNOEBbIIm5nIs67gp68bELuKyuZdvFSeOlygqiTWKfaV8zeZ3H8bOsv B/kLUbHtUTL/JZk+GffrjKmaXljR4h5uEBE6RAtV05qNRSrRw5F6vTiDtJ8ynRh7drw== X-Received: by 2002:aa7:ca15:: with SMTP id y21-v6mr16736696eds.285.1535387236831; Mon, 27 Aug 2018 09:27:16 -0700 (PDT) X-Google-Smtp-Source: ANB0VdYxVG5MVXAE2ON5EWLM0ohCmYp0OoVu50VyKoZcJ2PvN+UPyEOFOwSMDsWCPuNYpHtI3stu X-Received: by 2002:aa7:ca15:: with SMTP id y21-v6mr16736618eds.285.1535387235666; Mon, 27 Aug 2018 09:27:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535387235; cv=none; d=google.com; s=arc-20160816; b=us0PGJCRJMC4W3dq0tZwhiMDVtB07aZ7sm/Zy+HTONUlOtxctXiCtDWLpYiK6nZnFK FHUSAsYQZP04hd5CMkFhJ1psDNUAnq17wxHBvxJWJNN4TKXbyhDXxCP3cd3WHWPaElrr lBfShsbOJFH+cDt18Mv1+KDK3jNtVILJmq0NvB/pE3z7zm7GhDMxrwhOEltKbzrKjFbi qjLPT8I6m5RbjXIbXaSwjRQHqQ7vbcg3UJX0vye6YLuFFXFyML+yFiXR1OosRyVUEMdq xLBzmVMN7bPKwi/CKlLl98bfhRFG/NgGYLAn+zDdeW5KVGcjRtv3l1Ju8vHIfz2E1Y5z 8msg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=spamdiagnosticmetadata:spamdiagnosticoutput:mime-version:message-id :date:subject:cc:to:from:dkim-signature:dkim-signature :arc-authentication-results; bh=J1cj7ZXg2ZGUw9fXBt8+jsCqFzx1mcYvf7SzwHTz9nk=; b=JO8wLz5H/7RLlebKjVSjreiV6FB4RGuwqtfZoBfctizmyY0cNu4UOWxwCv1xMU3zqb b/w68YjrR4qWyzA7hBgau29cF074Klynai1r3zggD/nP8Fq0EGuCHlgOMQyhjQZUJc1J LxJeEGIwwXRRos7wEeoLneVaK1AZxJyJOlp4P/uQUra+stFHgoLpset5EKO6x33tTF5e cRl0OnWnG8S+4Ey+W39+CAjiX9CfaOuOs5xkY7ryQ+8UyRNjrfBIf190FQ+Q98hGVBqu CVHYSizes4+dLoixvM2+mIEFJVR/SEx9OJPYmQupvtWvtJf3I5ZAOrHJmrC2+dO7fYxo K0tA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=OrQ7K5Uq; dkim=pass header.i=@fb.onmicrosoft.com header.s=selector1-fb-com header.b=QtV9vROn; spf=pass (google.com: domain of prvs=177771a572=guro@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=177771a572=guro@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id k19-v6si3130155edq.27.2018.08.27.09.27.15 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Aug 2018 09:27:15 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=177771a572=guro@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=OrQ7K5Uq; dkim=pass header.i=@fb.onmicrosoft.com header.s=selector1-fb-com header.b=QtV9vROn; spf=pass (google.com: domain of prvs=177771a572=guro@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=177771a572=guro@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0001255.ppops.net [127.0.0.1]) by mx0b-00082601.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w7RGM8Fo012182; Mon, 27 Aug 2018 09:27:06 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : mime-version : content-type; s=facebook; bh=J1cj7ZXg2ZGUw9fXBt8+jsCqFzx1mcYvf7SzwHTz9nk=; b=OrQ7K5Uqoe3TBdl0Mky/jCYBxei+maPny1Rke0Tr1azCaTTThPCThTrm3zprhg+A5q2j vuP9N/7BtIjg8+wU1cZC8IpUf4Q6DZGIBlgoD6junED8qtx3K1SDmdTjljVti1nLcVRj dHTOu0FW7NN0n1gcYblAmHoeUZc1Kur62OM= Received: from maileast.thefacebook.com ([199.201.65.23]) by mx0b-00082601.pphosted.com with ESMTP id 2m4h470sg4-1 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 27 Aug 2018 09:27:06 -0700 Received: from NAM05-BY2-obe.outbound.protection.outlook.com (192.168.183.28) by o365-in.thefacebook.com (192.168.177.29) with Microsoft SMTP Server (TLS) id 14.3.361.1; Mon, 27 Aug 2018 12:27:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=J1cj7ZXg2ZGUw9fXBt8+jsCqFzx1mcYvf7SzwHTz9nk=; b=QtV9vROnwBbyPeRNB4A7ALdA69DGl1Nmr/6oFWYN8wTzO8Bfq3IJNCCF5DLaQX9WhftA9zKJdJm6i9fL90HRBRR592KpRPFV9nhjCGCheGQjZM56R1V4dyHCM3lm4d3cxUG4fngxybWLiEIqjVvK7Ti2ghB/FyKe3tMKBt4Z2Ls= Received: from tower.thefacebook.com (2620:10d:c090:200::4:4b2f) by CY1PR15MB0170.namprd15.prod.outlook.com (2a01:111:e400:58bb::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1080.15; Mon, 27 Aug 2018 16:26:59 +0000 From: Roman Gushchin To: CC: , , Shakeel Butt , Michal Hocko , Andrew Morton , Roman Gushchin , Johannes Weiner , Andy Lutomirski , Konstantin Khlebnikov , Tejun Heo Subject: [PATCH v3 1/3] mm: rework memcg kernel stack accounting Date: Mon, 27 Aug 2018 09:26:19 -0700 Message-ID: <20180827162621.30187-1-guro@fb.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-Originating-IP: [2620:10d:c090:200::4:4b2f] X-ClientProxiedBy: MWHPR04CA0045.namprd04.prod.outlook.com (2603:10b6:300:ee::31) To CY1PR15MB0170.namprd15.prod.outlook.com (2a01:111:e400:58bb::16) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8d698cbf-f456-46b6-95f4-08d60c39ea11 X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(2017052603328)(7153060)(7193020);SRVR:CY1PR15MB0170; X-Microsoft-Exchange-Diagnostics: 1;CY1PR15MB0170;3:V7guu/R7DFX/el7iYo1/Qf4pf0Cu7eKR4vmKg1JvGPQx6oxTN+Jh6uXdF85qeVhlkef0ANyMdn4zPYqic8bTqxxvI3tOv7BOPo998GkmKG7K+IZCYy1/0OjMDNi8wJj7WEod9FK1v7rJ+QjYpTCqZrT+Lc0kqEOqZubmo/uEa05Q/cQSEkcPpC+m4gPT+4qf1mw41xISXwCwjhdoxw3+HflM2MC+LVcSq0RgqjUGCPw/7xZz0IKu16zpEq/EW/Nd;25:D/n8VUYdHiWmQmmLDsk1vRkbD5Ug8bvlDuurD6EqydivXPgmZrrrtoDDTUuowD13arSzfgGHEyzJKN4cyz1S/pyNOy60J2va+cS53jGpmfPPYukAnBciMbEzvJPmD6EMOKndCCe2MOJeAjnOgIYOMaCvCFevoHC/rEYSwqFo0A4AgWqvnY9TrJ+dG8exoN0XERvljqYQkXqlXQE3ZOHQWsl338jv7NmYvjl8HXXIpA4pMSo+LxMWpxU807slmeLhrm2SthwnGvnc4fUlAha2UnSw+tNd6V7d2xLyASOMcCNKMTQtmkfkdMkRlcYbJPzoaCT5BJjxD6+i2+bHmeZzCw==;31:JuviIVRhsgJ0/mtxcPxjTVMKgy/FnF7DVJXZrMWa510i7Fri3bUigzFJVC7TRv4KAIDZ6yJgYVEj5zELpJLtZyyxntbZIqHZzaahG0pm9+fK4TjzdhuY9jtpyLnvoZefq3re/y8q4rigNP2zcGFwGotulPmjT6ocgH9otdd3FJxKNYJ69FA8DRfksPz2J5lI2y76e/Caooj8lRVYY7Jumv+0SlXQV3XO3AbmKxOruHs= X-MS-TrafficTypeDiagnostic: CY1PR15MB0170: X-Microsoft-Exchange-Diagnostics: 1;CY1PR15MB0170;20:cI4V/AgaiT4y06E1C54k5vJXuJWDmXNdDHodtUYVxhw3voa7oSxuJRIYw2hC4Ak7mYrJ7D4Y4Du84ZAU/tiug5y+NVQr9kIHPlWh+01wdCY7C95kA1FPgfI/b1dIIi3/YdCitWKK6Oby5jhj3MItE9br9UqacpYYRhJg4jahuEaSmUNMhfD6Xxj+IK885us5l4gLnNr5bqRZ2k2pI00nVxNz0CkB/kheMVy+9kOdgjpQ6T0VC6/q+BnGX8IfQp/MjaL1UtrAw+48dk8XExh2Z6sJc+UxpbjFd8BsBDHaMxJ2+ec6YXc9KqhZQ+AAmKgSAJw0BK313TzyWCtHbL5TJeS67PZR4LQWH5GeC3LVwTOxn2M3PaGFIeMTyQF0RHVBZrmkrmlXebz0e8UAscm8387Idb8oe7En6Hq4CBbsFFlFP5GL3hQD8iOswkhnve/lo8ZNyJ77AepOaGdzEFBQtXxo8rUsKAudxAn+2JU/C0pDz1K63b1oQH75j3FGsO2R;4:aao+Q3FYqBj9A6zQaDBrMMsmdkVkrwNeUyyb5w911WBAG0UrPMEWnlD0PpZRHWX83tZRMfx2n4GrKAOzEG1suZmVXR/cg6XxG5zqFniutxaAFViyQnp/5m34oYwPOBBh712GHODnAstYQhxcfOmCYf3lpboywxOV7ktwMaPnBdPm5qtdQZ/NfIv/KY8XRa9GBU8AVaXdEO3Zlt2S/sEoKJKiRsowXyOpx98PeRDZn7aB3ZpiEgqdfIczAbT4FZ9rEsB6W5ABWp5OoaYJfVCTw+AllVY8Q3jGp5X5QI1klRd+ITppGYAg1Vttqun4p7eBKQzlWkL8pDO6kc3MjqAdj7g0fCWVrNGVUa3dbFQp5kIYe6Gs+nWKbWINuTiHS+phN4HUqt1ObtiSerX1kid9ME3NTptdAyDY5U9+OB rWy64= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(85827821059158)(67672495146484)(211936372134217)(153496737603132); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(823301075)(93006095)(93001095)(3002001)(10201501046)(3231311)(11241501184)(944501410)(52105095)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123560045)(20161123558120)(20161123564045)(201708071742011)(7699016);SRVR:CY1PR15MB0170;BCL:0;PCL:0;RULEID:;SRVR:CY1PR15MB0170; X-Forefront-PRVS: 07778E4001 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(396003)(376002)(366004)(39860400002)(346002)(136003)(199004)(189003)(8936002)(16586007)(97736004)(5660300001)(81166006)(316002)(81156014)(6916009)(6666003)(54906003)(6506007)(386003)(186003)(16526019)(52396003)(52116002)(53416004)(51416003)(15650500001)(47776003)(2351001)(69596002)(1076002)(105586002)(6116002)(106356001)(478600001)(46003)(8676002)(2616005)(476003)(48376002)(50466002)(7736002)(2906002)(486006)(305945005)(14444005)(36756003)(68736007)(6512007)(25786009)(86362001)(575784001)(6486002)(53936002)(50226002)(4326008)(39060400002)(2361001)(42262002);DIR:OUT;SFP:1102;SCL:1;SRVR:CY1PR15MB0170;H:tower.thefacebook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; Received-SPF: None (protection.outlook.com: fb.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: 1;CY1PR15MB0170;23:2M07mbOYny2ynFkW1aHmcms2fjIO1WNiDN2PnrRl0Bzok7jDMQ03ZpZ8MwRHJpiGVmxD9qQ6xzpB+lPnyavyySLPlzQcHAPtY8v0Nq6gKMT1OYB4YWJc+HAtPu+APzwccpqreC64gy6hUV8dGybVhBM70pMwrjGpp/zhXy7BWYWaULix2+yysSE3dRdZ1B3gY+67MwhXK3r7cGkpy+N7aD/nAfJefalnehoMfaMhEZtB/UUKYZCI11lRWrUtPsLnEohMBA8uAdiDTkU9nhrcU8Dpi9+J4kBgcDKPnHntATZhTaSh7HTNXWH+U6OfdTqv+LeLSOcZLbf8RVm9cHHND39dmjsnYwoQJyVrHkzd7fFLdBtuXhsFpzB/JAzybwi6SwwJZq7+oquLsdHGLp6gaZZyFerJVJjJzXVpmAOG11ApCSq7zx1Pi50xw7VY/wMN1Tz75JpwsCtEiKKTc8YaP3xz/gSuIgeVxtcAv5Z+A3GGC46xwa3igekXALgNXSawx3cv7xcuHtl/5d/gGLzZeZGB9aIgFfbsk1DS4MggP79wsbsZtrae5l8G7MzEQ2YiT8bkCMPYBAkqMElHaaQ1pp+X0h/OVxFSVYIkdrUJWNtUx9bhpSca9twgL9MWOuR0h6mcozE1Db/nOPMUy40sEa7KIMRWTaAKlvV8kdRqB5pdP/VWA8gy8t+ISZWRbz76z930oHKYgsNY4T+NBdUstjQmbO18LVfQBUL19LaDNjtNxxlxFsRQtUCE7M5JrFWG/fOiMQ3sdXBHrqyq9Ex8N1gX680OmpaAps3SXQiwdyoWyCzVWwBEHuuFj0a+MeJXNk1c4+HjMSCiBj0M9cSJGQhI4U71SchrfQgr2mgZSEcew4GNY1qbfXuZfk1dpa1J9ps21zDRvqofx1TMJLoXoWAvDBXTvs9i+soz1t+0A9K3BkgnlmwdaldTFbxnrqX0jyzTSEsvIp4o/Ef1MJK/6JbHkW8gynjhC/lVaAU+0yOrKNvAcx3mvhc6lhRyswzw9Up7kR/VMXGtNv7Ugevow2lOhkreZF0Bhg6gPLnrgipTLVV4jpL5pjI/uCn8EzDebtM31kIRX4qdwtA5TuHtD60pu9BEF07cq8MMvmnwGRHhSCVx20Fz3Hmfp33wYwh3UJ3Tc4eyz48KUVmN0pOThV64VzgX5aDcXDQ9ez/A4Hqkoa5BmU0uXtQp6S/kEzm458/dfYUk8d+s9k5sin7nDP/ZYDD5Eabv0KkQ3HOX1/ogeDBfbrXwA7O2CA8gJ/DOcdL8U4g4u4uT/3DPzEeqa+ybedFCNebABCmigeVjzDw= X-Microsoft-Antispam-Message-Info: Co4iOzAvYt6g6t3n+Q73aC9jiI2kRuJgIeeSB6w1gqEhOguwKgxhO7d65R+0AoK3NE93hiFDpynnPxhV6zZsgUb3lTSL4lcKR2kAIwTW6CUyxLuMy1k0HLvQw/uI36DM8HeS/uwotrp03zBQUiySIl14pKXh9p5hOrWtkKTfeLod/JYl54GiDTFlgYAW0a0uozP9zwiz/mkfIRhb5f+fNQlJ+8/P2bsW6lpR2bsxWeN5Jqr7hZbGanfr2+1ELjk6J5tOgRpYydAycyfr+V+i8DCDsmGi8of1HMfuG5XFLQqr6pEoNQJsF06Z4aTmBSVv3GircGfHaLQxvr/fS1UITxVugl3g9MuRj3OsI4Obev0= X-Microsoft-Exchange-Diagnostics: 1;CY1PR15MB0170;6:KlW30G8EMPLB9k4qV74LprDS6/Pe0iH/FbX1lfUFeZ43dV4Na6kthxyQIFKY5mxj8tWGLz+xQ1cT3BV9v26I+7ESfs9gp/66aXJamvzi3HGOlx1/HDt6gt42Rd4Ua9e/tNEUGELjtPD/v6XWEy8wv54i7sxVC75cX6BoA3esgr49RDYG/Snu3csWBbDfRYtGcaEGmSyURTwsXFjsC2isNgZX9/XSF5IH4pvnjT22FMTtUTEhNfsJ690D36FqR2NbKOGhTG4LL3iXspkR7dInVP5eN9m0R/H/Pw7fokFUDMNQeANLgptYoejBoqDP6sES6CGGLMSBp6ywyTIQfSxLG7GZ0S4toHW0NwS7llgaKf7/UcREtn6t9lI8LD0ZtCneRXDhpXbKYVu+yo5apenZJZq7px/MGxsGi8CnuFuGP3H9FV3x7xD9RukGxZoSEvt7PI70QI+hGbIoyXd940r1AQ==;5:JFswgye3GdrmM4pEFb3rCDER51luOVZQsVQxOF1X/ZYg5Or7EY+skECkwQru6DLhvvC02Fgl+mPhd/3E3HxmrFomrYNTSrM91AO+48ojaPa90//qzz2kh1yUDjS/b+Z+aZSybGnErhVSTG16Mb6ibHN4NsvcPY4rsu/qrs5Mrdg=;7:oO737xMkoGYJe2dHfhzeiwSbHKEb4ksz77iolNcL7/94sfmu1dn/0A+CGq1LpGjINWuL5gAJQ48T8Eqj1s1MLZ2/PJE0ywcdnFLmWEirLAT/f148t/qxCuC5p5KNRF1/NGu3Zx7bv12KEPVIhB+0qc3rmUMEtrbtiIhjeY31vQ7NlSYrqe4aVRL+LS2iisSA9rLrjzsxUFiUJmwR7KTnZKP44eNBQe5u6fVDdwiNJVj/uR9u0RGlolaEs+uY0THv SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;CY1PR15MB0170;20:kumW2C1CX2K4cRwHwgExDSADjFPdhj3C+KmSqXhqqt9peV5TQDlwFXcdScQEZkyhMUekxGM9ZFGF5UO0ei/yF/xXgmuBMECxTtHUn4FhjGTgTwl8bZH6L4zqS/3yLK73iV4aB02rraIKORYdAK3sqw8pCuDHFbPWOmXjPVz3z5c= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Aug 2018 16:26:59.1850 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8d698cbf-f456-46b6-95f4-08d60c39ea11 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR15MB0170 X-OriginatorOrg: fb.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-08-27_06:,, signatures=0 X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP If CONFIG_VMAP_STACK is set, kernel stacks are allocated using __vmalloc_node_range() with __GFP_ACCOUNT. So kernel stack pages are charged against corresponding memory cgroups on allocation and uncharged on releasing them. The problem is that we do cache kernel stacks in small per-cpu caches and do reuse them for new tasks, which can belong to different memory cgroups. Each stack page still holds a reference to the original cgroup, so the cgroup can't be released until the vmap area is released. To make this happen we need more than two subsequent exits without forks in between on the current cpu, which makes it very unlikely to happen. As a result, I saw a significant number of dying cgroups (in theory, up to 2 * number_of_cpu + number_of_tasks), which can't be released even by significant memory pressure. As a cgroup structure can take a significant amount of memory (first of all, per-cpu data like memcg statistics), it leads to a noticeable waste of memory. Fixes: ac496bf48d97 ("fork: Optimize task creation by caching two thread stacks per CPU if CONFIG_VMAP_STACK=y") Signed-off-by: Roman Gushchin Reviewed-by: Shakeel Butt Acked-by: Michal Hocko Cc: Johannes Weiner Cc: Andy Lutomirski Cc: Konstantin Khlebnikov Cc: Tejun Heo Signed-off-by: Roman Gushchin Reviewed-by: Shakeel Butt Acked-by: Michal Hocko --- include/linux/memcontrol.h | 13 ++++++++- kernel/fork.c | 55 +++++++++++++++++++++++++++++++++----- 2 files changed, 61 insertions(+), 7 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 652f602167df..4399cc3f00e4 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1268,10 +1268,11 @@ struct kmem_cache *memcg_kmem_get_cache(struct kmem_cache *cachep); void memcg_kmem_put_cache(struct kmem_cache *cachep); int memcg_kmem_charge_memcg(struct page *page, gfp_t gfp, int order, struct mem_cgroup *memcg); + +#ifdef CONFIG_MEMCG_KMEM int memcg_kmem_charge(struct page *page, gfp_t gfp, int order); void memcg_kmem_uncharge(struct page *page, int order); -#ifdef CONFIG_MEMCG_KMEM extern struct static_key_false memcg_kmem_enabled_key; extern struct workqueue_struct *memcg_kmem_cache_wq; @@ -1307,6 +1308,16 @@ extern int memcg_expand_shrinker_maps(int new_id); extern void memcg_set_shrinker_bit(struct mem_cgroup *memcg, int nid, int shrinker_id); #else + +static inline int memcg_kmem_charge(struct page *page, gfp_t gfp, int order) +{ + return 0; +} + +static inline void memcg_kmem_uncharge(struct page *page, int order) +{ +} + #define for_each_memcg_cache_index(_idx) \ for (; NULL; ) diff --git a/kernel/fork.c b/kernel/fork.c index 6ad26f6ef456..c0fb8d00f3cb 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -224,9 +224,14 @@ static unsigned long *alloc_thread_stack_node(struct task_struct *tsk, int node) return s->addr; } + /* + * Allocated stacks are cached and later reused by new threads, + * so memcg accounting is performed manually on assigning/releasing + * stacks to tasks. Drop __GFP_ACCOUNT. + */ stack = __vmalloc_node_range(THREAD_SIZE, THREAD_ALIGN, VMALLOC_START, VMALLOC_END, - THREADINFO_GFP, + THREADINFO_GFP & ~__GFP_ACCOUNT, PAGE_KERNEL, 0, node, __builtin_return_address(0)); @@ -249,9 +254,19 @@ static unsigned long *alloc_thread_stack_node(struct task_struct *tsk, int node) static inline void free_thread_stack(struct task_struct *tsk) { #ifdef CONFIG_VMAP_STACK - if (task_stack_vm_area(tsk)) { + struct vm_struct *vm = task_stack_vm_area(tsk); + + if (vm) { int i; + for (i = 0; i < THREAD_SIZE / PAGE_SIZE; i++) { + mod_memcg_page_state(vm->pages[i], + MEMCG_KERNEL_STACK_KB, + -(int)(PAGE_SIZE / 1024)); + + memcg_kmem_uncharge(vm->pages[i], 0); + } + for (i = 0; i < NR_CACHED_STACKS; i++) { if (this_cpu_cmpxchg(cached_stacks[i], NULL, tsk->stack_vm_area) != NULL) @@ -352,10 +367,6 @@ static void account_kernel_stack(struct task_struct *tsk, int account) NR_KERNEL_STACK_KB, PAGE_SIZE / 1024 * account); } - - /* All stack pages belong to the same memcg. */ - mod_memcg_page_state(vm->pages[0], MEMCG_KERNEL_STACK_KB, - account * (THREAD_SIZE / 1024)); } else { /* * All stack pages are in the same zone and belong to the @@ -371,6 +382,35 @@ static void account_kernel_stack(struct task_struct *tsk, int account) } } +static int memcg_charge_kernel_stack(struct task_struct *tsk) +{ +#ifdef CONFIG_VMAP_STACK + struct vm_struct *vm = task_stack_vm_area(tsk); + int ret; + + if (vm) { + int i; + + for (i = 0; i < THREAD_SIZE / PAGE_SIZE; i++) { + /* + * If memcg_kmem_charge() fails, page->mem_cgroup + * pointer is NULL, and both memcg_kmem_uncharge() + * and mod_memcg_page_state() in free_thread_stack() + * will ignore this page. So it's safe. + */ + ret = memcg_kmem_charge(vm->pages[i], GFP_KERNEL, 0); + if (ret) + return ret; + + mod_memcg_page_state(vm->pages[i], + MEMCG_KERNEL_STACK_KB, + PAGE_SIZE / 1024); + } + } +#endif + return 0; +} + static void release_task_stack(struct task_struct *tsk) { if (WARN_ON(tsk->state != TASK_DEAD)) @@ -808,6 +848,9 @@ static struct task_struct *dup_task_struct(struct task_struct *orig, int node) if (!stack) goto free_tsk; + if (memcg_charge_kernel_stack(tsk)) + goto free_stack; + stack_vm_area = task_stack_vm_area(tsk); err = arch_dup_task_struct(tsk, orig); From patchwork Mon Aug 27 16:26:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roman Gushchin X-Patchwork-Id: 10577419 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 11CF513B8 for ; Mon, 27 Aug 2018 16:27:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EFE1D29B3E for ; Mon, 27 Aug 2018 16:27:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ED1F529DBE; Mon, 27 Aug 2018 16:27:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 663B329DAB for ; Mon, 27 Aug 2018 16:27:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 208BD6B4167; Mon, 27 Aug 2018 12:27:18 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1B0286B4168; Mon, 27 Aug 2018 12:27:18 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 07A366B4169; Mon, 27 Aug 2018 12:27:18 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-lf1-f70.google.com (mail-lf1-f70.google.com [209.85.167.70]) by kanga.kvack.org (Postfix) with ESMTP id 7F3326B4167 for ; Mon, 27 Aug 2018 12:27:17 -0400 (EDT) Received: by mail-lf1-f70.google.com with SMTP id o4-v6so3665734lfg.11 for ; Mon, 27 Aug 2018 09:27:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:dkim-signature:from:to:cc:subject :date:message-id:in-reply-to:references:mime-version :spamdiagnosticoutput:spamdiagnosticmetadata; bh=2itD7Bv4bnoeSEaWjV2FXU35V3cT3moJ5yiJs1BMCuA=; b=Hv8strIdnWgieY2azM+BsIMBTyb0xqJBbDyfL8QjHjEDmC5A1vFaWKzFfi3dDcqXl5 8h5KO1Z06VgVp88BFtcPfxYnALiwLxKoeilKQg5VgKUYxMQwKAFBLkLXeSj4nAphBe3I lUKXpYw9yEYTidW74X/8i+PrH6kXAMJSUs4Rdthve8q1yRGqP4jkPCzqpBJ/jwnl59XG 6GoL1xj7CjFiFtU3KjumxRzFKlji8lAleJmPORQ5HjsQlkAIj4djTQdD2OgyAVqCLaOI 46U/W7OFRQR49Y7VkZleMVHCNjLHRig8+7NLMkUOBFQ6HfFiaQldjn7OzV+PEK9wdmK7 q9QQ== X-Gm-Message-State: APzg51BkqvF98xZYffAlD6+NEwz6sDbbFvUVr6WWlT/+Wfwml9PjVKev 55z/iZybti98ybZwDYTcJJV7xeIp41BFA8RnWxpmKrTlC0eitQfDVqp0onh2SdglrvZw+fr1/y0 N1Dbeyyx5zKUEPHJeKQ+a4dpZGJFY+9A0PVLrkaZG+4RZdfGv7yKEOhRK3VGLDT+2bQ== X-Received: by 2002:a2e:9cd0:: with SMTP id g16-v6mr9519572ljj.110.1535387236955; Mon, 27 Aug 2018 09:27:16 -0700 (PDT) X-Google-Smtp-Source: ANB0Vdb79hVxOgsmm9tXO5nWc5kXHR7CzaCRwR5m4XT75UrfseYq7STEIYBrzDhLHWG5sBMYD55c X-Received: by 2002:a2e:9cd0:: with SMTP id g16-v6mr9519537ljj.110.1535387235938; Mon, 27 Aug 2018 09:27:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535387235; cv=none; d=google.com; s=arc-20160816; b=aIgaP93qDX51cO/5AdM0q82sABmImy+RHnNnhlYCnbNudVNXDIuFKgzIZPj+4SN8EI B67etPFrnMDRzYMOQOuImWVY4lVn1WV+c5Dc1GDj/LCDJZsdDNpiQu2D5+czncNVs43v 9fP4Ni1dfsW35KL+MH7oiXP1F8zIGWVJXfx4M1s0VucwsgRmQ4s/a2VVUHZQxdSuLaBz rTX8tkA5ZwtrIfKgZLurpy1I9P5NH1GRT1Bf80hs+W5+dK5GTd1E28uvqBVwrDxwQmhP 0YksPm0MasjkWqSzswlTf0zu+EhxmNjRnq4mCInMBLOg+b3VRO06Yof+Ot1Dza8wJtR8 eFkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=spamdiagnosticmetadata:spamdiagnosticoutput:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-signature:arc-authentication-results; bh=2itD7Bv4bnoeSEaWjV2FXU35V3cT3moJ5yiJs1BMCuA=; b=WncjND0tTs8kQBhZ/8ze26LqhbIRbFpzbw2PvZMM1gfbuVpBf03yZEMbUBkSd3Rzop xPh4r4q8114pkGzu1FfIx+5Y+NEhHRPKc6B240QywQNn0MuFFRDxi4HwBcZ+yU5cOBaV uLkvzvKORxuc0voSBuYJ3HiL+LfOSaTPIlVV7raaoBxer0YT1LtXpR6rVIRQ3KCo8K0z 91cesM/OHPbPwjhcydfhK7UYPp1mcVsUaGwmVVzvXTuZJVzfyj5dn7txZ3OwBWHnacia 0wVfY5TCggo/NLKOoQwQkHDMAVO9h6Ltg02e8QjcK5T3lN0CvSABOuweSAh+IiXis9Qa 5eXw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=Uyp+3svT; dkim=pass header.i=@fb.onmicrosoft.com header.s=selector1-fb-com header.b="Faz/O0UV"; spf=pass (google.com: domain of prvs=177771a572=guro@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=177771a572=guro@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com. [67.231.153.30]) by mx.google.com with ESMTPS id v66-v6si6779745lfd.107.2018.08.27.09.27.15 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Aug 2018 09:27:15 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=177771a572=guro@fb.com designates 67.231.153.30 as permitted sender) client-ip=67.231.153.30; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=Uyp+3svT; dkim=pass header.i=@fb.onmicrosoft.com header.s=selector1-fb-com header.b="Faz/O0UV"; spf=pass (google.com: domain of prvs=177771a572=guro@fb.com designates 67.231.153.30 as permitted sender) smtp.mailfrom="prvs=177771a572=guro@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0001255.ppops.net [127.0.0.1]) by mx0b-00082601.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w7RGM8Fp012182; Mon, 27 Aug 2018 09:27:07 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=2itD7Bv4bnoeSEaWjV2FXU35V3cT3moJ5yiJs1BMCuA=; b=Uyp+3svTszTz280Vynv6RD0DoBbOSMtK+GGM4l7VMdPOeqdHj82x2L/uwO70AIXwfq3y I2isMcGLcG6yVaR8nSSnmUIJQQ9S/cWx/ASPkyRtaZX+WoPnOiocSRGgBvJHttVMFSlm l+KYCjdY/OJmumGAwfWOBW+YrGG8XvqeP9k= Received: from maileast.thefacebook.com ([199.201.65.23]) by mx0b-00082601.pphosted.com with ESMTP id 2m4h470sg4-2 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 27 Aug 2018 09:27:07 -0700 Received: from NAM05-BY2-obe.outbound.protection.outlook.com (192.168.183.28) by o365-in.thefacebook.com (192.168.177.29) with Microsoft SMTP Server (TLS) id 14.3.361.1; Mon, 27 Aug 2018 12:27:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2itD7Bv4bnoeSEaWjV2FXU35V3cT3moJ5yiJs1BMCuA=; b=Faz/O0UVGADygy6w5ogQQuxk65lk4u1DrSQ5ZAiLfj/71HqyItbEYWIfrzvtytftiX8VOBpVLRJpEjywDSy3iI9efP15sgY/EAoWIdZ9Beo+yE3RfYuFGfeA6NCI7HthUwGm4XOfkWwepSkI8DgeTf2CDLxo9ROkiyZTI58aW28= Received: from tower.thefacebook.com (2620:10d:c090:200::4:4b2f) by CY1PR15MB0170.namprd15.prod.outlook.com (2a01:111:e400:58bb::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1080.15; Mon, 27 Aug 2018 16:27:00 +0000 From: Roman Gushchin To: CC: , , Shakeel Butt , Michal Hocko , Andrew Morton , Roman Gushchin , Johannes Weiner , Konstantin Khlebnikov , Tejun Heo Subject: [PATCH v3 2/3] mm: drain memcg stocks on css offlining Date: Mon, 27 Aug 2018 09:26:20 -0700 Message-ID: <20180827162621.30187-2-guro@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180827162621.30187-1-guro@fb.com> References: <20180827162621.30187-1-guro@fb.com> MIME-Version: 1.0 X-Originating-IP: [2620:10d:c090:200::4:4b2f] X-ClientProxiedBy: MWHPR04CA0045.namprd04.prod.outlook.com (2603:10b6:300:ee::31) To CY1PR15MB0170.namprd15.prod.outlook.com (2a01:111:e400:58bb::16) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f61cde31-c3b9-4af1-8f84-08d60c39eaac X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(2017052603328)(7153060)(7193020);SRVR:CY1PR15MB0170; X-Microsoft-Exchange-Diagnostics: 1;CY1PR15MB0170;3:ZNcK2zqpMZ1RSztEjgyv/y2XHVz/+vwzGoNTYCCCq3aS5ANfXLJ9yCXOxbTrRjrikJ4BKg+mzzyMcqmMYfUUgQkgU+wxo6r89S5U4pTdROId1W3uMabSQfvCJ94dP2IJzinfAFivqnS3qRvUKrKxs2jFjXH0AKkdFPY6CpXPKDEThlxFesNusmhxnHD4KnnjTlEdyXcHRu5K5YsL0t1GLcyEqxcuxYTRKTU+zuSuZm66vH01oOsJaz/cI4AZe53Q;25:j47U/Vil8uCi9pNvNYIkaTb12Id8Vh/zoClIucyAzTa4gaONXzrRj9U3LFB/weXdsFsP40O2cVMGrkQvSIDYEXzV1+1cjfEv7PWylSFqoHIKd8v+dByBVyUveq0qjLjyQPVtdhI5ygtFBwXK0lDE54Cck1WPKDyhuL5yu8pfso4+o6zQfLVsHgE+mV5wtjTiKHATs6GpF16X09Z7yKhlD3vL+33errtplysGnpRaQ8rhnyaTgsET1PkZhTIiOKNfEKxFslJ8Y3FmMMw1s+aTVThMwSXA2F7OBCYWBgoSK5dKJmoxMWjErTWywlPl597kzS1YwFfbMpjUwnpi2hBUHw==;31:wLK/hZHrdLFJU1Nm3g77iRev2H55sNT/TTmgz+s/G9pOY9ln6EKEqJ5WgD/QObyjPdW7WhGai+Gcn1i1wI2rRmgDhFcaa7S1ZOPk9bTXDdIl8m7LQoF/JjE5r4LKqsTr5g0720MI4+p+LhB67zPLEx2FUjMWJD6VEOGhf2Yx9gnBHUiHnlvEgw6LChPHUJ1WxuhIYnYE9uEvDtRC3JPnTWIQLhx6apoeItuWQ64m6aQ= X-MS-TrafficTypeDiagnostic: CY1PR15MB0170: X-Microsoft-Exchange-Diagnostics: 1;CY1PR15MB0170;20:5h6QnUwp54RJAdwr0FxZUy2c1AMJVqJxR9L7EAbdRNKsOWaiZHbQLxo1D54IIs6xy311KgYOMdhuIHz5mim3v4ocXmpyA/O0P6C4cCqaaTOMi8Fqgsv6GkWBiMSMHsHJcbUhxq4mahh8paZe+zYB5nO2UQwxBEmvbpSH/NxY5G5JtaaXeZiX5c8nDiIVuAL5wA4ZMxJjiRNEjJoWEq2FpvLrNcwLhUrDcpM0Zlz1b5vjfWozfQ0tmQZ50sNVCPXYfmBcGTx/xQUzhBQr6+GkGIs/HwdRpuZH4stsvKFvFRvOBd7gYl9p2q6zeIKdUQ5g+YTlqUff74EOJdkiQpBEslYqaHj6BUg6U7oVtKz3vJV5cUEGfut4oeyA5ee0Fy9IMP9gmk5VPpVWynyb+HhzGkGO+liTFW8et+mwXFGuPCkzMHqTb1fQSGsQuPth1ASkNUQwpzVnITDPtZXRtlAbUx4c770kPYN4cFOhMd0NgJnHQxWQbhPa9CdtdaYlD9/k;4:3tddJFV0d0W6cu13e3GlFcmXVO4GCkkLno55aJvyCTwjGRS74JyYvYM4Ee8M/jpJY0GZzvGKR6+uIgDf8u+Nxk6TPtOGCkRjl0aE1YMJAmuojmr2oOjLz9nsLMzRC0WIe5btwW9sRfLOvuyowRtySkGqRf01lEM2m1PCd7Ax5kmAuHnyOROpcDUWtEEh8l+lJbVQSCyZygTOZjy+aa+DdM1x2SBfgsk99TJa+x9bkXjWEcxsPGc0Bektn3q3TlJLbYCstPKWfBlauiYff7I79HLnUuyvvH7VvBakMY+CJPZuTtZDmy6ZV5XHRLCssH2UcM/FnS/EhiPSDn1Gh8No0RkjCUcZjTBotE/DNnOoviY7QyG5UkNUqQDMt8SakHmIqajqLdWU++sjYNBSlnkJsvrvZINXtigal21Kqs PLQqk= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(85827821059158)(67672495146484)(211936372134217)(153496737603132); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(823301075)(93006095)(93001095)(3002001)(10201501046)(3231311)(11241501184)(944501410)(52105095)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123560045)(20161123558120)(20161123564045)(201708071742011)(7699016);SRVR:CY1PR15MB0170;BCL:0;PCL:0;RULEID:;SRVR:CY1PR15MB0170; X-Forefront-PRVS: 07778E4001 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(396003)(376002)(366004)(39860400002)(346002)(136003)(199004)(189003)(8936002)(16586007)(97736004)(5660300001)(81166006)(316002)(81156014)(76176011)(6916009)(6666003)(54906003)(6506007)(386003)(186003)(16526019)(52396003)(52116002)(53416004)(51416003)(47776003)(2351001)(69596002)(1076002)(105586002)(6116002)(106356001)(478600001)(46003)(8676002)(11346002)(2616005)(476003)(48376002)(50466002)(7736002)(2906002)(486006)(446003)(305945005)(36756003)(68736007)(6512007)(25786009)(86362001)(6486002)(53936002)(50226002)(4326008)(39060400002)(2361001)(42262002);DIR:OUT;SFP:1102;SCL:1;SRVR:CY1PR15MB0170;H:tower.thefacebook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; Received-SPF: None (protection.outlook.com: fb.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: 1;CY1PR15MB0170;23:Xo4G+J5cXa0cRKAMLoNJhpMLxX3MBkk+fVZGbOL6Z7jdpARAygUHIrMQ4bk4sdpADs9H3zN8QLEkHsbBqBK5JWSMiN6SPuc8/1noh1+HUuaRavYc1DVBaz30KGf0uW5G6IplLzQT+aUELxL+E3pXoWTVE00ToG5OtaKmIPqHAU6fC1LEUcSaMXObd6aHgjBB4OMVTqb0yssvVXJJCce+X9364lGAgr6v6DziiF5tiUSQy8Sr0439niI3kQ2zEVRg7kHkR4/VOCNUItx4uI617mNPe9vE8Hjjil83hbecgSNxQg913NlLxbYlaB62jq7BjUkr7FeJAgzbRv79oBftSK60ci4Uxq+bC2gyTEjbn7fc5P5PVveW2XwgrUkaAnAUcwKROaqDKxwB7dHaOfctT7Y+Bk59agoI7CsBw+AXtUSKYzetRJ/twKYZdXcdOlZRMWayP/7E4owugDBrw0k2rT6XwH0VzK3AY56R5sIEUQO6CxCcu+qE7w6r+qwpxP+vAbKw5UZI4qiyiT9iS6Q4Y8OZWDdEAVLLRSga2ebmeomKBjWC44c4HXz2kRmPg/QkqcuPYdlLvY4lL+cCbElzNHjAIKOxFRM4797/iZhoOys+zLtu6PUd0KGF53X/YHqjfFuhqDUbtrgCSEa+XOqO32gJpWmZ7xMykVXk4QzIfEHnqyq+CnmyJbi8/fZLz3GB4JvDk2EKsdIQs+km2PSnCya9aFQNvp7FZABN0w4u3mouk1YvIbPQqTiZFRRmOPyuiA7qJDm3A2MqE2nzYwJ1f8v7USmliKNBhqfGk+A6yMFVZjrAfD2UYSu+GfnhIdHurvdM3fiCryOrKSsRzeRhjw/BPUubslAH0TXceG9gjJ5iY/VU2o9OJwSAWKIAvkiglxq5kJR5XEjOZYJAiDDUxrzyRH+Beau4cYowBS2OKiOEkIkD4Iowjptq+6CsWVgC8IR8nbpJg1W3w4tbMyB1vFYS1eHw/KgYI2F4KXl2Yaq0NG7mGajqc6tYcH754tKNrgjQS2I6T1aNzyzQ0T+JmNJYKrkQitB8BZG+MtLVL86PaeFzYaZWtKLTPy4rpvutaGDu4AU4pGovdDdlJ2noySjFXwgEDJkuqV9LHccBHUjSRghUTFfPPedHQjVtsX782PnVQ9OdKdLIms9G5zSpYg/fb2zLGx2wt+LBrLoImla5Aw2vnQ18VGEqZmtvisNsH05wnq36XyzgWde3DSpG3dYvkXRURo2rIrnvXg7ko2Y83Miyeb5hiHPj0ZMsvHAbwyouTSG1ioUysMy0QflwyA== X-Microsoft-Antispam-Message-Info: jBYdhUYOBk13AGCQ1RLAQ6wO4lBRyp7hjgjrJeDrjespGBSiOpU/y0MaWSrbdZgZjBWBIXcUGnfBOsfDITaJmg/Tg3/oYAWIvEq5OoagPTglpeJy6xcDRAx0g1DYFmTz6Vp2Q1et75lDuuoYNw30fXJMS2bP44Jx/zdfxNyWMnGlZvFbI1oS1LLfhD2bAhBVQz1ShboouC3bLixzB7HFIB0ToR4i0H2LCqVNf+SGucdtIRb87S4yEApUP0cbC38PsEAqj+eUzVWjToP3wXrSgpsRkA33B7CkU3ckym6LSkDEyPa372SQwLS1FcWcTQi74BoSP/GugnfxF5gcNqGlsJe/fi8MWLerA/dpFLLjZ9k= X-Microsoft-Exchange-Diagnostics: 1;CY1PR15MB0170;6:KrtaSSMOnYoVTJD/xZBORC+p0QbEyLe7mvDWJayGy0o/1pq5VABImm/uif+Ttabn/6hjl1jTA8RIo7b3O5q3CWRgwKwXwR8JIFbEScYU5aCAwqESqd1uAhiaiQlg0TKXVt28Tnnel4n+1XI18esRdVNdL8SWLDq21Ey0sLid2NByzHBSHgLV4iN1nHxYA6mQ2/gNiI7u0TtHoLJe+6aXXoG2u/sdcHs8n7WukI3UwuNT+RDfh4qftQXDLcs/b4aosoJvJDIt3Ft0hibZ2+bRNEoPUBmdJqZFOxArJf7BMA7huYuPUSVr0Xx4h7F7XE7SI0Y/8XBW/aQrY1uRRLSNc4ayjRI27hFusmAFbUDLtmJy16fWDojwmBuQ/UsuwTNw1101MHTSdfNjhVwdbEgJevr5fc1hfvaMFtbKnSF1j3/Eqb7H2uFRcaOIpK1fcugXSd2GR7J3+bjlAtC72Ut+eg==;5:S2BFp4QUkRT09qIabLT/fGKtoiE+QeUiFPfhAImQWtUsqRcX96eFk79/8F1zDbN5Ulhau2+vNd8NyVsychFgSx2sBZJ8IXe80F5A8a3XWcCsgUYikR2XgOi2jTlfPVmuO4Ep4vPm00xWyy+Bw3iyMPIqJSNw793Em8L1fhl5ZNE=;7:22t3RXWEGeqNAavV+Gk+R5QOP7/crUAMhnqKWKhNMxfs+8zRl1Vk4nic2WCrTBDXGPIqSzIpCkkO10lSu+x3lmgKu/CMn5CaOpidTeVFsfxAcigvN5CpaUj6rP52vwAp3js/z2gDgZYFEB2SA+THqF3kKfRk9RHiX7VGyDjI8v0lD5dkfInKNGV+q1UnmP6Jb1ffxatcRfgzcorGMnot5MuHWHr45mY79JhsApSXnFQj4QL2km1CDH+MuwT4KADX SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;CY1PR15MB0170;20:VtR3We2WYz1ue6GrTzaga8b8OdwVJ9DczYblW/VxWizg+s/h44xKFtO4s2LjInO+JBYe7yvPOswARFi1k6KXb7p0Dn13zqPikTUq/v+9i9Fw2/cHEjUiNAiXhOGagxVCALDYIiQ+p4HmLm/zg1EC7z4sdMC2sE1LjT/+jyNnCEA= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Aug 2018 16:27:00.3100 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f61cde31-c3b9-4af1-8f84-08d60c39eaac X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR15MB0170 X-OriginatorOrg: fb.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-08-27_06:,, signatures=0 X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Memcg charge is batched using per-cpu stocks, so an offline memcg can be pinned by a cached charge up to a moment, when a process belonging to some other cgroup will charge some memory on the same cpu. In other words, cached charges can prevent a memory cgroup from being reclaimed for some time, without any clear need. Let's optimize it by explicit draining of all stocks on css offlining. As draining is performed asynchronously, and is skipped if any parallel draining is happening, it's cheap. Signed-off-by: Roman Gushchin Reviewed-by: Shakeel Butt Acked-by: Michal Hocko Cc: Johannes Weiner Cc: Konstantin Khlebnikov Cc: Tejun Heo --- mm/memcontrol.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 29d9d1a69b36..17ce6f2e6caf 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -4573,6 +4573,8 @@ static void mem_cgroup_css_offline(struct cgroup_subsys_state *css) memcg_offline_kmem(memcg); wb_memcg_offline(memcg); + drain_all_stock(memcg); + mem_cgroup_id_put(memcg); } From patchwork Mon Aug 27 16:26:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roman Gushchin X-Patchwork-Id: 10577421 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6A5F014E1 for ; Mon, 27 Aug 2018 16:27:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5620929418 for ; Mon, 27 Aug 2018 16:27:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 53A6629AC6; Mon, 27 Aug 2018 16:27:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9415229A62 for ; Mon, 27 Aug 2018 16:27:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9CAC76B4169; Mon, 27 Aug 2018 12:27:37 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 97A166B416B; Mon, 27 Aug 2018 12:27:37 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81B196B416C; Mon, 27 Aug 2018 12:27:37 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt0-f198.google.com (mail-qt0-f198.google.com [209.85.216.198]) by kanga.kvack.org (Postfix) with ESMTP id 5A4E86B4169 for ; Mon, 27 Aug 2018 12:27:37 -0400 (EDT) Received: by mail-qt0-f198.google.com with SMTP id l7-v6so16032753qte.2 for ; Mon, 27 Aug 2018 09:27:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:dkim-signature:from:to:cc:subject :date:message-id:in-reply-to:references:mime-version :spamdiagnosticoutput:spamdiagnosticmetadata; bh=5+LthTYFyo7iLSLe+0yUlHxKcSKZ/m1exZcyclpsPEc=; b=tQoKUtLUiqBhR9A5BGN5rORYD/XUh9egb8GKdt/Mr+InDo/EcT9ljYYp3cLNErb1Ic m73uj1sJ2PuAdfWmNbzC4+5FB2nCunDbwYjdJSmBLc18c9gzQiW6vu0UuR9CoY9rlTb3 EP3eUooPSB++dtjvCXX0c0B74V8LEAa7nEjFdiPuUx3iUwQbsMlqujqwk7tx1L7oygDd gFUsKFYhFvZz1aNopl8QEUFZ9myZQnd2wvOTz1IHG/dUtJ2DxgE7sD3hJS21w2/htcgb 6Jnzh+1St3N87Yh/vh6hB0Y/zdy+6YYEu9WjWdQNgXHk0hRIgssfwqvhbwLqgNRLbn96 QHZQ== X-Gm-Message-State: APzg51BOxSiJzVxp5O6fZbIyuhC+VJVnJyrkhSqW05qAIGqO8Ywc3lc5 J1T16GpRXj1NvQGbpBvcxMrxK4OmCqdbArcXSvXaUZersw9iOmEJvEeiOKk45M8uvLjCaSsvazH pGCVWxgt2yTRwJLrtK1ksjZFwjMuf4Y5zwmvJhJcmXSW9WUAy2QapSmPP44oLzzNgTw== X-Received: by 2002:a37:434a:: with SMTP id q71-v6mr13955228qka.277.1535387257076; Mon, 27 Aug 2018 09:27:37 -0700 (PDT) X-Google-Smtp-Source: ANB0VdZb/LzQ9kkozbfq8NkNJahR3aucfu4kUz4cXZgSWIVP7QVL5/+1B53A2oJSJcWNE1EpWzly X-Received: by 2002:a37:434a:: with SMTP id q71-v6mr13955191qka.277.1535387256456; Mon, 27 Aug 2018 09:27:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535387256; cv=none; d=google.com; s=arc-20160816; b=XjJzcmllMdFQsY9o7fElab5LSM+ieqTxCKUqn/NXR7AoMe/e5CxT32UimJYH2QG+Qb 8QMgx32HFJ9T0ByIbZSDAstfyrYUCBmweJrnCdC/0PJesC1GMON/okynNADtVm2ZLVv9 za/XUG8XRCK3rkX57BZ2ykN8WXP4KwC3piyToq1mBMxq8vOzs8gyW/E07LETjWbbs4Jh OpV7B1E2EJlu21qLDu7utK2+my9shnYtaKSQPYgdKvG3QiVN2CnwGysHKNCsUTmZQs4X kzLRG7c2Br/VikBNNpXzzsRwLnRj4LEBvw7t9omE+e5ynn6aZ+gZp52boFUFVePPwzS3 Jvow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=spamdiagnosticmetadata:spamdiagnosticoutput:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:dkim-signature :dkim-signature:arc-authentication-results; bh=5+LthTYFyo7iLSLe+0yUlHxKcSKZ/m1exZcyclpsPEc=; b=m/bc4R3OjH06dX1Mb6WxPokKaZDJe/h7pYrJuLbvCg4xpIWIgLZFfkOgPZ4Jrb3lXW 1VKmGo/c51Yn3Py+T6Tvw1637R02WDA6UJ3iZv6dgYVXMHc767efT94VMRJVLx2/wpp4 PfGcBqTSOkfbOQeDp1qjoEM0oJwWWTzsALsdBF473rggMIf3s1q7fPny++emdKcFoW+/ 0rN9RylEKtyXo6JviABHLIx6ED7iJwcy+7CoVGtG+v/YFoOvOk/ymoUtB8pSg8QeK8hR 5gJtNHy6vqVpri0hycqb9UJd12PITkQNR7Pbw+ymPtXuTvLmg1rcWmBmZATSDaB6ib4o nV/w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=CJIsTT+S; dkim=pass header.i=@fb.onmicrosoft.com header.s=selector1-fb-com header.b=KlfQ4Rap; spf=pass (google.com: domain of prvs=177771a572=guro@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=177771a572=guro@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com. [67.231.145.42]) by mx.google.com with ESMTPS id s17-v6si2943309qke.302.2018.08.27.09.27.36 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Aug 2018 09:27:36 -0700 (PDT) Received-SPF: pass (google.com: domain of prvs=177771a572=guro@fb.com designates 67.231.145.42 as permitted sender) client-ip=67.231.145.42; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=CJIsTT+S; dkim=pass header.i=@fb.onmicrosoft.com header.s=selector1-fb-com header.b=KlfQ4Rap; spf=pass (google.com: domain of prvs=177771a572=guro@fb.com designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=177771a572=guro@fb.com"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w7RGHr8h029248; Mon, 27 Aug 2018 09:27:08 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-type; s=facebook; bh=5+LthTYFyo7iLSLe+0yUlHxKcSKZ/m1exZcyclpsPEc=; b=CJIsTT+SUoZXAPbNVAStk8hdoDVseK3hE1eWzZDTJeYCcXOdayNMF5asy3dmr6JqxQsv khMMuKM93pfND7g06t4UwSyI2RA8bWowcNkk00TdTiRwHtZHvDdVxJlHLFwAp0F3T6zN Yar0aNgVIgnjBj3E7v13sb/Y0eSsBt/ui18= Received: from maileast.thefacebook.com ([199.201.65.23]) by mx0a-00082601.pphosted.com with ESMTP id 2m4m8nr487-1 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 27 Aug 2018 09:27:08 -0700 Received: from NAM05-BY2-obe.outbound.protection.outlook.com (192.168.183.28) by o365-in.thefacebook.com (192.168.177.29) with Microsoft SMTP Server (TLS) id 14.3.361.1; Mon, 27 Aug 2018 12:27:05 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5+LthTYFyo7iLSLe+0yUlHxKcSKZ/m1exZcyclpsPEc=; b=KlfQ4RapWZe8e8OGtlkLY1S8yRcCSWibgWMIQ2zoF6iAShjEq/vDIAr6biO08d077Un0OXsBTmp/jjpfNFr2GeMyiFYeCz0MZ/e+9q1GnGzE++5QDUhn9kzSP0z3DrCiBwsOL9+Guz2ycvfKrki2PKmPpiCLLheMN2if07iVSeQ= Received: from tower.thefacebook.com (2620:10d:c090:200::4:4b2f) by CY1PR15MB0170.namprd15.prod.outlook.com (2a01:111:e400:58bb::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1080.15; Mon, 27 Aug 2018 16:27:01 +0000 From: Roman Gushchin To: CC: , , Shakeel Butt , Michal Hocko , Andrew Morton , Roman Gushchin , Johannes Weiner , Tejun Heo , Rik van Riel , Konstantin Khlebnikov , Matthew Wilcox Subject: [PATCH v3 3/3] mm: don't miss the last page because of round-off error Date: Mon, 27 Aug 2018 09:26:21 -0700 Message-ID: <20180827162621.30187-3-guro@fb.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180827162621.30187-1-guro@fb.com> References: <20180827162621.30187-1-guro@fb.com> MIME-Version: 1.0 X-Originating-IP: [2620:10d:c090:200::4:4b2f] X-ClientProxiedBy: MWHPR04CA0045.namprd04.prod.outlook.com (2603:10b6:300:ee::31) To CY1PR15MB0170.namprd15.prod.outlook.com (2a01:111:e400:58bb::16) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6c2a5f6b-c76e-4307-1280-08d60c39eb51 X-Microsoft-Antispam: BCL:0;PCL:0;RULEID:(7020095)(4652040)(8989137)(4534165)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(2017052603328)(7153060)(7193020);SRVR:CY1PR15MB0170; X-Microsoft-Exchange-Diagnostics: 1;CY1PR15MB0170;3:liS0mtix+WurjRHFZec5GVp1qGTaq+gEz+xOYAlPli1098JQ8gS7lbQx8VIOjrcg0uWdBLds+s6YuuhCNiAaOrJUABjR5pWvaG2XZkkoE1kgnvmg6o8imS5nPK7X3zX1f+K65myCL4nYlBy9VEm8y7ppHdeJaPSmicqMBvVfbnFj2vOwXdXGNsI2SmwpiPkaJWfhMuvPd3S95uSdUpywlHl+kyxat1rq6ireXNNN0JjtVTSOr1UpuYzKNAyKs64z;25:q2PCPtHkrxMjTJ3LwGp+2c6EXlv9ukV5ruNUt7FtDPv9e4wr+rmhycvEONS/mjObN5+rs+YKhrmlAdv7hQxQbwNPZEnOj0UjGi+WHuNwvwAVE7Qq3sVJHUf4Up9TZTuy3Re7j0WeTI/8i6SIaalLp+5lXsWzUk8outsLoou5Qhrlh9lkqeprh53H9sz7vloo+NlDWuiJLxuBV4CImVPCbUVLyeYRoIOKsAdK7Y5QEhq6Tox0TCuPPplBUarA2Tyf8yDBQTV5DFx92NzujI1zx6afPnOPu9vr+TWh69ri9bHUqVzOvvWS5O1yVhAJQShRMPPMb5RSXMCzb2WkT/P2Ag==;31:+m7RyNPEpiW7mOasj/T/lNoGth8Cdf9SYz9SU+Ads8CEqeopbEmAyDyglVJZEw/NFW9tyUWyf7nsQI/yRWekc+iTrRkw07DVszXla6sDY+FAFOwgSTkwfN8hMErCdC98GootrKm6FJBac9VGoo+x5MMP5Q7O1KhBxYQ0eqnjWoqHOYswCtkDWdJL96BsVrTo1GlXbBxW02kYjkUCJtzOEPVazbtNJk6vqxokZ9qz1dU= X-MS-TrafficTypeDiagnostic: CY1PR15MB0170: X-Microsoft-Exchange-Diagnostics: 1;CY1PR15MB0170;20:j55lwM0oqQ9cXGYKIkTQnPrAkOdyGeBod9701TR9RyhMDJxtK2avuudNNw+lJNqMvSHAMMp+Gt12AM31tmZvxpf4q9YkhtdEmWT1wD0IL1MIpANQ2Vmx1OSvWfAMl7qttr2/RD+qPsh5/hKGRmnzAupBJTWOhjcA2nL6/x/EpIzHNd6la6BYyRe7umXWF3Xyxi23Y7uSx6zTkSnNXYNFtN21iMe7ITuVqYaUoCftUB5f8kd1Caj1tQKb2upFD54klIdHfiwy7u33/fdaTQuNJQSB8ky2fRuDn08Jte5hiqyVJvI86Ghg/5T57OdmYkU/8YkzPdoiDitn4btwaQW4YjTnBi9Prn5qYXOC+AqjO3JJ39gS+ttk/Jr0Z8+Wv5ekNlgxnAohCVN/rp11Kzgg3Uy5DOCH7bYWlA2s6dgLZnxi7gHVHlL+6ydatHnjr9N6R/7DQh5C6VxYWxKuTMpcIi574CAtgzamZvtYGPKArEr46MCwMohfAbgKOhMFQ0j6;4:H2vvC0vZJDdQor8u8CmPbKxgJKxnRtY9/bYbZzjXSPkEC7B08rBrDguM79fbBW9CsQLlLJh9A2nr45t6ewjoPwAl2MlpX2wKt78+sGQKQReRb61I2gykO2cOo47o/MPK6idtaO3HOAbUf1mJqa4uV52R2IwJcjlnI+JKOlXPEPe/MtxZ2TLEIhIduUs0Ha/G9/PSd2if+LgLdthGSegjNQUVsF1Ez46tOBM8PgoP2ro3QSGvllEbybcuCQOHFbbWajggJTaPMzhVX2RQJK1uAgXJwl5TQQ6q5B8T6i8o0GFep7mdc0K5Qj70Tjd/cJm1QfWglgtOp3dAkO7unJwUnM6gkxXP4WtQ5TKAu2EwV/g= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(85827821059158)(67672495146484); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(823301075)(93006095)(93001095)(3002001)(10201501046)(3231311)(11241501184)(944501410)(52105095)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123560045)(20161123558120)(20161123564045)(201708071742011)(7699016);SRVR:CY1PR15MB0170;BCL:0;PCL:0;RULEID:;SRVR:CY1PR15MB0170; X-Forefront-PRVS: 07778E4001 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(396003)(376002)(366004)(39860400002)(346002)(136003)(199004)(189003)(8936002)(16586007)(97736004)(5660300001)(81166006)(316002)(81156014)(76176011)(6916009)(6666003)(54906003)(6506007)(386003)(186003)(16526019)(52396003)(52116002)(53416004)(51416003)(47776003)(2351001)(69596002)(1076002)(105586002)(6116002)(106356001)(478600001)(46003)(8676002)(11346002)(2616005)(476003)(48376002)(50466002)(7736002)(2906002)(486006)(446003)(305945005)(36756003)(68736007)(7416002)(6512007)(25786009)(86362001)(6486002)(53936002)(50226002)(4326008)(39060400002)(2361001)(32033001)(42262002);DIR:OUT;SFP:1102;SCL:1;SRVR:CY1PR15MB0170;H:tower.thefacebook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; Received-SPF: None (protection.outlook.com: fb.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: 1;CY1PR15MB0170;23:bKWeAg8q1/aa5Hju/73fSn45mOVLGvfcwdEc8VNWuAgNQo30ioPUFBV4Ki8xy+5Xz/0Dq6CNWz8oC4PqGrQtFib/rmOAlUuzZ9FqGPvnC5qIMQ8+gcLQgOH9hMTZoO8ceGBFCVtVTRgOlOMDjDZLXN3DQ+d6Ll0s1HZ98CMxW8p2CW3YiqAO/kSlxmxwcN6i0NGFLtJFwwdP/qI6QTXginAUnbde/le5NxtPX1SmoPe5EofOgDSwfsHDdQZM9B7OHTEywRwRrl2IpQGfMY0u7/YKTymTDXlkMoEUkgFX3KXpm7arNV1b+QYlXIlRZc/U+3dIvphKc0Tz+cl+aOGat7MAGse6aBSFwWxzeM0+nGi0gJIgjO+0IEeSgkEpOjSzRKHKZyJKCjcMGz9C1G+djtAAM1h54qXM9YCj4Yknioaso0LntGd8aYSrBKENMwSwcHPSwEdGCvF3EoAVxV8Xy/a3yWP0WBf6ZsQ3V6yPlkztF7RPT5kW1MNws8Ha1uvsPSYBLJ83mZBvcouJ40/+DxTE2E5QI31OFeqNwZC1veJBV8GUfzZplLgfPvoabwBwoAviLRCVeQ3+wx1k/KfMYcbZWPKeIGfXeA0o9qQJEBNU+u0glOxAbjFzherCu8XGerfYhEyDPA+QAv9TgS2l5QZpcqXIx8jBAju6HESDw0qLoZ5D3kAHMwRrypKgCg3el8Iu/YYLe7+zqH1YNPzVU3TlifKcA6l7EDr1z/+mmfQ48WtZY5EiZ2yzsrmenepOyTG0JV876ijfDZ9RrvTSvH6tqjh29t1seb2gJw8rTKcGbQYhW7CubSgINwBlaYqeESyYjNziEoWx8jbqCdt996qNP2/DDnpu/I2xyERiAgZYCXY3iZCHtd/0lgBn4Hjmv30kXMlUJTyZa1Ft73CnDir8kgm4PIC8SaGdWEcmxtKFoOx8u+ml5dWDfdRqy0bGiaqBQ+GUps78J+FOojA8juI7D5TuHGEWLFwoer1qAdNfljOWnPwA42nbkcI7hM2Yp1VIQ23SyqxqCm21XPMpREH0KrXk5MrxVKJ8dPolJfqtUPUbDQ3a8qjXW5jK+x6vEx91vl7wbizrsd5REyQsDf0WuD5iAFZ/rw55kCZENWO5oYxvRUE6ABazPl7t/G4d6eCl+DBS4WNzNJrMet3c777s/zIRcDQ2nPCwbjFFi3Ow/oDMkqFiGx3rFu8vAae52f2ixDC7uOxpPDBDPaZkLflnGxu0d6m12Jp+90+ylNiLKr5DqFQtbh6LKOo733eu0aKsy1aZQqnwh3SsGqa9Bx05Zmydmt9Oolffj33QLpB6Y7Sg7hBqFQqlFu4hFKgp X-Microsoft-Antispam-Message-Info: eejWE9H0IX9vETYj420SfExQj2eeC74ggZRJZ0R+eDRBLrrQOU5MFqSuqCyhphxQ0x0aKWki/LePobopTFS8ETp4JJ+Gph9/U/aUOOQ2qR+75pnlndYSUhiCzviWHQ8DGxqqba/d2mamORl55I7nMqnshSQaJpjKYodKoqunYU/HLO7ixJ8Q/uVPK7Fw7aTJLbv1N6hmLU91xeEXGE7hH310qVVyr799/nj38gRGZ3kwMFfb26WdAPsq5ktYMLan4oBQy/ge7Wti0C6QyWiTOB2V6L/9I+NC3sPG5AGww0NFZ8K1TtTAuQyQcOWvMT3rns8eGY/iGrfJD5WVZk4aFFDtNygEkY7pez88VEqTATI= X-Microsoft-Exchange-Diagnostics: 1;CY1PR15MB0170;6:9Urj2Xm1C4MSSqLM0iNj8C32/KUaphAWfpoj0oYrivVUDMjS5NTJSLMsCkCGFJ0ilSL1ZDGExCknv35By4wMlyWvkJwWDoK2G4KtP5nna8t8NijdM8qSDHnDbN/vViSrjhVdsX1imCx9fSdl1r48Jh0bpKYFuuvmJ82R6G+xZl/Ek4PhpnAOPLWXI1WR6GTK26OQNNx4101pnwV8BPRwo+29N3+dtvzHNGsccY2vJSa9RXvbfbF6REDrhwj1FKojP9b3Lb13f1E84barSsSxCDw8FhSpG9GQccRjFA5P3i3kti5jdD+xFoUK4URg++r/+WXSXzbwOoi1KHT7Rv/tXs4x9E0P5Dc+/1UixCdDyKga5N9qaTkw7QejDFb3QzSBPMEoC344e1TLHIPgkHWQ68gTpKZHPBUFrB82YwQ0gT4TClKnufbVSH4Anxcfm9NVOzQAOaRS1vALXo5YS5YUIQ==;5:1yBp4aAn4VheZEVoqujMSSqgXlaHgktqU3F7SIApLPtGafA66+0wqInQzMFghmRzidS+eh86IT6JYQWv7B7t/Wc308o20UDqj81mO0H35uOC6NaTFTNGA8azoi2EzZPjmKsFujG5Z8nN7wvDkrbkQU373f7yrp65WcY7e+9dV8Y=;7:tSkABIUSojNGu40yKUogPM4xdZoI3pICIHATxFfY6iKLaJF9p/WYEE/KaSpBUVnqLAksIGHvD9atSOdfOygWEvFfBqECfNjJgoiXJLoqIBFqkcK2yDCvFk1CvBg5gVmrfBBuTLLoQkK8rYCJS3+YiVqy2sFRhgXYkBd/ucxqM4Ymm1JpqfrLn5cBABqSJDcVDQ9rxpbZyqDM1KzRw+XDLtiExNomzaDhb4+PPeWgzlzY5pE19dhEFk+Zu/iwVoWm SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;CY1PR15MB0170;20:vEVxW970IJwbt6/R4QbwdPnIfGGYpZ6aKIFxwki0oZnBfiDEbWDCBrTHsZlqg23F9+/geLHmocFAGetRQo5SsoEho2tSYEmg4prKqA31z3AsDy/ChwHUPT8qsoF+fUAPgc2Pp7Nr/5bwg0u7MpIhRy9OwoXtUA+9GIt0TN9CdH0= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Aug 2018 16:27:01.2944 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6c2a5f6b-c76e-4307-1280-08d60c39eb51 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR15MB0170 X-OriginatorOrg: fb.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-08-27_06:,, signatures=0 X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP I've noticed, that dying memory cgroups are often pinned in memory by a single pagecache page. Even under moderate memory pressure they sometimes stayed in such state for a long time. That looked strange. My investigation showed that the problem is caused by applying the LRU pressure balancing math: scan = div64_u64(scan * fraction[lru], denominator), where denominator = fraction[anon] + fraction[file] + 1. Because fraction[lru] is always less than denominator, if the initial scan size is 1, the result is always 0. This means the last page is not scanned and has no chances to be reclaimed. Fix this by rounding up the result of the division. In practice this change significantly improves the speed of dying cgroups reclaim. Signed-off-by: Roman Gushchin Cc: Andrew Morton Cc: Johannes Weiner Cc: Michal Hocko Cc: Tejun Heo Cc: Rik van Riel Cc: Konstantin Khlebnikov Cc: Matthew Wilcox Signed-off-by: Roman Gushchin --- include/linux/math64.h | 2 ++ mm/vmscan.c | 6 ++++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/include/linux/math64.h b/include/linux/math64.h index 837f2f2d1d34..94af3d9c73e7 100644 --- a/include/linux/math64.h +++ b/include/linux/math64.h @@ -281,4 +281,6 @@ static inline u64 mul_u64_u32_div(u64 a, u32 mul, u32 divisor) } #endif /* mul_u64_u32_div */ +#define DIV64_U64_ROUND_UP(ll, d) div64_u64((ll) + (d) - 1, (d)) + #endif /* _LINUX_MATH64_H */ diff --git a/mm/vmscan.c b/mm/vmscan.c index d649b242b989..2c67a0121c6d 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2446,9 +2446,11 @@ static void get_scan_count(struct lruvec *lruvec, struct mem_cgroup *memcg, /* * Scan types proportional to swappiness and * their relative recent reclaim efficiency. + * Make sure we don't miss the last page + * because of a round-off error. */ - scan = div64_u64(scan * fraction[file], - denominator); + scan = DIV64_U64_ROUND_UP(scan * fraction[file], + denominator); break; case SCAN_FILE: case SCAN_ANON: