From patchwork Tue Dec 6 17:13:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13066170 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9CF7EC352A1 for ; Tue, 6 Dec 2022 17:14:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 362B48E0006; Tue, 6 Dec 2022 12:14:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 312878E0001; Tue, 6 Dec 2022 12:14:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1E4AD8E0006; Tue, 6 Dec 2022 12:14:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 0F2638E0001 for ; Tue, 6 Dec 2022 12:14:06 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id AE45C1C65BB for ; Tue, 6 Dec 2022 17:14:05 +0000 (UTC) X-FDA: 80212529250.13.3BB1CAC Received: from mail-ed1-f48.google.com (mail-ed1-f48.google.com [209.85.208.48]) by imf22.hostedemail.com (Postfix) with ESMTP id 51520C0005 for ; Tue, 6 Dec 2022 17:14:04 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=cmpxchg-org.20210112.gappssmtp.com header.s=20210112 header.b=U1tH6w6X; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf22.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.208.48 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1670346845; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KduHn+5gyrCZCJ3PZZKBhevPs/yCR7iZhiHXyVA0CdE=; b=zGNSrHSg+awmVQtB/ngA48Dz2ffQiu2K5an1OhhJmMBJNw8VAQREyVkzzxfyW9xhIGR3gO +MwKxm2UfHW/3XH88P/DKE411GdjaVGESywsyMKrypgy9h1WRDaW0IfLQuz74dq8hz1OoV mp7TP0BBwNPYWJyU8MweZbIcy/kC/Bg= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=cmpxchg-org.20210112.gappssmtp.com header.s=20210112 header.b=U1tH6w6X; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf22.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.208.48 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1670346845; a=rsa-sha256; cv=none; b=nDazfrDzoNB2D9b13hjdavQAPgRTpTn3GJ9mC5i9guEp/pcMBkHImbwkteVBwYrTITqFzF mFHK0b+lDLzHI3bLMc4gWRULg47vywx+DdlKnOVnQPBOBoHoVYh/g3MmT8rWr5Kq/v1FpY o0e3kUQU+FelHWt39mhTQA9n8qioZcc= Received: by mail-ed1-f48.google.com with SMTP id f7so21239016edc.6 for ; Tue, 06 Dec 2022 09:14:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KduHn+5gyrCZCJ3PZZKBhevPs/yCR7iZhiHXyVA0CdE=; b=U1tH6w6XQXpYI+Hv+M2BpssrkbT69EZiZtCJ8KUrJiRcGor3DEDhd1mEQhmYBwNQAL 9YedqqgQJoMza6N6wOnCopBmkDUVl4TGszL7044J1ET/RB82yJLOSkO3A10sVEkWg8tw 0z38g0G+6klmx7+DUWF/Lj59zdXgVZZkiBhgPm7HtpP6UoAVuRquXY0sRXXXI3tZin53 VD8Oobr5c3lkHwIXOQw6vLPebCgDSNO7nkyR39EwRxodTc4l3/xeYHpMD0CAYMriMrd2 N6OnCJlPli+HAoMxQiW5waMNstoUqDSvHSRvQNN/Uv8NqyA+jeyP7SgUNW7gld1NZOSf l4zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KduHn+5gyrCZCJ3PZZKBhevPs/yCR7iZhiHXyVA0CdE=; b=ro0McSKwtiFqs4D+zcdWwym5PSJD/QN23HNi8Ga+YTigRY3Ft7E7zkWXyuAbdY7uY6 syq/KvMxGbDUPccRLsNTjzYJ4Gnyas6lZgRyh8frgBUkn7R8jDWh8YTBoGON5NKuPuDB vLoA/ktILRuQxKTC7yvpC4YR2GWf/QcRj0TCHuqPf0NkImcGekJrHOfquzM9uhqj/YEP qShw0vmrPfgz8xw4zr3S53d8V0IsmTis3KEtQjNHM9aaOa7cmlWiZo1kYNhOokRYm0UJ yGZfIhFrwn5ugS9VJwSPIixCqeFyuOhCnrdbnU24V9DFZ/9zXQYFkwnMKV3aDmQqENRJ leKQ== X-Gm-Message-State: ANoB5pkjIUfRtT3IjA8TN5wa3VSSp7wwiqDzakJvKYZbWQK08oEHtnjc l9Fx4NwicoJroROi8C+DfAgzvQ== X-Google-Smtp-Source: AA0mqf7Ryd9NRT+3N9jQ0YqZ+vYbjjy0DyrVXIILJI2vcnGZiLsDKaLX+crjVJIhIeHPDsTlOyd56w== X-Received: by 2002:a05:6402:444a:b0:459:401:c23e with SMTP id o10-20020a056402444a00b004590401c23emr65038318edb.23.1670346843685; Tue, 06 Dec 2022 09:14:03 -0800 (PST) Received: from localhost ([2a02:8070:6387:ab20:15aa:3c87:c206:d15e]) by smtp.gmail.com with ESMTPSA id 9-20020a170906200900b007c0688a68cbsm7534622ejo.176.2022.12.06.09.14.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Dec 2022 09:14:03 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Linus Torvalds , Hugh Dickins , Shakeel Butt , Michal Hocko , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/3] mm: memcontrol: skip moving non-present pages that are mapped elsewhere Date: Tue, 6 Dec 2022 18:13:39 +0100 Message-Id: <20221206171340.139790-2-hannes@cmpxchg.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221206171340.139790-1-hannes@cmpxchg.org> References: <20221206171340.139790-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Spamd-Result: default: False [6.14 / 9.00]; SORBS_IRL_BL(3.00)[209.85.208.48:from]; R_MISSING_CHARSET(2.50)[]; MID_CONTAINS_FROM(1.00)[]; BAYES_HAM(-0.46)[72.55%]; RCVD_NO_TLS_LAST(0.10)[]; MIME_GOOD(-0.10)[text/plain]; BAD_REP_POLICIES(0.10)[]; TO_DN_SOME(0.00)[]; R_DKIM_ALLOW(0.00)[cmpxchg-org.20210112.gappssmtp.com:s=20210112]; RCPT_COUNT_SEVEN(0.00)[8]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; R_SPF_ALLOW(0.00)[+ip4:209.85.128.0/17]; DMARC_POLICY_ALLOW(0.00)[cmpxchg.org,none]; TO_MATCH_ENVRCPT_SOME(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[linux-mm@kvack.org]; DKIM_TRACE(0.00)[cmpxchg-org.20210112.gappssmtp.com:+]; ARC_SIGNED(0.00)[hostedemail.com:s=arc-20220608:i=1]; RCVD_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 51520C0005 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: xitioyhwtpds3z7i1amdi99qr9tmgfnx X-HE-Tag: 1670346844-482011 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: During charge moving, the pte lock and the page lock cover nearly all cases of stabilizing page_mapped(). The only exception is when we're looking at a non-present pte and find a page in the page cache or in the swapcache: if the page is mapped elsewhere, it can become unmapped outside of our control. For this reason, rmap needs lock_page_memcg(). We don't like cgroup-specific locks in generic MM code - especially in performance-critical MM code - and for a legacy feature that's unlikely to have many users left - if any. So remove the exception. Arguably that's better semantics anyway: the page is shared, and another process seems to be the more active user. Once we stop moving such pages, rmap doesn't need lock_page_memcg() anymore. The next patch will remove it. Suggested-by: Hugh Dickins Signed-off-by: Johannes Weiner Acked-by: Hugh Dickins Acked-by: Shakeel Butt --- mm/memcontrol.c | 52 ++++++++++++++++++++++++++++++++++++------------- 1 file changed, 38 insertions(+), 14 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 48c44229cf47..b696354c1b21 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -5681,7 +5681,7 @@ static struct page *mc_handle_file_pte(struct vm_area_struct *vma, * @from: mem_cgroup which the page is moved from. * @to: mem_cgroup which the page is moved to. @from != @to. * - * The caller must make sure the page is not on LRU (isolate_page() is useful.) + * The page must be locked and not on the LRU. * * This function doesn't do "charge" to new cgroup and doesn't do "uncharge" * from old cgroup. @@ -5698,20 +5698,13 @@ static int mem_cgroup_move_account(struct page *page, int nid, ret; VM_BUG_ON(from == to); + VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); VM_BUG_ON_FOLIO(folio_test_lru(folio), folio); VM_BUG_ON(compound && !folio_test_large(folio)); - /* - * Prevent mem_cgroup_migrate() from looking at - * page's memory cgroup of its source page while we change it. - */ - ret = -EBUSY; - if (!folio_trylock(folio)) - goto out; - ret = -EINVAL; if (folio_memcg(folio) != from) - goto out_unlock; + goto out; pgdat = folio_pgdat(folio); from_vec = mem_cgroup_lruvec(from, pgdat); @@ -5798,8 +5791,6 @@ static int mem_cgroup_move_account(struct page *page, mem_cgroup_charge_statistics(from, -nr_pages); memcg_check_events(from, nid); local_irq_enable(); -out_unlock: - folio_unlock(folio); out: return ret; } @@ -5848,6 +5839,29 @@ static enum mc_target_type get_mctgt_type(struct vm_area_struct *vma, else if (is_swap_pte(ptent)) page = mc_handle_swap_pte(vma, ptent, &ent); + if (target && page) { + if (!trylock_page(page)) { + put_page(page); + return ret; + } + /* + * page_mapped() must be stable during the move. This + * pte is locked, so if it's present, the page cannot + * become unmapped. If it isn't, we have only partial + * control over the mapped state: the page lock will + * prevent new faults against pagecache and swapcache, + * so an unmapped page cannot become mapped. However, + * if the page is already mapped elsewhere, it can + * unmap, and there is nothing we can do about it. + * Alas, skip moving the page in this case. + */ + if (!pte_present(ptent) && page_mapped(page)) { + unlock_page(page); + put_page(page); + return ret; + } + } + if (!page && !ent.val) return ret; if (page) { @@ -5864,8 +5878,11 @@ static enum mc_target_type get_mctgt_type(struct vm_area_struct *vma, if (target) target->page = page; } - if (!ret || !target) + if (!ret || !target) { + if (target) + unlock_page(page); put_page(page); + } } /* * There is a swap entry and a page doesn't exist or isn't charged. @@ -5905,6 +5922,10 @@ static enum mc_target_type get_mctgt_type_thp(struct vm_area_struct *vma, ret = MC_TARGET_PAGE; if (target) { get_page(page); + if (!trylock_page(page)) { + put_page(page); + return MC_TARGET_NONE; + } target->page = page; } } @@ -6143,6 +6164,7 @@ static int mem_cgroup_move_charge_pte_range(pmd_t *pmd, } putback_lru_page(page); } + unlock_page(page); put_page(page); } else if (target_type == MC_TARGET_DEVICE) { page = target.page; @@ -6151,6 +6173,7 @@ static int mem_cgroup_move_charge_pte_range(pmd_t *pmd, mc.precharge -= HPAGE_PMD_NR; mc.moved_charge += HPAGE_PMD_NR; } + unlock_page(page); put_page(page); } spin_unlock(ptl); @@ -6193,7 +6216,8 @@ static int mem_cgroup_move_charge_pte_range(pmd_t *pmd, } if (!device) putback_lru_page(page); -put: /* get_mctgt_type() gets the page */ +put: /* get_mctgt_type() gets & locks the page */ + unlock_page(page); put_page(page); break; case MC_TARGET_SWAP: From patchwork Tue Dec 6 17:13:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13066171 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4738DC352A1 for ; Tue, 6 Dec 2022 17:14:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CD7EB8E0007; Tue, 6 Dec 2022 12:14:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C88D78E0001; Tue, 6 Dec 2022 12:14:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B29948E0007; Tue, 6 Dec 2022 12:14:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A44128E0001 for ; Tue, 6 Dec 2022 12:14:20 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 856AAAB0EF for ; Tue, 6 Dec 2022 17:14:20 +0000 (UTC) X-FDA: 80212529880.23.2D170D6 Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) by imf03.hostedemail.com (Postfix) with ESMTP id 2A6C020012 for ; Tue, 6 Dec 2022 17:14:19 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=cmpxchg-org.20210112.gappssmtp.com header.s=20210112 header.b=uhHfurOu; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf03.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.218.53 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1670346860; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=10IT05+HKhb7xyirRRNdh88kI/5idGFXpcCcjOl6q8w=; b=UjzJxqtbaJFdnT/geIdrvSCtBguNVF1oraoop88D/Ohcx/Yobsf3oFpSo9vMCTtB4yS+dv XFhW19kMOAakeUO2dxZwJ+tfeO+lDw2qTQu2T2tt8ofX2H6e8j/zs6W6VK2uLyJ4W4xIDx Mkj40l5cGaWiCoJjb+917djGtqM0T+o= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=cmpxchg-org.20210112.gappssmtp.com header.s=20210112 header.b=uhHfurOu; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf03.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.218.53 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1670346860; a=rsa-sha256; cv=none; b=0Aed7tlBcOqdilUHOuZVjfpUzKlBviSlU3Zgo72st0aK6oQJBl00KYSr9Ly0bDkqD4nrdw 91iMY6xF++dmIjobUnnTX/jvaBbvJoaJXjcPkzrlKpvxfkYd0WT3vxXB4GA+Hw8sMRaF+V EtAangwvGuDMl9K+CTynBUygvaXD3iw= Received: by mail-ej1-f53.google.com with SMTP id fc4so7762818ejc.12 for ; Tue, 06 Dec 2022 09:14:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=10IT05+HKhb7xyirRRNdh88kI/5idGFXpcCcjOl6q8w=; b=uhHfurOuRNSHba5bcgYmHYTUiA0/YQOE6uTGKT8Nlcmry8Tts8/UpjfQ4dm10Rm3yF uwkMG3xtgqkmQ2FFsCCiTFe2q2QL5Ml99J3lDmyTYSbiu/vlsJJTOzc/rc6Ry1jRSKvY iqu4J98q8ZTNmGg26MTPVifMLjCyjIlyOmcx4ifNd5Tr368zq5meHCenw/bXIzm6w1FA fCnOb5UmPsn1+RqvMqwZKgSO2IcO8MmnYWogG426iZHQQb4cILJz/fSHqpNipzX9kxdN XCSKZBMRd1s3w6KC1U9lxsD54O3g3JBbdyxhjnR4Y8JhslpAg4hiNFX9CYv8+rpKR3at wjKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=10IT05+HKhb7xyirRRNdh88kI/5idGFXpcCcjOl6q8w=; b=nPaB3RpIqQCRLRj03PaQsLmMvLjdncs5vNRAZ5lKnxIruOCcLDdBUAzW7pineSibwc EeefVi7e5oN380Hwip2A74hmCWAuQ7tl8g6i+cyZWja+aykQ1CsK0PZYP6xybHYeIboI Oa5P29kQuTtYFovu41ujA4lb+1XmybHh1CS5IafjF9y6bN0bBd4xW4ulSleyIE3eSKZy Kkq7POKASiO8muzrP1axX/mSzUw2oUrMXrE0ytbYuhD3m+3ZyTRC3zHluumKFIqsYzRW OwNqFT4X3oxiMVM/q/yzBmjGi3Z6j5QzI/Pd+Gqw/r94sWNLzvRbjkX1r/1qWuEKhqKM +wwg== X-Gm-Message-State: ANoB5ploBVSjzx2hnb2YA1nuXLcHiWn3z9PFeyLRy2hVHVOZjuzUG0g3 cpW2RiEiF9Uuf2DH/93bxXwtGQ== X-Google-Smtp-Source: AA0mqf7kQJJ4LPTmjKqwgzg8jBUYoouC//Y9xZrafAmdsNdsR1Uk81RoFo7FkZKbUBxLLtJ04Q/PaA== X-Received: by 2002:a17:906:32ce:b0:78d:9022:f146 with SMTP id k14-20020a17090632ce00b0078d9022f146mr56850399ejk.656.1670346859021; Tue, 06 Dec 2022 09:14:19 -0800 (PST) Received: from localhost ([2a02:8070:6387:ab20:15aa:3c87:c206:d15e]) by smtp.gmail.com with ESMTPSA id r11-20020a056402034b00b0045bd14e241csm1187950edw.76.2022.12.06.09.14.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Dec 2022 09:14:18 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Linus Torvalds , Hugh Dickins , Shakeel Butt , Michal Hocko , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/3] mm: rmap: remove lock_page_memcg() Date: Tue, 6 Dec 2022 18:13:40 +0100 Message-Id: <20221206171340.139790-3-hannes@cmpxchg.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221206171340.139790-1-hannes@cmpxchg.org> References: <20221206171340.139790-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 2A6C020012 X-Stat-Signature: x9fsfpddn95uug6rhh8q17g97o3thyd4 X-Spamd-Result: default: False [3.57 / 9.00]; BAYES_HAM(-4.03)[95.14%]; SORBS_IRL_BL(3.00)[209.85.218.53:from]; R_MISSING_CHARSET(2.50)[]; MID_CONTAINS_FROM(1.00)[]; SUBJECT_HAS_UNDERSCORES(1.00)[]; RCVD_NO_TLS_LAST(0.10)[]; MIME_GOOD(-0.10)[text/plain]; BAD_REP_POLICIES(0.10)[]; TO_DN_SOME(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_SEVEN(0.00)[8]; R_DKIM_ALLOW(0.00)[cmpxchg-org.20210112.gappssmtp.com:s=20210112]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; R_SPF_ALLOW(0.00)[+ip4:209.85.128.0/17]; DMARC_POLICY_ALLOW(0.00)[cmpxchg.org,none]; TO_MATCH_ENVRCPT_SOME(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[linux-mm@kvack.org]; DKIM_TRACE(0.00)[cmpxchg-org.20210112.gappssmtp.com:+]; ARC_SIGNED(0.00)[hostedemail.com:s=arc-20220608:i=1]; RCVD_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[] X-HE-Tag: 1670346859-688243 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The previous patch made sure charge moving only touches pages for which page_mapped() is stable. lock_page_memcg() is no longer needed. Signed-off-by: Johannes Weiner Acked-by: Hugh Dickins Acked-by: Shakeel Butt --- mm/rmap.c | 26 ++++++++------------------ 1 file changed, 8 insertions(+), 18 deletions(-) diff --git a/mm/rmap.c b/mm/rmap.c index b616870a09be..32e48b1c5847 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1222,9 +1222,6 @@ void page_add_anon_rmap(struct page *page, bool compound = flags & RMAP_COMPOUND; bool first = true; - if (unlikely(PageKsm(page))) - lock_page_memcg(page); - /* Is page being mapped by PTE? Is this its first map to be added? */ if (likely(!compound)) { first = atomic_inc_and_test(&page->_mapcount); @@ -1262,15 +1259,14 @@ void page_add_anon_rmap(struct page *page, if (nr) __mod_lruvec_page_state(page, NR_ANON_MAPPED, nr); - if (unlikely(PageKsm(page))) - unlock_page_memcg(page); - - /* address might be in next vma when migration races vma_adjust */ - else if (first) - __page_set_anon_rmap(page, vma, address, - !!(flags & RMAP_EXCLUSIVE)); - else - __page_check_anon_rmap(page, vma, address); + if (likely(!PageKsm(page))) { + /* address might be in next vma when migration races vma_adjust */ + if (first) + __page_set_anon_rmap(page, vma, address, + !!(flags & RMAP_EXCLUSIVE)); + else + __page_check_anon_rmap(page, vma, address); + } mlock_vma_page(page, vma, compound); } @@ -1329,7 +1325,6 @@ void page_add_file_rmap(struct page *page, bool first; VM_BUG_ON_PAGE(compound && !PageTransHuge(page), page); - lock_page_memcg(page); /* Is page being mapped by PTE? Is this its first map to be added? */ if (likely(!compound)) { @@ -1365,7 +1360,6 @@ void page_add_file_rmap(struct page *page, NR_SHMEM_PMDMAPPED : NR_FILE_PMDMAPPED, nr_pmdmapped); if (nr) __mod_lruvec_page_state(page, NR_FILE_MAPPED, nr); - unlock_page_memcg(page); mlock_vma_page(page, vma, compound); } @@ -1394,8 +1388,6 @@ void page_remove_rmap(struct page *page, return; } - lock_page_memcg(page); - /* Is page being unmapped by PTE? Is this its last map to be removed? */ if (likely(!compound)) { last = atomic_add_negative(-1, &page->_mapcount); @@ -1451,8 +1443,6 @@ void page_remove_rmap(struct page *page, * and remember that it's only reliable while mapped. */ - unlock_page_memcg(page); - munlock_vma_page(page, vma, compound); } From patchwork Tue Dec 6 17:13:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13066172 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D34FC3A5A7 for ; Tue, 6 Dec 2022 17:14:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9F1B18E0008; Tue, 6 Dec 2022 12:14:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9A34F8E0001; Tue, 6 Dec 2022 12:14:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 86A278E0008; Tue, 6 Dec 2022 12:14:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 77C998E0001 for ; Tue, 6 Dec 2022 12:14:27 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 4D9E6AB317 for ; Tue, 6 Dec 2022 17:14:27 +0000 (UTC) X-FDA: 80212530174.21.D50F0BA Received: from mail-ed1-f44.google.com (mail-ed1-f44.google.com [209.85.208.44]) by imf26.hostedemail.com (Postfix) with ESMTP id 8466A14000F for ; Tue, 6 Dec 2022 17:14:26 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=cmpxchg-org.20210112.gappssmtp.com header.s=20210112 header.b=SVtFnYIH; spf=pass (imf26.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.208.44 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1670346866; a=rsa-sha256; cv=none; b=KHQX95+YlWxN1Wo4Ca9hgyF0Nx6dyLOjVofj8gRI4+P+nU7RdEeFi/mUeu1aG1cmfxhln1 zVwT1OdBQ78nWt90GmpHxzfuwXAaC7G0E+5Ih0QfUgToBVWZJt1fN8o51OEKn4Ux7M8fFQ zu/3Z3A+i8hGqZWBk23KG7hY2uQjBHQ= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=cmpxchg-org.20210112.gappssmtp.com header.s=20210112 header.b=SVtFnYIH; spf=pass (imf26.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.208.44 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1670346866; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=i7DLHKnfgUHUqbOM/z8uZ79UX5oa3K/ENnh6OCW5tqs=; b=6BR+GUNpxn40YJQ3ybF8gChBJ3ZxXHmCPxogQbGJS1r3A0pB/DgStJDswW9B+7ZRJJOgnF 9OBmEvqo2mmGttXEAGTpmqVik0viAOoCfKgdbuI3kry8JdoYc3it88csQIxO6ue/Umk4M+ QWMeqAuNw8skECMQzf2aTXphVk3F8eM= Received: by mail-ed1-f44.google.com with SMTP id v8so21254021edi.3 for ; Tue, 06 Dec 2022 09:14:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=i7DLHKnfgUHUqbOM/z8uZ79UX5oa3K/ENnh6OCW5tqs=; b=SVtFnYIHXMecHvxYJx77nJRHucuFwk5/9UG0Gzc0VixuyFsGI76/q4dIZTOj1/SyqT O82nl1FhPWAuz6lthA4angI6hMP1oZ1qGqag/8sKuVxBGm3pxTJTOWSvPrnOqtbj5jfd dswMPCRu9hqrDyIeHIcBn4yV+mIpsTk4mtqkiH6YM7R3FEaAd3SHSxjIa0Zn6WW0paME 3RAvrTuiv3qqMZnPDfVp3gFd0wYptMFNXnJ/2pl0ugruoWzEj4p0JQBQjH47NBbb0t+v fuorAUnWM8LTnOrOiOvW4DKQWz57wcJP0qAtpCExb4/f/oq8NDnkZRVOa6F4b3YFJBO1 Hedg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=i7DLHKnfgUHUqbOM/z8uZ79UX5oa3K/ENnh6OCW5tqs=; b=DdTXUKG93g7xi2cDKWLC3mSobltUlbPIsFcWLEyUSFFJkXCfG83ZQHE/xPIBq/1i0F D1JtvDc9zNBKk6FUrAdNULTiakbUkENSeXAc324slFpPA4W0B6ecaEPevSQrzlUNa3kS 2xZGMMBgdlymhrrxVXUb2zPRsvX5OOs0MkHEW2JxhWEGJYaYDHXXCgTBa0ASnYK58kbH zXy+x0afRriXFCf54LxvNPdqwfFxEN9/rZrq8/foW8ZA5M7XSw0lNcaUXAIrnpSBr8wW GeAA5es4yocUKCulDhmJ2/t2xJBfYGULU8J9vT/1ZVwYyAvXwRN/fqHLLP5scIiq5FB0 BoYw== X-Gm-Message-State: ANoB5pmgQeBn5gx3nGoXqXRMEdA7ggrqBxlwJe6A8GFsiomOr/X1g0zr rqrK7McOYK8nfUi9VJ2vPyRi+g== X-Google-Smtp-Source: AA0mqf4zM0Xfm3o50FPw1HED00hbjDIYwQEpUTYuG4CSMG/pv2gDG2TkVQ9Z4W/PFOBStf5o5pM4yg== X-Received: by 2002:a05:6402:2213:b0:46b:1d60:f60a with SMTP id cq19-20020a056402221300b0046b1d60f60amr37331636edb.193.1670346865233; Tue, 06 Dec 2022 09:14:25 -0800 (PST) Received: from localhost ([2a02:8070:6387:ab20:15aa:3c87:c206:d15e]) by smtp.gmail.com with ESMTPSA id kw25-20020a170907771900b007c0ae8569d6sm1458481ejc.146.2022.12.06.09.14.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Dec 2022 09:14:24 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Linus Torvalds , Hugh Dickins , Shakeel Butt , Michal Hocko , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/3] mm: memcontrol: deprecate charge moving Date: Tue, 6 Dec 2022 18:13:41 +0100 Message-Id: <20221206171340.139790-4-hannes@cmpxchg.org> X-Mailer: git-send-email 2.38.1 In-Reply-To: <20221206171340.139790-1-hannes@cmpxchg.org> References: <20221206171340.139790-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Spamd-Result: default: False [0.60 / 9.00]; BAYES_HAM(-6.00)[100.00%]; SORBS_IRL_BL(3.00)[209.85.208.44:from]; R_MISSING_CHARSET(2.50)[]; MID_CONTAINS_FROM(1.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_NO_TLS_LAST(0.10)[]; BAD_REP_POLICIES(0.10)[]; TO_DN_SOME(0.00)[]; R_DKIM_ALLOW(0.00)[cmpxchg-org.20210112.gappssmtp.com:s=20210112]; RCPT_COUNT_SEVEN(0.00)[8]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; R_SPF_ALLOW(0.00)[+ip4:209.85.128.0/17]; DMARC_POLICY_ALLOW(0.00)[cmpxchg.org,none]; TO_MATCH_ENVRCPT_SOME(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[linux-mm@kvack.org]; DKIM_TRACE(0.00)[cmpxchg-org.20210112.gappssmtp.com:+]; ARC_SIGNED(0.00)[hostedemail.com:s=arc-20220608:i=1]; RCVD_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 8466A14000F X-Stat-Signature: cnbupbx5ahn7pit7yb7zhagbkzy9cwnh X-HE-Tag: 1670346866-82641 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Charge moving mode in cgroup1 allows memory to follow tasks as they migrate between cgroups. This is, and always has been, a questionable thing to do - for several reasons. First, it's expensive. Pages need to be identified, locked and isolated from various MM operations, and reassigned, one by one. Second, it's unreliable. Once pages are charged to a cgroup, there isn't always a clear owner task anymore. Cache isn't moved at all, for example. Mapped memory is moved - but if trylocking or isolating a page fails, it's arbitrarily left behind. Frequent moving between domains may leave a task's memory scattered all over the place. Third, it isn't really needed. Launcher tasks can kick off workload tasks directly in their target cgroup. Using dedicated per-workload groups allows fine-grained policy adjustments - no need to move tasks and their physical pages between control domains. The feature was never forward-ported to cgroup2, and it hasn't been missed. Despite it being a niche usecase, the maintenance overhead of supporting it is enormous. Because pages are moved while they are live and subject to various MM operations, the synchronization rules are complicated. There are lock_page_memcg() in MM and FS code, which non-cgroup people don't understand. In some cases we've been able to shift code and cgroup API calls around such that we can rely on native locking as much as possible. But that's fragile, and sometimes we need to hold MM locks for longer than we otherwise would (pte lock e.g.). Mark the feature deprecated. Hopefully we can remove it soon. Signed-off-by: Johannes Weiner Acked-by: Shakeel Butt Acked-by: Hugh Dickins Signed-off-by: Johannes Weiner Acked-by: Shakeel Butt Acked-by: Hugh Dickins --- Documentation/admin-guide/cgroup-v1/memory.rst | 11 ++++++++++- mm/memcontrol.c | 4 ++++ 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/cgroup-v1/memory.rst b/Documentation/admin-guide/cgroup-v1/memory.rst index 60370f2c67b9..87d7877b98ec 100644 --- a/Documentation/admin-guide/cgroup-v1/memory.rst +++ b/Documentation/admin-guide/cgroup-v1/memory.rst @@ -86,6 +86,8 @@ Brief summary of control files. memory.swappiness set/show swappiness parameter of vmscan (See sysctl's vm.swappiness) memory.move_charge_at_immigrate set/show controls of moving charges + This knob is deprecated and shouldn't be + used. memory.oom_control set/show oom controls. memory.numa_stat show the number of memory usage per numa node @@ -717,9 +719,16 @@ Soft limits can be setup by using the following commands (in this example we It is recommended to set the soft limit always below the hard limit, otherwise the hard limit will take precedence. -8. Move charges at task migration +8. Move charges at task migration (DEPRECATED!) ================================= +THIS IS DEPRECATED! + +It's expensive and unreliable! It's better practice to launch workload +tasks directly from inside their target cgroup. Use dedicated workload +cgroups to allow fine-grained policy adjustments without having to +move physical pages between control domains. + Users can move charges associated with a task along with task migration, that is, uncharge task's pages from the old cgroup and charge them to the new cgroup. This feature is not supported in !CONFIG_MMU environments because of lack of diff --git a/mm/memcontrol.c b/mm/memcontrol.c index b696354c1b21..e650a38d9a90 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3919,6 +3919,10 @@ static int mem_cgroup_move_charge_write(struct cgroup_subsys_state *css, { struct mem_cgroup *memcg = mem_cgroup_from_css(css); + pr_warn_once("Cgroup memory moving is deprecated. " + "Please report your usecase to linux-mm@kvack.org if you " + "depend on this functionality.\n"); + if (val & ~MOVE_MASK) return -EINVAL;