From patchwork Mon Jan 14 09:54:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 10761849 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2D15413B5 for ; Mon, 14 Jan 2019 09:55:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2A8C628A75 for ; Mon, 14 Jan 2019 09:54:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1E42D28AD2; Mon, 14 Jan 2019 09:54:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8C2F728A75 for ; Mon, 14 Jan 2019 09:54:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8426E8E0004; Mon, 14 Jan 2019 04:54:56 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7A0518E0002; Mon, 14 Jan 2019 04:54:56 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 66A108E0004; Mon, 14 Jan 2019 04:54:56 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) by kanga.kvack.org (Postfix) with ESMTP id 269A38E0002 for ; Mon, 14 Jan 2019 04:54:56 -0500 (EST) Received: by mail-pf1-f200.google.com with SMTP id 75so15730540pfq.8 for ; Mon, 14 Jan 2019 01:54:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:in-reply-to:references:mime-version :content-transfer-encoding:message-id; bh=9xdCTWvbVvGXiqgce93VVd9Xu5FSL/9lTKkEFR2KI7s=; b=QxaKMx2nOzxRWSR7ojzAXrGbDlHe/1iSOu80pNFhMTLV/cuzmAKdUH/EPZGNw8M98x 8a0Z16jdRxpXl5/7q1XL86nXfbOI2cj0em7ArZzPvn0MTPsa8GtY8ae0KaspviWh5F+E VWCyaME0QqAWHuyVqLvqAZrLzU5dfY3CDTIenLsz0WnVq0Kt0BwFvqHpwtPiNA3n12CA yJOa7QjvB+URII9r2V+kQr3m6GCs4prd8HuXzYj2XujwxSsSS645E+5CwwnCHIAF0iT/ D8cQZafshlRGYEEGqQe07gsyb8sHiun7JocozoiSg/5f994b8a8MuPWVJCFSETYVhEdG 0XhQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com X-Gm-Message-State: AJcUukd7yxtpBiheNGEtRI//YE9Vt3/y/n0dryzXyMvdKksHABcTicir hshC+HiS3vzN6h0PUSPONc+m3J0PRfP35GzOjy6EzhkFJ9ddFoZIT0sOpcFyf/Mvs813MVuAOSp FxwhgondBfAfzIBnnrjg/KHQWIgIeHCGbWc8dMLn5QuoN5ljgu+i/B62RgBo/cmXOog== X-Received: by 2002:a17:902:9692:: with SMTP id n18mr25091493plp.333.1547459695809; Mon, 14 Jan 2019 01:54:55 -0800 (PST) X-Google-Smtp-Source: ALg8bN7FcutnIZPaf7nRUCAtVUxqB/5rrzn2+XsUD5msaZi5oAXCmHGo9kG6LNtsT/PbzZaYsWdU X-Received: by 2002:a17:902:9692:: with SMTP id n18mr25091451plp.333.1547459694894; Mon, 14 Jan 2019 01:54:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547459694; cv=none; d=google.com; s=arc-20160816; b=KsWAPs3wpuSV629GM+eLrYy2oJb7LW3r80o/CdUA5jvm47TAqQq+pQDzqV4FgAOlFy lYBBmm4f/ZJ/Lbhu1IZg6O5PyvWO/5fc4H/3C32Zep4v7poVItD56M/UAX4zpkiNKwR7 UVCyX8Sb4omeQ3U/lYsvRIajYbVMNttike0d0qc/koxuaq2e1mdf0CoEklyqpJTYVQbN YzLpA0stoef0w8q6AmC80k6f2IlwGyn8GpldxlefhiwRr4jAY3tgnanFPdgNPsmuN7TU jQuZjAuXUb1Pgi/FAY0ItGe2ulUg7yrrgjncMjwtqUYuGtG97PzGVssqUoBcg4lKsWPl BZOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:content-transfer-encoding:mime-version:references :in-reply-to:date:subject:cc:to:from; bh=9xdCTWvbVvGXiqgce93VVd9Xu5FSL/9lTKkEFR2KI7s=; b=aV32zRtR8cDNrZWchHZu9sExKhOpIkZfLII1mckGorlYEPo0GpoRhO7a5TnDX++WZx 7FHeOQaZOSFpKGBm/nM0ls3dTmOa+RzrdRGaFFRdJGIuzwsb0JUYkIj/njhxU72wFUm8 rlFqykU4wmt2xcPYSMayMcf6HF+AXuMJ+Q0FvzFkjGoxjWRI2DzmJiW9jkoa2VK7eNNy kv43Pm5AvAn6rYwZPrkfvh+72LQmNBLB2dH/y5v3IdA/To0GfN4KTmOd3+on1h7OrmL2 Po4ZaR2ddCm19P8ZX8Ek8CvRgR//IWxHQszOaAT+uYcsdVQzLmEGsb8Vw2nJkQtJ6/aS AJ9Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com. [148.163.156.1]) by mx.google.com with ESMTPS id t5si79223729pgc.369.2019.01.14.01.54.54 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 14 Jan 2019 01:54:54 -0800 (PST) Received-SPF: pass (google.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) client-ip=148.163.156.1; Authentication-Results: mx.google.com; spf=pass (google.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id x0E9mXuG088720 for ; Mon, 14 Jan 2019 04:54:54 -0500 Received: from e14.ny.us.ibm.com (e14.ny.us.ibm.com [129.33.205.204]) by mx0a-001b2d01.pphosted.com with ESMTP id 2q0ntt6jb3-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Mon, 14 Jan 2019 04:54:54 -0500 Received: from localhost by e14.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Mon, 14 Jan 2019 09:54:52 -0000 Received: from b01cxnp23033.gho.pok.ibm.com (9.57.198.28) by e14.ny.us.ibm.com (146.89.104.201) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Mon, 14 Jan 2019 09:54:50 -0000 Received: from b01ledav005.gho.pok.ibm.com (b01ledav005.gho.pok.ibm.com [9.57.199.110]) by b01cxnp23033.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id x0E9snT225559102 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Mon, 14 Jan 2019 09:54:49 GMT Received: from b01ledav005.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6871BAE063; Mon, 14 Jan 2019 09:54:49 +0000 (GMT) Received: from b01ledav005.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 02920AE06A; Mon, 14 Jan 2019 09:54:46 +0000 (GMT) Received: from skywalker.in.ibm.com (unknown [9.124.31.106]) by b01ledav005.gho.pok.ibm.com (Postfix) with ESMTP; Mon, 14 Jan 2019 09:54:46 +0000 (GMT) From: "Aneesh Kumar K.V" To: akpm@linux-foundation.org, Michal Hocko , Alexey Kardashevskiy , David Gibson , Andrea Arcangeli , mpe@ellerman.id.au Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, "Aneesh Kumar K.V" Subject: [PATCH V7 1/4] mm/cma: Add PF flag to force non cma alloc Date: Mon, 14 Jan 2019 15:24:33 +0530 X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190114095438.32470-1-aneesh.kumar@linux.ibm.com> References: <20190114095438.32470-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 19011409-0052-0000-0000-000003780AE6 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00010402; HX=3.00000242; KW=3.00000007; PH=3.00000004; SC=3.00000274; SDB=6.01146283; UDB=6.00597007; IPR=6.00926570; MB=3.00025118; MTD=3.00000008; XFM=3.00000015; UTC=2019-01-14 09:54:52 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 19011409-0053-0000-0000-00005F75E2CA Message-Id: <20190114095438.32470-2-aneesh.kumar@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-01-14_05:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1901140081 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch adds PF_MEMALLOC_NOCMA which make sure any allocation in that context is marked non-movable and hence cannot be satisfied by CMA region. This is useful with get_user_pages_longterm where we want to take a page pin by migrating pages from CMA region. Marking the section PF_MEMALLOC_NOCMA ensures that we avoid unnecessary page migration later. Suggested-by: Andrea Arcangeli Reviewed-by: Andrea Arcangeli Signed-off-by: Aneesh Kumar K.V --- include/linux/sched.h | 1 + include/linux/sched/mm.h | 48 +++++++++++++++++++++++++++++++++------- 2 files changed, 41 insertions(+), 8 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 224666226e87..687b0edf97b2 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1406,6 +1406,7 @@ extern struct pid *cad_pid; #define PF_RANDOMIZE 0x00400000 /* Randomize virtual address space */ #define PF_SWAPWRITE 0x00800000 /* Allowed to write to swap */ #define PF_MEMSTALL 0x01000000 /* Stalled due to lack of memory */ +#define PF_MEMALLOC_NOCMA 0x02000000 /* All allocation request will have _GFP_MOVABLE cleared */ #define PF_NO_SETAFFINITY 0x04000000 /* Userland is not allowed to meddle with cpus_allowed */ #define PF_MCE_EARLY 0x08000000 /* Early kill for mce process policy */ #define PF_MUTEX_TESTER 0x20000000 /* Thread belongs to the rt mutex tester */ diff --git a/include/linux/sched/mm.h b/include/linux/sched/mm.h index 3bfa6a0cbba4..0cd9f10423fb 100644 --- a/include/linux/sched/mm.h +++ b/include/linux/sched/mm.h @@ -148,17 +148,25 @@ static inline bool in_vfork(struct task_struct *tsk) * Applies per-task gfp context to the given allocation flags. * PF_MEMALLOC_NOIO implies GFP_NOIO * PF_MEMALLOC_NOFS implies GFP_NOFS + * PF_MEMALLOC_NOCMA implies no allocation from CMA region. */ static inline gfp_t current_gfp_context(gfp_t flags) { - /* - * NOIO implies both NOIO and NOFS and it is a weaker context - * so always make sure it makes precedence - */ - if (unlikely(current->flags & PF_MEMALLOC_NOIO)) - flags &= ~(__GFP_IO | __GFP_FS); - else if (unlikely(current->flags & PF_MEMALLOC_NOFS)) - flags &= ~__GFP_FS; + if (unlikely(current->flags & + (PF_MEMALLOC_NOIO | PF_MEMALLOC_NOFS | PF_MEMALLOC_NOCMA))) { + /* + * NOIO implies both NOIO and NOFS and it is a weaker context + * so always make sure it makes precedence + */ + if (current->flags & PF_MEMALLOC_NOIO) + flags &= ~(__GFP_IO | __GFP_FS); + else if (current->flags & PF_MEMALLOC_NOFS) + flags &= ~__GFP_FS; +#ifdef CONFIG_CMA + if (current->flags & PF_MEMALLOC_NOCMA) + flags &= ~__GFP_MOVABLE; +#endif + } return flags; } @@ -248,6 +256,30 @@ static inline void memalloc_noreclaim_restore(unsigned int flags) current->flags = (current->flags & ~PF_MEMALLOC) | flags; } +#ifdef CONFIG_CMA +static inline unsigned int memalloc_nocma_save(void) +{ + unsigned int flags = current->flags & PF_MEMALLOC_NOCMA; + + current->flags |= PF_MEMALLOC_NOCMA; + return flags; +} + +static inline void memalloc_nocma_restore(unsigned int flags) +{ + current->flags = (current->flags & ~PF_MEMALLOC_NOCMA) | flags; +} +#else +static inline unsigned int memalloc_nocma_save(void) +{ + return 0; +} + +static inline void memalloc_nocma_restore(unsigned int flags) +{ +} +#endif + #ifdef CONFIG_MEMCG /** * memalloc_use_memcg - Starts the remote memcg charging scope.