From patchwork Wed Aug 14 03:54:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13762818 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53E02C3DA4A for ; Wed, 14 Aug 2024 03:55:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DABE86B0085; Tue, 13 Aug 2024 23:55:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D33B06B0089; Tue, 13 Aug 2024 23:55:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B0E786B008A; Tue, 13 Aug 2024 23:55:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 8851B6B0085 for ; Tue, 13 Aug 2024 23:55:03 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 33C3CC0BD8 for ; Wed, 14 Aug 2024 03:55:03 +0000 (UTC) X-FDA: 82449485286.02.CCE35E1 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf07.hostedemail.com (Postfix) with ESMTP id 6C9874000E for ; Wed, 14 Aug 2024 03:55:01 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=qy4RYayM; spf=pass (imf07.hostedemail.com: domain of 3lCq8ZgYKCBcLHM4xB3BB381.zB985AHK-997Ixz7.BE3@flex--yuzhao.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3lCq8ZgYKCBcLHM4xB3BB381.zB985AHK-997Ixz7.BE3@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723607631; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KXVmwVL4hZRiA8cTojFTWKVHpRNtdM/qQOsCSMq7lmE=; b=7WlROoIgw46ervClNAu03VzRGxmHhaNpIwL/KXkxP+zJNKy6bUcd841xRqmL9smj2Cwf83 IDeLUocLZWOY1AF9szeh44w4qRt6wdoAdIU8dgeuND51myO1mbbhCMUTz5gILiXXZ4cwpN hZJFSD1N8ilAN+5fBJma8DtHvrPgQN0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723607631; a=rsa-sha256; cv=none; b=tPIZIb5jWw38LKvM0Mqid/PPa6ZsnvDlsQyYKrAqnAmVkEPRqRe70VPheyzGyBvKDA7GC3 RzgEpTL5oJZ/exi4aM276UEa8IDM21LtMlw/vA196kO1twLwjD55iNC3iCmcC2mSaf7Kxz 4ZHWwMd6JAcg4ss5VtnbG5n0xSq+Af4= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=qy4RYayM; spf=pass (imf07.hostedemail.com: domain of 3lCq8ZgYKCBcLHM4xB3BB381.zB985AHK-997Ixz7.BE3@flex--yuzhao.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3lCq8ZgYKCBcLHM4xB3BB381.zB985AHK-997Ixz7.BE3@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e05e3938a37so10222201276.2 for ; Tue, 13 Aug 2024 20:55:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723607700; x=1724212500; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=KXVmwVL4hZRiA8cTojFTWKVHpRNtdM/qQOsCSMq7lmE=; b=qy4RYayM0wWAER/bOasTAPGflQIxUDnAGkINKJRNQ3FokXBI8xOTyYT46fJnyPgt+O mjgiQh/vLjQs3FOCXeWg4yQ9ges7GawLY2wSe7GEq1poU3A8193nhAG4duTJI/FPIyad tAM3Of0ilWxCTjb/jJ1rqx0zxyUbYHbz68cPFQAISrmevQSh+Jjk+ot79VrzSIU3QyfR icmxIh9IsPNsqYRH9egzk5mmRfvmthDZPMVSGtqY1JyGbLYO81Pg7Iv+k32gI7HTe4f+ EFkv8jh0VRIne6IPZG+5UXXzTuU3ABW51hX9G7UMAKjPwSbTACMY5x2on3taprC/1yry 47dA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723607700; x=1724212500; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KXVmwVL4hZRiA8cTojFTWKVHpRNtdM/qQOsCSMq7lmE=; b=kzOtPoUelOgGzfExgHMmXBFyc8bXGl57ZyE2kIoTxt75rDoSRS9o5Ok2OEB5KA72gs 2JBSz/9TAMfpSSJjm5wXYAuZIodCErMdBkofOVHt0/fyH+aDdDAa7/xqMSShBf9am3zb z2VvHQpqiAvLYLkuxfODJvMZ2pQsIXan+gsyc3m+au/81O6syqYzm6PvOH9E6if0bBJH gjVyMMMihAtt2kdvRlQYqhM3uSkx2VUl4AHsnv+8XRrr8QDZnjLQHT9//0ujCWMDClCM tJhQfq9k0hNGcH3FtFruCiTGaAS4QkEurKemD10TUNYV3FoTxuWqZbcwSbjp6B2Bs/Gi l6og== X-Forwarded-Encrypted: i=1; AJvYcCVhdg4sYQkFRIsJEMkrH3WxcmW+yBq+oZx+5HPJzT+prJcsH27KX6+Dh2rN4NWz1bWwYTtNZAaSvsPO3f72Di91CoU= X-Gm-Message-State: AOJu0YySaxhZwBCwhAIDASF9g5cwweq8sjeYlopASJV6ojqUNaTUBNml cYtNhGS9bCEKTgNvrAGwiQvOmYgr8k0PZymvvjjEYeCu74xMqVzPDf3zPrwjwnoVEyZGF7+tyDo NkQ== X-Google-Smtp-Source: AGHT+IHdxjtU6GE8e6uUmdDhkDkgKFsun57md8iAQGJ8zy2FVnTlUkIoTwELvFGK+N9W879dEyGV8DeAgrM= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:c8ad:fcc7:f7cf:f2fd]) (user=yuzhao job=sendgmr) by 2002:a25:c788:0:b0:e03:62dc:63de with SMTP id 3f1490d57ef6-e1155ada202mr81659276.6.1723607700515; Tue, 13 Aug 2024 20:55:00 -0700 (PDT) Date: Tue, 13 Aug 2024 21:54:50 -0600 In-Reply-To: <20240814035451.773331-1-yuzhao@google.com> Mime-Version: 1.0 References: <20240814035451.773331-1-yuzhao@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240814035451.773331-3-yuzhao@google.com> Subject: [PATCH mm-unstable v2 2/3] mm/cma: add cma_{alloc,free}_folio() From: Yu Zhao To: Andrew Morton , Muchun Song Cc: "Matthew Wilcox (Oracle)" , Zi Yan , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao X-Rspamd-Queue-Id: 6C9874000E X-Stat-Signature: a37faq3een887sq7s58p9uu1qu4kcf5e X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1723607701-217259 X-HE-Meta: U2FsdGVkX19MQItmHsmr2Rx1tXoF5u+r1pbt23aRP7GO3IaxtIdJRUotT8sb4kuw8oTzVGkTU8/pKgi/B7l8EQeuOkcrT9wGOOfWovkGkZT4sQ9+TQhB1UNY6a+OOK/CvtzvYLWz2ff6GKJuwtA5DvMckGQ2Jzp4s0uHXmKzT46DqAM2MnGaWx4/zb/jR98iFFTj4rM2NUsukomVdIN+UHxZYC9AYQph5Ep2eq9KEOg2Dm6wsL0aJnmCA7awp7Q+3lZhP2iFTs7zBQqbkrFIJIpaQCIENzS4KreJDYS6T7mivnPGtx8YUdsc654N88v5AmMEdwkQ8K6MxkiXyWk9IXscM82HoWA0aDwsKIKyT7U2YtsJnc2yZ43PFL/HEuu0GP26wb/aJxntUC0cd6X86PE/6UpbbgaoFuqyWglyHS/uE5PQpSScKGLoW3M0IbINfsKsEOlDhy5MV8a5L7HXJ0CSP3OIF2cUUmqJXWkgih8vh0kZUhyqXY0rQi1i6lfSFMc+2UAvBHCgH+M7J2BTFcKC8RxqrvtJwi56aImML7s7ePFnbRiLa4lRGjIGdM1/zRfneeQrWqVqs2Mczq5wt3LAGhLrVEMOAuqNyCKyioDw3m5wbNszszYg+FUfC3CSkMF2nN4X5MQVF/eV4HGud69qINFa8/5VfNXs3Dv3dr1cS67IHui9hCLZin74k8eTY3GOMd2qJrwk6m/sEuPHD1ZCSib97TeXHlrAQ4143CN8jpx7StjYPvNpvceaaiYleHW82tUvydKlrfA1SUecFXd7H2Ha73kiwSlXS9MVguNRcQh9l7MphzAUhK+gPth2beZ2btgnru6qkCP+aOlVhrHCYYx7rx3NHKc0q9N34937f47ROER7ZAMGSvxCX1EOHsSlKICFr3VtYqojDwoOKJBP6XqZdl7cQEediG1zDsSAmuFUJLNKjOHpUwOkFlXmccMSB3Mumv6pGOg+DPV DWCWHDaj al36Qui6TESjh5YVeJtkxJnh/ddEKqSrFvW3lB13DqtU6XZHoNicvwGhQBskSmtZD3MQZ9IcV+YLxLjRREqkXh8fiGy5oDjMDQsHz50HUHXQu+7WCKbLEQDzVFKFCbLwlv6I2shQFqe2wbnJ6R/DdOs3agloNl37RG2+Ksu5WQuZsFPRunbUQK9SN+Y467QGsPUaIpVzeAzTMCZgE3qoqRkWi3PUZdclPdCVbpc0whEAQfvIq8XkO2Apj8ImIpR+p9O3bSVFqjF5cC5sG0F6gjOX/YaW7lugFwSWdVQik7J00TIy7kdGNw8tZIq2xi/wVnzavISyF4/Y1BIVEJ3cYIfSsOWgBJjMDRnhqejxOmfJ96vFgBMyoW4EyuPjN1d4Iq0eKlkoAZNrXD6USYFh/j2yND9NaGlg/QAOV5nCItgJklU61xRwIJN8cSajqCXJ5RHVijBvClvj+3R5qykMEqy22Y0Yr8PBkhJcAD9Rtuhec4fTFY7/zQpgQk6Ui02CPAJTb7iQa7xDwDaKDuvPveHWTw/2lAUYgmKk4yr1OOvizWz5sefLmzifUZA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: With alloc_contig_range() and free_contig_range() supporting large folios, CMA can allocate and free large folios too, by cma_alloc_folio() and cma_free_folio(). Signed-off-by: Yu Zhao --- include/linux/cma.h | 16 +++++++++++++ mm/cma.c | 55 ++++++++++++++++++++++++++++++++------------- 2 files changed, 56 insertions(+), 15 deletions(-) diff --git a/include/linux/cma.h b/include/linux/cma.h index 9db877506ea8..d15b64f51336 100644 --- a/include/linux/cma.h +++ b/include/linux/cma.h @@ -52,4 +52,20 @@ extern bool cma_release(struct cma *cma, const struct page *pages, unsigned long extern int cma_for_each_area(int (*it)(struct cma *cma, void *data), void *data); extern void cma_reserve_pages_on_error(struct cma *cma); + +#ifdef CONFIG_CMA +struct folio *cma_alloc_folio(struct cma *cma, int order, gfp_t gfp); +bool cma_free_folio(struct cma *cma, const struct folio *folio); +#else +static inline struct folio *cma_alloc_folio(struct cma *cma, int order, gfp_t gfp) +{ + return NULL; +} + +static inline bool cma_free_folio(struct cma *cma, const struct folio *folio) +{ + return false; +} +#endif + #endif diff --git a/mm/cma.c b/mm/cma.c index 95d6950e177b..4354823d28cf 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -403,18 +403,8 @@ static void cma_debug_show_areas(struct cma *cma) spin_unlock_irq(&cma->lock); } -/** - * cma_alloc() - allocate pages from contiguous area - * @cma: Contiguous memory region for which the allocation is performed. - * @count: Requested number of pages. - * @align: Requested alignment of pages (in PAGE_SIZE order). - * @no_warn: Avoid printing message about failed allocation - * - * This function allocates part of contiguous memory on specific - * contiguous memory area. - */ -struct page *cma_alloc(struct cma *cma, unsigned long count, - unsigned int align, bool no_warn) +static struct page *__cma_alloc(struct cma *cma, unsigned long count, + unsigned int align, gfp_t gfp) { unsigned long mask, offset; unsigned long pfn = -1; @@ -463,8 +453,7 @@ struct page *cma_alloc(struct cma *cma, unsigned long count, pfn = cma->base_pfn + (bitmap_no << cma->order_per_bit); mutex_lock(&cma_mutex); - ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA, - GFP_KERNEL | (no_warn ? __GFP_NOWARN : 0)); + ret = alloc_contig_range(pfn, pfn + count, MIGRATE_CMA, gfp); mutex_unlock(&cma_mutex); if (ret == 0) { page = pfn_to_page(pfn); @@ -494,7 +483,7 @@ struct page *cma_alloc(struct cma *cma, unsigned long count, page_kasan_tag_reset(nth_page(page, i)); } - if (ret && !no_warn) { + if (ret && !(gfp & __GFP_NOWARN)) { pr_err_ratelimited("%s: %s: alloc failed, req-size: %lu pages, ret: %d\n", __func__, cma->name, count, ret); cma_debug_show_areas(cma); @@ -513,6 +502,34 @@ struct page *cma_alloc(struct cma *cma, unsigned long count, return page; } +/** + * cma_alloc() - allocate pages from contiguous area + * @cma: Contiguous memory region for which the allocation is performed. + * @count: Requested number of pages. + * @align: Requested alignment of pages (in PAGE_SIZE order). + * @no_warn: Avoid printing message about failed allocation + * + * This function allocates part of contiguous memory on specific + * contiguous memory area. + */ +struct page *cma_alloc(struct cma *cma, unsigned long count, + unsigned int align, bool no_warn) +{ + return __cma_alloc(cma, count, align, GFP_KERNEL | (no_warn ? __GFP_NOWARN : 0)); +} + +struct folio *cma_alloc_folio(struct cma *cma, int order, gfp_t gfp) +{ + struct page *page; + + if (WARN_ON(!order || !(gfp | __GFP_COMP))) + return NULL; + + page = __cma_alloc(cma, 1 << order, order, gfp); + + return page ? page_folio(page) : NULL; +} + bool cma_pages_valid(struct cma *cma, const struct page *pages, unsigned long count) { @@ -564,6 +581,14 @@ bool cma_release(struct cma *cma, const struct page *pages, return true; } +bool cma_free_folio(struct cma *cma, const struct folio *folio) +{ + if (WARN_ON(!folio_test_large(folio))) + return false; + + return cma_release(cma, &folio->page, folio_nr_pages(folio)); +} + int cma_for_each_area(int (*it)(struct cma *cma, void *data), void *data) { int i;