From patchwork Mon Nov 22 15:32:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Hocko X-Patchwork-Id: 12632141 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFF09C433EF for ; Mon, 22 Nov 2021 15:33:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 233B86B0072; Mon, 22 Nov 2021 10:33:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1E2E56B0073; Mon, 22 Nov 2021 10:33:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0ABF26B0074; Mon, 22 Nov 2021 10:33:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0068.hostedemail.com [216.40.44.68]) by kanga.kvack.org (Postfix) with ESMTP id F10366B0072 for ; Mon, 22 Nov 2021 10:33:00 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id B996789D2B for ; Mon, 22 Nov 2021 15:32:50 +0000 (UTC) X-FDA: 78836958942.29.BDE90A8 Received: from mail-ed1-f41.google.com (mail-ed1-f41.google.com [209.85.208.41]) by imf26.hostedemail.com (Postfix) with ESMTP id 8D1FE20019EC for ; Mon, 22 Nov 2021 15:32:49 +0000 (UTC) Received: by mail-ed1-f41.google.com with SMTP id y13so78559364edd.13 for ; Mon, 22 Nov 2021 07:32:50 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zzYIrpFjBHpBF/zYBCBZI1LNzRlLubcW/OTa711WwFs=; b=d3XYUDhaPU562QsVwa1kE08hqxbEndmQY1lAysMgccfdAcxnrgVNh/z9VkrrIwyvGI 9V4e5jLs67wKCOf9UPQ2xI1GCwj6XCPgMnNA7nT4zvZn9vHcU4nRTBrrqpdnrwaZ4eV4 gChc/kOz6TIi2lEV7Qo7bAlp7P5lBhZFPYKF1alewWpXh3lHgh8uMSKV7uFLEwni6E1b /HKWnWvqngNlmBecXBRbmjReyCd1EE84qwWt2k4pN1gS5dK5FoPYuwvgQQA9sJPJTnWI ODU40RK/hik/3pubZTpdZ3ZwqWsh58F2svVnqPa+KgeBC5I8dvIm3b88/VzrE9m8zmxc otww== X-Gm-Message-State: AOAM530oaX9M9hTsstSblH1KIJ6lXpyAD9tCeGpPZRao3Kpctl1oajR6 h+YHtPBBsfM4pmggys20xfk= X-Google-Smtp-Source: ABdhPJzdKoN82zhhsiyJJx1JG38ywi+mBrmuy8XTeco3UnBKVtMBJkCdWQ8DQwOdPSfVftlXDckg1g== X-Received: by 2002:a17:907:3f18:: with SMTP id hq24mr41593437ejc.506.1637595166854; Mon, 22 Nov 2021 07:32:46 -0800 (PST) Received: from localhost.localdomain (ip-85-160-4-65.eurotel.cz. [85.160.4.65]) by smtp.gmail.com with ESMTPSA id q7sm4247757edr.9.2021.11.22.07.32.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Nov 2021 07:32:46 -0800 (PST) From: Michal Hocko To: Andrew Morton Cc: Dave Chinner , Neil Brown , Christoph Hellwig , Uladzislau Rezki , , , LKML , Ilya Dryomov , Jeff Layton , Michal Hocko Subject: [PATCH v2 1/4] mm/vmalloc: alloc GFP_NO{FS,IO} for vmalloc Date: Mon, 22 Nov 2021 16:32:30 +0100 Message-Id: <20211122153233.9924-2-mhocko@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211122153233.9924-1-mhocko@kernel.org> References: <20211122153233.9924-1-mhocko@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 8D1FE20019EC X-Stat-Signature: 6msjgfwo84jrm8jn1jawqc8noomkoy77 Authentication-Results: imf26.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=kernel.org (policy=none); spf=pass (imf26.hostedemail.com: domain of mstsxfx@gmail.com designates 209.85.208.41 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com X-Rspamd-Server: rspam02 X-HE-Tag: 1637595169-219962 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Michal Hocko vmalloc historically hasn't supported GFP_NO{FS,IO} requests because page table allocations do not support externally provided gfp mask and performed GFP_KERNEL like allocations. Since few years we have scope (memalloc_no{fs,io}_{save,restore}) APIs to enforce NOFS and NOIO constrains implicitly to all allocators within the scope. There was a hope that those scopes would be defined on a higher level when the reclaim recursion boundary starts/stops (e.g. when a lock required during the memory reclaim is required etc.). It seems that not all NOFS/NOIO users have adopted this approach and instead they have taken a workaround approach to wrap a single [k]vmalloc allocation by a scope API. These workarounds do not serve the purpose of a better reclaim recursion documentation and reduction of explicit GFP_NO{FS,IO} usege so let's just provide them with the semantic they are asking for without a need for workarounds. Add support for GFP_NOFS and GFP_NOIO to vmalloc directly. All internal allocations already comply with the given gfp_mask. The only current exception is vmap_pages_range which maps kernel page tables. Infer the proper scope API based on the given gfp mask. Signed-off-by: Michal Hocko Reviewed-by: Uladzislau Rezki (Sony) Acked-by: Vlastimil Babka --- mm/vmalloc.c | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index d2a00ad4e1dd..17ca7001de1f 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2926,6 +2926,8 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, unsigned long array_size; unsigned int nr_small_pages = size >> PAGE_SHIFT; unsigned int page_order; + unsigned int flags; + int ret; array_size = (unsigned long)nr_small_pages * sizeof(struct page *); gfp_mask |= __GFP_NOWARN; @@ -2967,8 +2969,24 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, goto fail; } - if (vmap_pages_range(addr, addr + size, prot, area->pages, - page_shift) < 0) { + /* + * page tables allocations ignore external gfp mask, enforce it + * by the scope API + */ + if ((gfp_mask & (__GFP_FS | __GFP_IO)) == __GFP_IO) + flags = memalloc_nofs_save(); + else if ((gfp_mask & (__GFP_FS | __GFP_IO)) == 0) + flags = memalloc_noio_save(); + + ret = vmap_pages_range(addr, addr + size, prot, area->pages, + page_shift); + + if ((gfp_mask & (__GFP_FS | __GFP_IO)) == __GFP_IO) + memalloc_nofs_restore(flags); + else if ((gfp_mask & (__GFP_FS | __GFP_IO)) == 0) + memalloc_noio_restore(flags); + + if (ret < 0) { warn_alloc(orig_gfp_mask, NULL, "vmalloc error: size %lu, failed to map pages", area->nr_pages * PAGE_SIZE); From patchwork Mon Nov 22 15:32:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Hocko X-Patchwork-Id: 12632143 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D69DC433EF for ; Mon, 22 Nov 2021 15:34:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B0FBD6B0073; Mon, 22 Nov 2021 10:33:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id ABCAC6B0074; Mon, 22 Nov 2021 10:33:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 983E26B0075; Mon, 22 Nov 2021 10:33:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0162.hostedemail.com [216.40.44.162]) by kanga.kvack.org (Postfix) with ESMTP id 8AEB66B0073 for ; Mon, 22 Nov 2021 10:33:01 -0500 (EST) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 55CFE81FA9B1 for ; Mon, 22 Nov 2021 15:32:51 +0000 (UTC) X-FDA: 78836958900.12.37AEA80 Received: from mail-ed1-f42.google.com (mail-ed1-f42.google.com [209.85.208.42]) by imf21.hostedemail.com (Postfix) with ESMTP id 4012ED0369F5 for ; Mon, 22 Nov 2021 15:32:48 +0000 (UTC) Received: by mail-ed1-f42.google.com with SMTP id y12so78689197eda.12 for ; Mon, 22 Nov 2021 07:32:50 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XhNo8C5VhREj9v7bG8TFdKTfEqQRTCL9W5KEynh9aDo=; b=Med7aO7o7aGan265Va4x5nwST5nHYX7CWdcdbcVQMOCK7Mn2sgegDbX6k7HTeYtfBq aeUWijaIx4QEKLWS+V2h6v3FNG55WpppMcAcECYqT3HL7vCQ1zt2gHLzwbwn4M/TyVWc CQuww9NpwqSMpDkOa53odHvYgHz0HNRNNQrlyADTG/hSX1e7jVDAj6W3CP6gG56giMN/ w62Dxc9lNFK+NvM3Z+s0ydWc3VCsumbepmKznRBqhlQ8/tsiVRDjmdyPHTmQTurVn1CB PQdRvQs/k9gvSX/gC7HclSO6LPyf9qacHsCrhdl+uEScnv3GNtkgv3xlyUxvFv/lYRFT nB/w== X-Gm-Message-State: AOAM531OCvG/bSSBgKs3eIaRXpcMZahkNTEkc/NSjfj9w0Ufo3s3Ykq8 QUdXCbxU/FFBoIjNaK0R51w= X-Google-Smtp-Source: ABdhPJxBFqIXl30IalHLlxGPJUoekpcMvT3WF//HONPA1J1pzrEdOxtNc1nx/4+x08fBLVojY8iYjg== X-Received: by 2002:a17:907:9802:: with SMTP id ji2mr39237198ejc.352.1637595168343; Mon, 22 Nov 2021 07:32:48 -0800 (PST) Received: from localhost.localdomain (ip-85-160-4-65.eurotel.cz. [85.160.4.65]) by smtp.gmail.com with ESMTPSA id q7sm4247757edr.9.2021.11.22.07.32.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Nov 2021 07:32:47 -0800 (PST) From: Michal Hocko To: Andrew Morton Cc: Dave Chinner , Neil Brown , Christoph Hellwig , Uladzislau Rezki , , , LKML , Ilya Dryomov , Jeff Layton , Michal Hocko Subject: [PATCH v2 2/4] mm/vmalloc: add support for __GFP_NOFAIL Date: Mon, 22 Nov 2021 16:32:31 +0100 Message-Id: <20211122153233.9924-3-mhocko@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211122153233.9924-1-mhocko@kernel.org> References: <20211122153233.9924-1-mhocko@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 4012ED0369F5 X-Stat-Signature: 5td4ndf83yppdnqihm9jde31smqxen79 Authentication-Results: imf21.hostedemail.com; dkim=none; spf=pass (imf21.hostedemail.com: domain of mstsxfx@gmail.com designates 209.85.208.42 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=kernel.org (policy=none) X-HE-Tag: 1637595168-437219 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Michal Hocko Dave Chinner has mentioned that some of the xfs code would benefit from kvmalloc support for __GFP_NOFAIL because they have allocations that cannot fail and they do not fit into a single page. The large part of the vmalloc implementation already complies with the given gfp flags so there is no work for those to be done. The area and page table allocations are an exception to that. Implement a retry loop for those. Add a short sleep before retrying. 1 jiffy is a completely random timeout. Ideally the retry would wait for an explicit event - e.g. a change to the vmalloc space change if the failure was caused by the space fragmentation or depletion. But there are multiple different reasons to retry and this could become much more complex. Keep the retry simple for now and just sleep to prevent from hogging CPUs. Signed-off-by: Michal Hocko Acked-by: Vlastimil Babka --- mm/vmalloc.c | 22 +++++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 17ca7001de1f..b6aed4f94a85 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2844,6 +2844,8 @@ vm_area_alloc_pages(gfp_t gfp, int nid, * more permissive. */ if (!order) { + gfp_t bulk_gfp = gfp & ~__GFP_NOFAIL; + while (nr_allocated < nr_pages) { unsigned int nr, nr_pages_request; @@ -2861,12 +2863,12 @@ vm_area_alloc_pages(gfp_t gfp, int nid, * but mempolcy want to alloc memory by interleaving. */ if (IS_ENABLED(CONFIG_NUMA) && nid == NUMA_NO_NODE) - nr = alloc_pages_bulk_array_mempolicy(gfp, + nr = alloc_pages_bulk_array_mempolicy(bulk_gfp, nr_pages_request, pages + nr_allocated); else - nr = alloc_pages_bulk_array_node(gfp, nid, + nr = alloc_pages_bulk_array_node(bulk_gfp, nid, nr_pages_request, pages + nr_allocated); @@ -2921,6 +2923,7 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, { const gfp_t nested_gfp = (gfp_mask & GFP_RECLAIM_MASK) | __GFP_ZERO; const gfp_t orig_gfp_mask = gfp_mask; + bool nofail = gfp_mask & __GFP_NOFAIL; unsigned long addr = (unsigned long)area->addr; unsigned long size = get_vm_area_size(area); unsigned long array_size; @@ -2978,8 +2981,12 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, else if ((gfp_mask & (__GFP_FS | __GFP_IO)) == 0) flags = memalloc_noio_save(); - ret = vmap_pages_range(addr, addr + size, prot, area->pages, + do { + ret = vmap_pages_range(addr, addr + size, prot, area->pages, page_shift); + if (nofail && (ret < 0)) + schedule_timeout_uninterruptible(1); + } while (nofail && (ret < 0)); if ((gfp_mask & (__GFP_FS | __GFP_IO)) == __GFP_IO) memalloc_nofs_restore(flags); @@ -3074,9 +3081,14 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align, VM_UNINITIALIZED | vm_flags, start, end, node, gfp_mask, caller); if (!area) { + bool nofail = gfp_mask & __GFP_NOFAIL; warn_alloc(gfp_mask, NULL, - "vmalloc error: size %lu, vm_struct allocation failed", - real_size); + "vmalloc error: size %lu, vm_struct allocation failed%s", + real_size, (nofail) ? ". Retrying." : ""); + if (nofail) { + schedule_timeout_uninterruptible(1); + goto again; + } goto fail; } From patchwork Mon Nov 22 15:32:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Hocko X-Patchwork-Id: 12632147 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CB2DC433F5 for ; Mon, 22 Nov 2021 15:35:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7B3046B0075; Mon, 22 Nov 2021 10:33:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 760C26B0078; Mon, 22 Nov 2021 10:33:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 650056B007B; Mon, 22 Nov 2021 10:33:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0135.hostedemail.com [216.40.44.135]) by kanga.kvack.org (Postfix) with ESMTP id 567BF6B0075 for ; Mon, 22 Nov 2021 10:33:05 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 0F99418017076 for ; Mon, 22 Nov 2021 15:32:55 +0000 (UTC) X-FDA: 78836959110.03.6159117 Received: from mail-ed1-f42.google.com (mail-ed1-f42.google.com [209.85.208.42]) by imf10.hostedemail.com (Postfix) with ESMTP id 47DF26001A99 for ; Mon, 22 Nov 2021 15:32:52 +0000 (UTC) Received: by mail-ed1-f42.google.com with SMTP id x15so78423481edv.1 for ; Mon, 22 Nov 2021 07:32:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=k8QBm+j0TklA89F5YujdZFJMji+n7PGQ6QO4Nu6Diks=; b=DcURU2tlqlNORGhgCeGyJ/WUHLO5M03KrK69i/d6IHWQqKF+bLnSksLUET6Dit0xYB lSYFFI/L6SL6IcsJ75RAVFMI4O9+WAyMKWSblca6fzhWv6F/FDfykj05YWrWAVwBtNUL ELoYECoCiG67zza+jBqYkb+vqnh37XPSCSKRuuaXBid7ZNZIKanHClQkf6cVd9bDwE1s K7Fl8XoWnCaLO49dRyBZ1sVqsyDI830eaARKOocdr9+fyuik1LEFMfMBUcvJWDD+Ypj0 BuaMy+1MITEhgo6UZ2Epscw7z+wg5SaJpIT2Ci94xrbbJfA/4kHWbFLqrYDvxN8Hs4sV 1/gA== X-Gm-Message-State: AOAM531aYYpxRbMw2Dxx89dcNyGFjJnC1kN7G4cHHEF8DNiEM1VKshcn +svezY9Wv4PRkiYkqLThwZY= X-Google-Smtp-Source: ABdhPJyrZ59goBTOtOeq5ZN98qlki8G/7NxPUJ41xf8jnZWrURoXBrw90PFMoBdBJ2nYjbq8b8unMw== X-Received: by 2002:a17:907:168f:: with SMTP id hc15mr42177530ejc.115.1637595169842; Mon, 22 Nov 2021 07:32:49 -0800 (PST) Received: from localhost.localdomain (ip-85-160-4-65.eurotel.cz. [85.160.4.65]) by smtp.gmail.com with ESMTPSA id q7sm4247757edr.9.2021.11.22.07.32.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Nov 2021 07:32:49 -0800 (PST) From: Michal Hocko To: Andrew Morton Cc: Dave Chinner , Neil Brown , Christoph Hellwig , Uladzislau Rezki , , , LKML , Ilya Dryomov , Jeff Layton , Michal Hocko Subject: [PATCH v2 3/4] mm/vmalloc: be more explicit about supported gfp flags. Date: Mon, 22 Nov 2021 16:32:32 +0100 Message-Id: <20211122153233.9924-4-mhocko@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211122153233.9924-1-mhocko@kernel.org> References: <20211122153233.9924-1-mhocko@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 47DF26001A99 Authentication-Results: imf10.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=kernel.org (policy=none); spf=pass (imf10.hostedemail.com: domain of mstsxfx@gmail.com designates 209.85.208.42 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com X-Stat-Signature: dggouy49txn4egm9h56rcjjxpacprxj9 X-HE-Tag: 1637595172-231220 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Michal Hocko b7d90e7a5ea8 ("mm/vmalloc: be more explicit about supported gfp flags") has been merged prematurely without the rest of the series and without addressed review feedback from Neil. Fix that up now. Only wording is changed slightly. Signed-off-by: Michal Hocko Reviewed-by: Uladzislau Rezki (Sony) Acked-by: Vlastimil Babka --- mm/vmalloc.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index b6aed4f94a85..b1c115ec13be 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -3021,12 +3021,14 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, * * Allocate enough pages to cover @size from the page level * allocator with @gfp_mask flags. Please note that the full set of gfp - * flags are not supported. GFP_KERNEL would be a preferred allocation mode - * but GFP_NOFS and GFP_NOIO are supported as well. Zone modifiers are not - * supported. From the reclaim modifiers__GFP_DIRECT_RECLAIM is required (aka - * GFP_NOWAIT is not supported) and only __GFP_NOFAIL is supported (aka - * __GFP_NORETRY and __GFP_RETRY_MAYFAIL are not supported). - * __GFP_NOWARN can be used to suppress error messages about failures. + * flags are not supported. GFP_KERNEL, GFP_NOFS and GFP_NOIO are all + * supported. + * Zone modifiers are not supported. From the reclaim modifiers + * __GFP_DIRECT_RECLAIM is required (aka GFP_NOWAIT is not supported) + * and only __GFP_NOFAIL is supported (i.e. __GFP_NORETRY and + * __GFP_RETRY_MAYFAIL are not supported). + * + * __GFP_NOWARN can be used to suppress failures messages. * * Map them into contiguous kernel virtual space, using a pagetable * protection of @prot. From patchwork Mon Nov 22 15:32:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Hocko X-Patchwork-Id: 12632145 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99C22C433F5 for ; Mon, 22 Nov 2021 15:34:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 20CE46B0074; Mon, 22 Nov 2021 10:33:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1BB976B0075; Mon, 22 Nov 2021 10:33:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 05EA26B0078; Mon, 22 Nov 2021 10:33:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0035.hostedemail.com [216.40.44.35]) by kanga.kvack.org (Postfix) with ESMTP id EBCDE6B0074 for ; Mon, 22 Nov 2021 10:33:04 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id AD1991808A9C2 for ; Mon, 22 Nov 2021 15:32:54 +0000 (UTC) X-FDA: 78836959320.05.8199210 Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) by imf25.hostedemail.com (Postfix) with ESMTP id 18930B000183 for ; Mon, 22 Nov 2021 15:32:51 +0000 (UTC) Received: by mail-ed1-f43.google.com with SMTP id y13so78560456edd.13 for ; Mon, 22 Nov 2021 07:32:54 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Zo6rdv8VWy1CbkCH3mCDWH3/iz/FJl3/IoCVBK7j9hk=; b=FXzIFcgdavB8kzdP6YVRmiZbRTimkTAPXMiyennXuC1KI0Vca9S1zE79AI8RObl2Fh 6dmzn4DStM/vJLV/3DdhscZXyxHv/X8h/TXzziNcH4YaNvO6YwcqhY/ydRpulaN0hR9d /yZIY6H5AsJ+cOhMmjb6glrBq4qErDkKIs5e8J9pRnegG0j/LN0aYxa749rRzqVXeyDn Lde5SH2YCNO16VB982Gk+tmDvr72eaITrBisj6aydKhMLnSz3ihj1muUH4vEaZ42YPsU gG0ntpC9ghUlEcxYH7I0Nq2v6LlPO08L+rzOq4yypSyV8yXb32YtQ3wHSMO/mDqpwlbh MZTQ== X-Gm-Message-State: AOAM533QM/OXm7YsEhAeDsvFKfyE7XiDsN5lSYVt3Vj/NBHg0JtDPvaE +KYJ39FIRo+L46ATIkZjV24= X-Google-Smtp-Source: ABdhPJwWaL8NNNJbUa6B7XJSuvfszHDZN1AlX5ClompjS+pcHwuTwksG4y4Lz+8Js376OffflN9Pnw== X-Received: by 2002:a05:6402:908:: with SMTP id g8mr61682534edz.59.1637595171483; Mon, 22 Nov 2021 07:32:51 -0800 (PST) Received: from localhost.localdomain (ip-85-160-4-65.eurotel.cz. [85.160.4.65]) by smtp.gmail.com with ESMTPSA id q7sm4247757edr.9.2021.11.22.07.32.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Nov 2021 07:32:50 -0800 (PST) From: Michal Hocko To: Andrew Morton Cc: Dave Chinner , Neil Brown , Christoph Hellwig , Uladzislau Rezki , , , LKML , Ilya Dryomov , Jeff Layton , Michal Hocko Subject: [PATCH v2 4/4] mm: allow !GFP_KERNEL allocations for kvmalloc Date: Mon, 22 Nov 2021 16:32:33 +0100 Message-Id: <20211122153233.9924-5-mhocko@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211122153233.9924-1-mhocko@kernel.org> References: <20211122153233.9924-1-mhocko@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 18930B000183 X-Stat-Signature: k371rqa65n6c1rgzjamj913dd6jt1zwd Authentication-Results: imf25.hostedemail.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=kernel.org (policy=none); spf=pass (imf25.hostedemail.com: domain of mstsxfx@gmail.com designates 209.85.208.43 as permitted sender) smtp.mailfrom=mstsxfx@gmail.com X-HE-Tag: 1637595171-19861 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Michal Hocko A support for GFP_NO{FS,IO} and __GFP_NOFAIL has been implemented by previous patches so we can allow the support for kvmalloc. This will allow some external users to simplify or completely remove their helpers. GFP_NOWAIT semantic hasn't been supported so far but it hasn't been explicitly documented so let's add a note about that. ceph_kvmalloc is the first helper to be dropped and changed to kvmalloc. Signed-off-by: Michal Hocko Reviewed-by: Uladzislau Rezki (Sony) Reviewed-by: Uladzislau Rezki (Sony) Acked-by: Vlastimil Babka --- include/linux/ceph/libceph.h | 1 - mm/util.c | 15 ++++----------- net/ceph/buffer.c | 4 ++-- net/ceph/ceph_common.c | 27 --------------------------- net/ceph/crypto.c | 2 +- net/ceph/messenger.c | 2 +- net/ceph/messenger_v2.c | 2 +- net/ceph/osdmap.c | 12 ++++++------ 8 files changed, 15 insertions(+), 50 deletions(-) diff --git a/include/linux/ceph/libceph.h b/include/linux/ceph/libceph.h index 409d8c29bc4f..309acbcb5a8a 100644 --- a/include/linux/ceph/libceph.h +++ b/include/linux/ceph/libceph.h @@ -295,7 +295,6 @@ extern bool libceph_compatible(void *data); extern const char *ceph_msg_type_name(int type); extern int ceph_check_fsid(struct ceph_client *client, struct ceph_fsid *fsid); -extern void *ceph_kvmalloc(size_t size, gfp_t flags); struct fs_parameter; struct fc_log; diff --git a/mm/util.c b/mm/util.c index e58151a61255..7275f2829e3f 100644 --- a/mm/util.c +++ b/mm/util.c @@ -549,13 +549,10 @@ EXPORT_SYMBOL(vm_mmap); * Uses kmalloc to get the memory but if the allocation fails then falls back * to the vmalloc allocator. Use kvfree for freeing the memory. * - * Reclaim modifiers - __GFP_NORETRY and __GFP_NOFAIL are not supported. + * GFP_NOWAIT and GFP_ATOMIC are not supported, neither is the __GFP_NORETRY modifier. * __GFP_RETRY_MAYFAIL is supported, and it should be used only if kmalloc is * preferable to the vmalloc fallback, due to visible performance drawbacks. * - * Please note that any use of gfp flags outside of GFP_KERNEL is careful to not - * fall back to vmalloc. - * * Return: pointer to the allocated memory of %NULL in case of failure */ void *kvmalloc_node(size_t size, gfp_t flags, int node) @@ -563,13 +560,6 @@ void *kvmalloc_node(size_t size, gfp_t flags, int node) gfp_t kmalloc_flags = flags; void *ret; - /* - * vmalloc uses GFP_KERNEL for some internal allocations (e.g page tables) - * so the given set of flags has to be compatible. - */ - if ((flags & GFP_KERNEL) != GFP_KERNEL) - return kmalloc_node(size, flags, node); - /* * We want to attempt a large physically contiguous block first because * it is less likely to fragment multiple larger blocks and therefore @@ -582,6 +572,9 @@ void *kvmalloc_node(size_t size, gfp_t flags, int node) if (!(kmalloc_flags & __GFP_RETRY_MAYFAIL)) kmalloc_flags |= __GFP_NORETRY; + + /* nofail semantic is implemented by the vmalloc fallback */ + kmalloc_flags &= ~__GFP_NOFAIL; } ret = kmalloc_node(size, kmalloc_flags, node); diff --git a/net/ceph/buffer.c b/net/ceph/buffer.c index 5622763ad402..7e51f128045d 100644 --- a/net/ceph/buffer.c +++ b/net/ceph/buffer.c @@ -7,7 +7,7 @@ #include #include -#include /* for ceph_kvmalloc */ +#include /* for kvmalloc */ struct ceph_buffer *ceph_buffer_new(size_t len, gfp_t gfp) { @@ -17,7 +17,7 @@ struct ceph_buffer *ceph_buffer_new(size_t len, gfp_t gfp) if (!b) return NULL; - b->vec.iov_base = ceph_kvmalloc(len, gfp); + b->vec.iov_base = kvmalloc(len, gfp); if (!b->vec.iov_base) { kfree(b); return NULL; diff --git a/net/ceph/ceph_common.c b/net/ceph/ceph_common.c index 97d6ea763e32..9441b4a4912b 100644 --- a/net/ceph/ceph_common.c +++ b/net/ceph/ceph_common.c @@ -190,33 +190,6 @@ int ceph_compare_options(struct ceph_options *new_opt, } EXPORT_SYMBOL(ceph_compare_options); -/* - * kvmalloc() doesn't fall back to the vmalloc allocator unless flags are - * compatible with (a superset of) GFP_KERNEL. This is because while the - * actual pages are allocated with the specified flags, the page table pages - * are always allocated with GFP_KERNEL. - * - * ceph_kvmalloc() may be called with GFP_KERNEL, GFP_NOFS or GFP_NOIO. - */ -void *ceph_kvmalloc(size_t size, gfp_t flags) -{ - void *p; - - if ((flags & (__GFP_IO | __GFP_FS)) == (__GFP_IO | __GFP_FS)) { - p = kvmalloc(size, flags); - } else if ((flags & (__GFP_IO | __GFP_FS)) == __GFP_IO) { - unsigned int nofs_flag = memalloc_nofs_save(); - p = kvmalloc(size, GFP_KERNEL); - memalloc_nofs_restore(nofs_flag); - } else { - unsigned int noio_flag = memalloc_noio_save(); - p = kvmalloc(size, GFP_KERNEL); - memalloc_noio_restore(noio_flag); - } - - return p; -} - static int parse_fsid(const char *str, struct ceph_fsid *fsid) { int i = 0; diff --git a/net/ceph/crypto.c b/net/ceph/crypto.c index 92d89b331645..051d22c0e4ad 100644 --- a/net/ceph/crypto.c +++ b/net/ceph/crypto.c @@ -147,7 +147,7 @@ void ceph_crypto_key_destroy(struct ceph_crypto_key *key) static const u8 *aes_iv = (u8 *)CEPH_AES_IV; /* - * Should be used for buffers allocated with ceph_kvmalloc(). + * Should be used for buffers allocated with kvmalloc(). * Currently these are encrypt out-buffer (ceph_buffer) and decrypt * in-buffer (msg front). * diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c index 57d043b382ed..7b891be799d2 100644 --- a/net/ceph/messenger.c +++ b/net/ceph/messenger.c @@ -1920,7 +1920,7 @@ struct ceph_msg *ceph_msg_new2(int type, int front_len, int max_data_items, /* front */ if (front_len) { - m->front.iov_base = ceph_kvmalloc(front_len, flags); + m->front.iov_base = kvmalloc(front_len, flags); if (m->front.iov_base == NULL) { dout("ceph_msg_new can't allocate %d bytes\n", front_len); diff --git a/net/ceph/messenger_v2.c b/net/ceph/messenger_v2.c index cc40ce4e02fb..c4099b641b38 100644 --- a/net/ceph/messenger_v2.c +++ b/net/ceph/messenger_v2.c @@ -308,7 +308,7 @@ static void *alloc_conn_buf(struct ceph_connection *con, int len) if (WARN_ON(con->v2.conn_buf_cnt >= ARRAY_SIZE(con->v2.conn_bufs))) return NULL; - buf = ceph_kvmalloc(len, GFP_NOIO); + buf = kvmalloc(len, GFP_NOIO); if (!buf) return NULL; diff --git a/net/ceph/osdmap.c b/net/ceph/osdmap.c index 75b738083523..2823bb3cff55 100644 --- a/net/ceph/osdmap.c +++ b/net/ceph/osdmap.c @@ -980,7 +980,7 @@ static struct crush_work *alloc_workspace(const struct crush_map *c) work_size = crush_work_size(c, CEPH_PG_MAX_SIZE); dout("%s work_size %zu bytes\n", __func__, work_size); - work = ceph_kvmalloc(work_size, GFP_NOIO); + work = kvmalloc(work_size, GFP_NOIO); if (!work) return NULL; @@ -1190,9 +1190,9 @@ static int osdmap_set_max_osd(struct ceph_osdmap *map, u32 max) if (max == map->max_osd) return 0; - state = ceph_kvmalloc(array_size(max, sizeof(*state)), GFP_NOFS); - weight = ceph_kvmalloc(array_size(max, sizeof(*weight)), GFP_NOFS); - addr = ceph_kvmalloc(array_size(max, sizeof(*addr)), GFP_NOFS); + state = kvmalloc(array_size(max, sizeof(*state)), GFP_NOFS); + weight = kvmalloc(array_size(max, sizeof(*weight)), GFP_NOFS); + addr = kvmalloc(array_size(max, sizeof(*addr)), GFP_NOFS); if (!state || !weight || !addr) { kvfree(state); kvfree(weight); @@ -1222,7 +1222,7 @@ static int osdmap_set_max_osd(struct ceph_osdmap *map, u32 max) if (map->osd_primary_affinity) { u32 *affinity; - affinity = ceph_kvmalloc(array_size(max, sizeof(*affinity)), + affinity = kvmalloc(array_size(max, sizeof(*affinity)), GFP_NOFS); if (!affinity) return -ENOMEM; @@ -1503,7 +1503,7 @@ static int set_primary_affinity(struct ceph_osdmap *map, int osd, u32 aff) if (!map->osd_primary_affinity) { int i; - map->osd_primary_affinity = ceph_kvmalloc( + map->osd_primary_affinity = kvmalloc( array_size(map->max_osd, sizeof(*map->osd_primary_affinity)), GFP_NOFS); if (!map->osd_primary_affinity)