From patchwork Mon Dec 6 21:22:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12694985 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5C1DFC433F5 for ; Mon, 6 Dec 2021 21:29:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Dm7jCEFMe+dWjwZb/BNbx67wSjTmaO9Pb4HxJzAXkDE=; b=CTaJ5CKty+GaSQ A3M/R+QHpn7iyHqn6TRdjaRT/Chk4DjrmrtwGmzHQy+t9OHfFWYDXYugVOi1TJP5uaL5KhedCVrtt RWnJ9LtguL1hGMmDIlplKOedlqawzaX5y+qtm85KYgB994SHyV95vV7omj+TctmZckB75mmfz548+ HQwvnLq4JJpaK89Urg/jOSZ5Nvx9HHulwn/lHX399iFRZZdlKUL34kp2dcL6DWyme6vu72hBKgFqd n6yuo18xK3c0+z3OhikU2oQHX0cwMosyXDxNx3+K2qESXE6X5oh2Pr8d7ZU+W8vjJvPJNghdJMJqi WI0e4pBD450SZ2x3yUVw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLWq-005slS-IE; Mon, 06 Dec 2021 21:28:09 +0000 Received: from out0.migadu.com ([94.23.1.103]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLRz-005qda-F8 for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:23:09 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638825778; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4G8h4hgySaJey6iKxW5y4/UhksHPaOzWIIS8r9vmE5w=; b=AfFY/elQmMDAk5i5itHsMNQvHFXdQLT4ic7jzsv8UwzTS0VPaDerdhdSsaTey86OWa8i7v rw9v1VfdGxit77WtIsFyFS/H6ur2KAWsi7W9DNM2EwzygIJnLxpjW3DkIixiYnU1dYKIkx /uM2sicDdMBWIwGV8br6yDwyPIjvt1M= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 01/34] kasan, page_alloc: deduplicate should_skip_kasan_poison Date: Mon, 6 Dec 2021 22:22:05 +0100 Message-Id: <9644e588df352734bb5c166caac2d440052cd04f.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_132307_859502_065EA17A X-CRM114-Status: GOOD ( 15.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Currently, should_skip_kasan_poison() has two definitions: one for when CONFIG_DEFERRED_STRUCT_PAGE_INIT is enabled, one for when it's not. Instead of duplicating the checks, add a deferred_pages_enabled() helper and use it in a single should_skip_kasan_poison() definition. Also move should_skip_kasan_poison() closer to its caller and clarify all conditions in the comment. Signed-off-by: Andrey Konovalov --- mm/page_alloc.c | 55 +++++++++++++++++++++++++++++-------------------- 1 file changed, 33 insertions(+), 22 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index c5952749ad40..c99566a3b67e 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -375,25 +375,9 @@ int page_group_by_mobility_disabled __read_mostly; */ static DEFINE_STATIC_KEY_TRUE(deferred_pages); -/* - * Calling kasan_poison_pages() only after deferred memory initialization - * has completed. Poisoning pages during deferred memory init will greatly - * lengthen the process and cause problem in large memory systems as the - * deferred pages initialization is done with interrupt disabled. - * - * Assuming that there will be no reference to those newly initialized - * pages before they are ever allocated, this should have no effect on - * KASAN memory tracking as the poison will be properly inserted at page - * allocation time. The only corner case is when pages are allocated by - * on-demand allocation and then freed again before the deferred pages - * initialization is done, but this is not likely to happen. - */ -static inline bool should_skip_kasan_poison(struct page *page, fpi_t fpi_flags) +static inline bool deferred_pages_enabled(void) { - return static_branch_unlikely(&deferred_pages) || - (!IS_ENABLED(CONFIG_KASAN_GENERIC) && - (fpi_flags & FPI_SKIP_KASAN_POISON)) || - PageSkipKASanPoison(page); + return static_branch_unlikely(&deferred_pages); } /* Returns true if the struct page for the pfn is uninitialised */ @@ -444,11 +428,9 @@ defer_init(int nid, unsigned long pfn, unsigned long end_pfn) return false; } #else -static inline bool should_skip_kasan_poison(struct page *page, fpi_t fpi_flags) +static inline bool deferred_pages_enabled(void) { - return (!IS_ENABLED(CONFIG_KASAN_GENERIC) && - (fpi_flags & FPI_SKIP_KASAN_POISON)) || - PageSkipKASanPoison(page); + return false; } static inline bool early_page_uninitialised(unsigned long pfn) @@ -1258,6 +1240,35 @@ static int free_tail_pages_check(struct page *head_page, struct page *page) return ret; } +/* + * Skip KASAN memory poisoning when either: + * + * 1. Deferred memory initialization has not yet completed, + * see the explanation below. + * 2. Skipping poisoning is requested via FPI_SKIP_KASAN_POISON, + * see the comment next to it. + * 3. Skipping poisoning is requested via __GFP_SKIP_KASAN_POISON, + * see the comment next to it. + * + * Poisoning pages during deferred memory init will greatly lengthen the + * process and cause problem in large memory systems as the deferred pages + * initialization is done with interrupt disabled. + * + * Assuming that there will be no reference to those newly initialized + * pages before they are ever allocated, this should have no effect on + * KASAN memory tracking as the poison will be properly inserted at page + * allocation time. The only corner case is when pages are allocated by + * on-demand allocation and then freed again before the deferred pages + * initialization is done, but this is not likely to happen. + */ +static inline bool should_skip_kasan_poison(struct page *page, fpi_t fpi_flags) +{ + return deferred_pages_enabled() || + (!IS_ENABLED(CONFIG_KASAN_GENERIC) && + (fpi_flags & FPI_SKIP_KASAN_POISON)) || + PageSkipKASanPoison(page); +} + static void kernel_init_free_pages(struct page *page, int numpages, bool zero_tags) { int i; From patchwork Mon Dec 6 21:31:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12694987 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C5210C433F5 for ; Mon, 6 Dec 2021 21:36:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wyGvUK/45v5RC8NMkPV6hBAXFPUHeL+hjIBhNy6F3JE=; b=PRO8y4i/iNCv4R fVdAC2w76BxaeP+C52xwh6Pfg3ibtoDF7UO0xnCY8OOtIUZLK4cSbpElkBc5BM9g9W9+2MJ61K0/c AP2oP9nn95s/Kf9qs9eByV5YQfJgeEDNqFWsyOQ7Osi0/BkF2KB80xVSuOv78m6np8DjuLuhEhwUY FFZ0BkwvOb1RglmV7qVCNb2OoOmrZHzOoLyFqZ/6NJc8qV3MEmCXnbpm7sFkmwUmxwf1/icMsyfOr pwrRCjnOOfjl8FnbqAVDh70JW3OvgHwjzsh8S2MfOrQdto6kbGRBX38C/zjFB7r0gb7Z5201ovVY6 5ri5LPtBiCJMEf0A6x8g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLdF-005vWV-Dg; Mon, 06 Dec 2021 21:34:46 +0000 Received: from out1.migadu.com ([91.121.223.63]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLaM-005uGA-Vn for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:31:48 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638826304; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=A+HW6k6OAcUIQ5pcvvYHE04liTPVazbrvXn/E55v5k4=; b=G8KjRNXo5Y05KabVgmRYMgc1kapn6FpN2ZKrG2QD5hDJGEDn7Tgi7gprNzRm620OoRUlWh fysL0arSZUir4OcxIwEJmUp1ixpSkbukKCOGx4X9E3Z8uoPUERQfMpDZ3RPzVpGLPrf91S UBLlmOt/Dw/nA9x6H4xAIem9Q/qthz0= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 02/34] kasan, page_alloc: move tag_clear_highpage out of kernel_init_free_pages Date: Mon, 6 Dec 2021 22:31:07 +0100 Message-Id: <2ace94811bd1ce8c87519bf55bcc163c2a78d3cd.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_133147_218163_56F62010 X-CRM114-Status: GOOD ( 14.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Currently, kernel_init_free_pages() serves two purposes: it either only zeroes memory or zeroes both memory and memory tags via a different code path. As this function has only two callers, each using only one code path, this behaviour is confusing. This patch pulls the code that zeroes both memory and tags out of kernel_init_free_pages(). As a result of this change, the code in free_pages_prepare() starts to look complicated, but this is improved in the few following patches. Those improvements are not integrated into this patch to make diffs easier to read. This patch does no functional changes. Signed-off-by: Andrey Konovalov Reviewed-by: Alexander Potapenko --- mm/page_alloc.c | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index c99566a3b67e..3589333b5b77 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1269,16 +1269,10 @@ static inline bool should_skip_kasan_poison(struct page *page, fpi_t fpi_flags) PageSkipKASanPoison(page); } -static void kernel_init_free_pages(struct page *page, int numpages, bool zero_tags) +static void kernel_init_free_pages(struct page *page, int numpages) { int i; - if (zero_tags) { - for (i = 0; i < numpages; i++) - tag_clear_highpage(page + i); - return; - } - /* s390's use of memset() could override KASAN redzones. */ kasan_disable_current(); for (i = 0; i < numpages; i++) { @@ -1372,7 +1366,7 @@ static __always_inline bool free_pages_prepare(struct page *page, bool init = want_init_on_free(); if (init) - kernel_init_free_pages(page, 1 << order, false); + kernel_init_free_pages(page, 1 << order); if (!skip_kasan_poison) kasan_poison_pages(page, order, init); } @@ -2415,9 +2409,17 @@ inline void post_alloc_hook(struct page *page, unsigned int order, bool init = !want_init_on_free() && want_init_on_alloc(gfp_flags); kasan_unpoison_pages(page, order, init); - if (init) - kernel_init_free_pages(page, 1 << order, - gfp_flags & __GFP_ZEROTAGS); + + if (init) { + if (gfp_flags & __GFP_ZEROTAGS) { + int i; + + for (i = 0; i < 1 << order; i++) + tag_clear_highpage(page + i); + } else { + kernel_init_free_pages(page, 1 << order); + } + } } set_page_owner(page, order, gfp_flags); From patchwork Mon Dec 6 21:31:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12694988 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D7C41C433EF for ; Mon, 6 Dec 2021 21:38:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=UTA2sV8BoMgZyJ8AKKWbkNjaXRI2jH1BuKLBhsbg2go=; b=L4krPuWp40clUP ymQIkTCXKvRTdY+xsIaUuksoK2QeQyvI8APSwVRoAFD8fW+rNpfHPm8X2B3qdLXvSrHKEJcngQnYj wxwdyptE2fF3KzBvADB72LotFFwXgKXvDNbVwIXuskw7DghOWRDd1K7mFNvgQT0eFo90ByBfUc8ku oMfaQoawh8wC3G9UbFPdSAFRC4On4vOaFL0u3yCZUSAxze/WtZr5TPYxhFM+5hAWcn5DwnYfjA1rQ pQojiuyz5omdL0XEo5UkiEemSxbYQtjd/RvG6ADxV0aeFvA6X7qwLqwaihNbmQZ+qDKzdbY12Bmcu IrklZQmxHHXtOA3+AjLg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLey-005w8e-HA; Mon, 06 Dec 2021 21:36:33 +0000 Received: from out2.migadu.com ([188.165.223.204]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLas-005uUH-Bb for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:32:20 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638826336; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fp6vcpTXqTgKp/R/kxaUnxOXIVcgUxd+g1l9srxGKbc=; b=GN1CK5QbfzTx/xE0G/CNlI8C4e6zKnLDox2F8CbrEpwMPCr4NHn1/dUM/CuAm5pFAb/C/T bKeP3SuTUvUgbidEGigFowNVHQBCFynB0dvCPjwDMG2S0cp9k1S1wQtz3FpbbbD6gRXsZ3 /cgEFpzF3MGwtSgxedlO7r/qa/4rrk0= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 03/34] kasan, page_alloc: merge kasan_free_pages into free_pages_prepare Date: Mon, 6 Dec 2021 22:31:38 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_133218_737059_F67BADF0 X-CRM114-Status: GOOD ( 16.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Currently, the code responsible for initializing and poisoning memory in free_pages_prepare() is scattered across two locations: kasan_free_pages() for HW_TAGS KASAN and free_pages_prepare() itself. This is confusing. This and a few following patches combine the code from these two locations. Along the way, these patches also simplify the performed checks to make them easier to follow. This patch replaces the only caller of kasan_free_pages() with its implementation. As kasan_has_integrated_init() is only true when CONFIG_KASAN_HW_TAGS is enabled, moving the code does no functional changes. This patch is not useful by itself but makes the simplifications in the following patches easier to follow. Signed-off-by: Andrey Konovalov Reviewed-by: Alexander Potapenko --- include/linux/kasan.h | 8 -------- mm/kasan/common.c | 2 +- mm/kasan/hw_tags.c | 11 ----------- mm/page_alloc.c | 6 ++++-- 4 files changed, 5 insertions(+), 22 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index d8783b682669..89a43d8ae4fe 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -95,7 +95,6 @@ static inline bool kasan_hw_tags_enabled(void) } void kasan_alloc_pages(struct page *page, unsigned int order, gfp_t flags); -void kasan_free_pages(struct page *page, unsigned int order); #else /* CONFIG_KASAN_HW_TAGS */ @@ -116,13 +115,6 @@ static __always_inline void kasan_alloc_pages(struct page *page, BUILD_BUG(); } -static __always_inline void kasan_free_pages(struct page *page, - unsigned int order) -{ - /* Only available for integrated init. */ - BUILD_BUG(); -} - #endif /* CONFIG_KASAN_HW_TAGS */ static inline bool kasan_has_integrated_init(void) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 8428da2aaf17..66078cc1b4f0 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -387,7 +387,7 @@ static inline bool ____kasan_kfree_large(void *ptr, unsigned long ip) } /* - * The object will be poisoned by kasan_free_pages() or + * The object will be poisoned by kasan_poison_pages() or * kasan_slab_free_mempool(). */ diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 7355cb534e4f..0b8225add2e4 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -213,17 +213,6 @@ void kasan_alloc_pages(struct page *page, unsigned int order, gfp_t flags) } } -void kasan_free_pages(struct page *page, unsigned int order) -{ - /* - * This condition should match the one in free_pages_prepare() in - * page_alloc.c. - */ - bool init = want_init_on_free(); - - kasan_poison_pages(page, order, init); -} - #if IS_ENABLED(CONFIG_KASAN_KUNIT_TEST) void kasan_enable_tagging_sync(void) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 3589333b5b77..3f3ea41f8c64 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1353,15 +1353,17 @@ static __always_inline bool free_pages_prepare(struct page *page, /* * As memory initialization might be integrated into KASAN, - * kasan_free_pages and kernel_init_free_pages must be + * KASAN poisoning and memory initialization code must be * kept together to avoid discrepancies in behavior. * * With hardware tag-based KASAN, memory tags must be set before the * page becomes unavailable via debug_pagealloc or arch_free_page. */ if (kasan_has_integrated_init()) { + bool init = want_init_on_free(); + if (!skip_kasan_poison) - kasan_free_pages(page, order); + kasan_poison_pages(page, order, init); } else { bool init = want_init_on_free(); From patchwork Mon Dec 6 21:43:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12694989 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C0EC8C433FE for ; Mon, 6 Dec 2021 21:46:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=GGxJLOP2RRSPqxFXj/g1+GRChAoXJ6IhvF6aiAnsYE4=; b=LzAXdrgtwNdKgT sjF9sODHmotFGO/ztUuVU6mYaq7QOAQzFNPqRkeAYs2t4nQ/wOA6zg+wPCZLdRbZo4h6g/AT2cpYe Wte/8+rTRMqd9On6CzFstljTc6kHQ1Jb+VMPc4OWObmEZ/7DLUlcAWTN2O51Yxif5DY8sqecI4dVW xrnoW0sj4Avt4/K7zH7wFYBbZ89c6289mxpZ2/TdWetJj5PdT+epGuqnhZvQNuTBOZgEfnVZgFgg/ SVTRP8294pkJS5MnLv0B8yAqit2q/9xF0VLjqSEBlzOS3tLbGhs6Zy8/gyf9ZKqwm58IFEsD9qqBg ADuEU1BhK6GX+yJA9NAw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLmX-005yfq-36; Mon, 06 Dec 2021 21:44:21 +0000 Received: from out2.migadu.com ([188.165.223.204]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLmR-005yeK-2N for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:44:16 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827053; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Gob2lt41MqESGzJ3aS1cO67hIxM237cUiv6TAp2u4fg=; b=fuNxtXvqdyLDZCHw8EYl/QZ9yB8Q4AI0Kt5FffmAuW1+81Vj0jDxwm5fnYFdYCoT07n7ON I3T1j1F+9/05GbBX80spdrQS/gD4lF42utJaAAe33bRU6dH1jZJgfdnVMe6mT/11FKvwwK uKZNObSdyGR8puOSzvYLSUI25qqEP2o= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 04/34] kasan, page_alloc: simplify kasan_poison_pages call site Date: Mon, 6 Dec 2021 22:43:41 +0100 Message-Id: <73d7d82c2b5cf44cb429fbc7cc16479fb8776bbe.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134415_293010_3E5AD504 X-CRM114-Status: UNSURE ( 9.72 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Simplify the code around calling kasan_poison_pages() in free_pages_prepare(). This patch does no functional changes. Signed-off-by: Andrey Konovalov --- Changes v1->v2: - Don't reorder kasan_poison_pages() and free_pages_prepare(). --- mm/page_alloc.c | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 3f3ea41f8c64..15f76bc1fa3e 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1289,6 +1289,7 @@ static __always_inline bool free_pages_prepare(struct page *page, { int bad = 0; bool skip_kasan_poison = should_skip_kasan_poison(page, fpi_flags); + bool init = want_init_on_free(); VM_BUG_ON_PAGE(PageTail(page), page); @@ -1359,19 +1360,10 @@ static __always_inline bool free_pages_prepare(struct page *page, * With hardware tag-based KASAN, memory tags must be set before the * page becomes unavailable via debug_pagealloc or arch_free_page. */ - if (kasan_has_integrated_init()) { - bool init = want_init_on_free(); - - if (!skip_kasan_poison) - kasan_poison_pages(page, order, init); - } else { - bool init = want_init_on_free(); - - if (init) - kernel_init_free_pages(page, 1 << order); - if (!skip_kasan_poison) - kasan_poison_pages(page, order, init); - } + if (init && !kasan_has_integrated_init()) + kernel_init_free_pages(page, 1 << order); + if (!skip_kasan_poison) + kasan_poison_pages(page, order, init); /* * arch_free_page() can make the page's contents inaccessible. s390 From patchwork Mon Dec 6 21:43:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12694990 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 039C3C433F5 for ; Mon, 6 Dec 2021 21:46:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=b8Yc3L1lYAY+YEJ6rULjVhdc8fuRZ2Lm/OOKvsyySNU=; b=Ncz1vO+tcNukV/ TMLu30pUkR36cKMVhaccjXQ7ESaP1r4iVPVhVV5bDF0x+7l/CpxdCJonVNrz25zmhNyx8p2QI3uUP DOhu1Nn3uzRQgEkTfroKsOF8IWfJnGc5qyF2MaWTsqBvphoChG4NIO5bE2yvjXXotJLPUeiBzObU7 9BciuCePHdyd9zrYuJfTzMgYk7GGPkXubaoUQQEf8p8Ul5Kq9WJbTpToGVAS14rHmSRdiN4lhHL8A FMzRQuqtuvGWEyzJyDybT7hXKKn++ipAyEZsEZy43rvMckjn7b4LbVdY9JP7U7krc7N6gEIzW1ZCV N+bb7bPJeHvKc69tNMsQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLmy-005yoB-U6; Mon, 06 Dec 2021 21:44:50 +0000 Received: from out2.migadu.com ([188.165.223.204]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLmX-005ygH-K1 for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:44:23 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827060; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CDfDFpOAo922pZ0chQAMNz9tzdyiLXIp8K+CHBQuOYo=; b=gbVFChOrfx9TbOhvcPZkQHwrv+Btc9qpYHzIIr39UxSGbk4K6PLE6TQJhV4xwYqedXDtA1 eteMfFOiRbRw6/xRRY7CVQvzYoJOgTnaN2FZmwfaA1uo8R8M0hgv1WVY+xbvnlc0QnCDgV qxPAc3KLVfumtS7wA8rwFlkCTln8M8o= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 05/34] kasan, page_alloc: init memory of skipped pages on free Date: Mon, 6 Dec 2021 22:43:42 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134421_855601_B7BF7DDA X-CRM114-Status: GOOD ( 12.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Since commit 7a3b83537188 ("kasan: use separate (un)poison implementation for integrated init"), when all init, kasan_has_integrated_init(), and skip_kasan_poison are true, free_pages_prepare() doesn't initialize the page. This is wrong. Fix it by remembering whether kasan_poison_pages() performed initialization, and call kernel_init_free_pages() if it didn't. Reordering kasan_poison_pages() and kernel_init_free_pages() is OK, since kernel_init_free_pages() can handle poisoned memory. Fixes: 7a3b83537188 ("kasan: use separate (un)poison implementation for integrated init") Signed-off-by: Andrey Konovalov --- Changes v1->v2: - Reorder kasan_poison_pages() and free_pages_prepare() in this patch instead of doing it in the previous one. --- mm/page_alloc.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 15f76bc1fa3e..2ada09a58e4b 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1360,11 +1360,16 @@ static __always_inline bool free_pages_prepare(struct page *page, * With hardware tag-based KASAN, memory tags must be set before the * page becomes unavailable via debug_pagealloc or arch_free_page. */ - if (init && !kasan_has_integrated_init()) - kernel_init_free_pages(page, 1 << order); - if (!skip_kasan_poison) + if (!skip_kasan_poison) { kasan_poison_pages(page, order, init); + /* Memory is already initialized if KASAN did it internally. */ + if (kasan_has_integrated_init()) + init = false; + } + if (init) + kernel_init_free_pages(page, 1 << order); + /* * arch_free_page() can make the page's contents inaccessible. s390 * does this. So nothing which can access the page's contents should From patchwork Mon Dec 6 21:43:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12694991 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2D442C433EF for ; Mon, 6 Dec 2021 21:47:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=M8Bu3eH0rkVW7esGzCuq3paC3MXnAMEl7MagvS636xo=; b=QEuSB5KHo400YO PojbRn7L3hUm3KyYd1rEmPGl9gPnI7rfuxdbdr5DSIn/OK2jo7NB2FH/w3heBt8Z3PvFN0OUhOuIz mru2Po8HELP5QMiAoujgBzvVszNz69/L6szYowBBp0xP2uEMdFIDSwqQqBiOwIHAMypCVM2R1uDGJ ezEl5P0IXpY7Xyy5UxwXiaGyfli46UPX16IIflAMIoUkZgJsFXU736TuhJtL2NbfNd+Y8CbYLHDlG z6toLYel9ii3trBxEOED7QJNVkSaSQ0a1wcrLaAMhQj3PsEEm2yqhtWRLiJYzKqVDEj/yrH1x6TLw +1zZJ5ueuUtH6nM/z1zg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLnb-005z2H-Td; Mon, 06 Dec 2021 21:45:28 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLme-005yiM-IG for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:44:30 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827066; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6WD5Ks+OgrOC+xp6cA4qnOUMHkVFyvh2xcK78VT3o/c=; b=ib6i3xZYh+ycDn2reE9Lz0gCBlUPKg4VO0JfrXwqG289FVhy/LmQ9+futRSZCwfoLSQhn4 tX1X2Fd2VyUPqcrbAdnMbEPtreAZ+2YrYpV3HD/1p5VZ7lzq9lPZip3s6lP6gAyTgQUlbP kydbkIGXueoY91URQSwMuAGgJ3IqCnM= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 06/34] kasan: drop skip_kasan_poison variable in free_pages_prepare Date: Mon, 6 Dec 2021 22:43:43 +0100 Message-Id: <82bf83ec678d19cf086bb62d92f5fe113de6e8b7.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134428_811205_89F54FF6 X-CRM114-Status: GOOD ( 11.21 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov skip_kasan_poison is only used in a single place. Call should_skip_kasan_poison() directly for simplicity. Signed-off-by: Andrey Konovalov Suggested-by: Marco Elver --- Changes v1->v2: - Add this patch. --- mm/page_alloc.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 2ada09a58e4b..f70bfa63a374 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1288,7 +1288,6 @@ static __always_inline bool free_pages_prepare(struct page *page, unsigned int order, bool check_free, fpi_t fpi_flags) { int bad = 0; - bool skip_kasan_poison = should_skip_kasan_poison(page, fpi_flags); bool init = want_init_on_free(); VM_BUG_ON_PAGE(PageTail(page), page); @@ -1360,7 +1359,7 @@ static __always_inline bool free_pages_prepare(struct page *page, * With hardware tag-based KASAN, memory tags must be set before the * page becomes unavailable via debug_pagealloc or arch_free_page. */ - if (!skip_kasan_poison) { + if (!should_skip_kasan_poison(page, fpi_flags)) { kasan_poison_pages(page, order, init); /* Memory is already initialized if KASAN did it internally. */ From patchwork Mon Dec 6 21:43:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12694992 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 92699C433EF for ; Mon, 6 Dec 2021 21:47:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=gJUEXnslc4iN4vNTI8ALP21g/q1hDRO8kSWpysOxW8g=; b=xqNR3sHE6VXQZy 9aDXEElaKRhOfHsBak/EcXciS7SjRvWsV8XEGsy2Mar40WglpUsLfWTqn0zCKYW+/cnBxp/xQl+lV njXXIWorZTLH1expwSCr77G32GVdHgcZpNmEUrcwzROPdgpQCaOe+tRUzy/jSemqIT9bO9p+cJ9pK Hb2FFiE1BG20ao2Na6Fvq9uc+PrXYJ5VVCGz/r2FU79ueuwb63wDWE5qDBdae6cB0sjRtRicI3+gw lhZdPN1xHxwPJy2WaRuu1wwH9KQLgUFbCqsK55dW8F9TePMT85eUTdjb5+/gA61lJMkF49L3rUABX N2vxX+vs6zkxYVLk1BXw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLoV-005zO0-1G; Mon, 06 Dec 2021 21:46:24 +0000 Received: from out2.migadu.com ([188.165.223.204]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLmk-005ykv-O3 for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:44:36 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827073; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2VlRNVgjD8s3LiQBttrzG+uL14gJGV2zDjLV32J+vek=; b=TD3DS7kX7rNKsdKtOiQfPuRZoLtKBoyf7SJg3jpF4rxc0wxHBoJHLYeR2c6s/YbyY6uLog 5APt5A1zwI3YC58xavMrB9D7v31ne2TlVFuxLLSpZf+HaEztsc2oWCXxMaaVxulkDWL0aM J4XRIiAZyUo1nVCnDbXOAGcx88Hqa+4= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 07/34] mm: clarify __GFP_ZEROTAGS comment Date: Mon, 6 Dec 2021 22:43:44 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134434_971228_9674DDC0 X-CRM114-Status: GOOD ( 12.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov __GFP_ZEROTAGS is intended as an optimization: if memory is zeroed during allocation, it's possible to set memory tags at the same time with little performance impact. Clarify this intention of __GFP_ZEROTAGS in the comment. Signed-off-by: Andrey Konovalov --- include/linux/gfp.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index b976c4177299..dddd7597689f 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -232,8 +232,8 @@ struct vm_area_struct; * * %__GFP_ZERO returns a zeroed page on success. * - * %__GFP_ZEROTAGS returns a page with zeroed memory tags on success, if - * __GFP_ZERO is set. + * %__GFP_ZEROTAGS zeroes memory tags at allocation time if the memory itself + * is being zeroed (either via __GFP_ZERO or via init_on_alloc). * * %__GFP_SKIP_KASAN_POISON returns a page which does not need to be poisoned * on deallocation. Typically used for userspace pages. Currently only has an From patchwork Mon Dec 6 21:43:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12694993 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C5E04C433F5 for ; Mon, 6 Dec 2021 21:49:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Djp3QxHPNSuBAcGH+MPtVWLutDMqAIPy2X0ib4BXQlM=; b=ffZpLhCaAow3Oy nD7NhIrhGbeqISuzZiLbkjTOxWLWPUkqppQ6lhxOpKRmrB77woJBzyPUkWfJpJxrwAERjWRpshSBW rc/HtxUr4qdyG0lue+7KEKDYDPZv9x8Aqa/JFQSxLooxgVvuDI8jJL7mnxKoiGQsTQkcvphLF67oK ce/LtcKgjY/Kk2AobgcGsbLbNQqZea/bBMqJNwXH+xWUfGAYzuB4juQDdwHPtaBg7RptM2eDoV/SN fC6cTebleNEXOeT28R4mSz++TQCkb/QB3u1wDP1xKP5KBXfWRS2IdqElSgdLE0AFNqnqEIl77nf2E 0xQWe0AGRIZoSuQRZCwA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLpQ-005zpk-Sk; Mon, 06 Dec 2021 21:47:21 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLmp-005ymR-JG for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:44:41 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827078; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TniNgsonCV7N4x8CfBP962yTwJ3ZQbzv223ievd0EoQ=; b=qXzJ+ayG5mWJ+4fHBRfgQ7rHV1m43f11tVvZTtueBObKZIqO936tA6hNbt2mtDAPM+wGfR j41BvO29uhf5X8RLEXtUoa8WtCjvOUbq96/E2or9Xa/6m1wNQ27y1bYuYdYr2LbQ5UqiiA M26Ur3GKXcnb6+j9Dajdhp6czI+tZnI= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 08/34] kasan: only apply __GFP_ZEROTAGS when memory is zeroed Date: Mon, 6 Dec 2021 22:43:45 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134439_836158_F2908A3E X-CRM114-Status: GOOD ( 10.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov __GFP_ZEROTAGS should only be effective if memory is being zeroed. Currently, hardware tag-based KASAN violates this requirement. Fix by including an initialization check along with checking for __GFP_ZEROTAGS. Signed-off-by: Andrey Konovalov Reviewed-by: Alexander Potapenko --- mm/kasan/hw_tags.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 0b8225add2e4..c643740b8599 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -199,11 +199,12 @@ void kasan_alloc_pages(struct page *page, unsigned int order, gfp_t flags) * page_alloc.c. */ bool init = !want_init_on_free() && want_init_on_alloc(flags); + bool init_tags = init && (flags & __GFP_ZEROTAGS); if (flags & __GFP_SKIP_KASAN_POISON) SetPageSkipKASanPoison(page); - if (flags & __GFP_ZEROTAGS) { + if (init_tags) { int i; for (i = 0; i != 1 << order; ++i) From patchwork Mon Dec 6 21:43:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12694994 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2DE92C433EF for ; Mon, 6 Dec 2021 21:49:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=D6eWlnOsnNOPIMaSnOtdeY++jkmxLuH3FyoBGYzpDf4=; b=no7IRXP00pTU5I VrUfYLx7JiN3iuVdFjFheBnoiE1QiFW0cSEp7wXKTHWfkyXeaBH0oJTJ6ds8iW6sn2pu1/qwTQdIP Or8ofKamlq53ELuvsEjct67ys2PuQXwAiUzY1SSvLBa0lIofAvgHgXx9kbuJG0pUIxp+ZjibLvsS6 baDWtZl16piDFLO4FEXChb4B7wXj+REvQmpHsIO9ouVQafMdLmXP33eQEFkHZWbG7tQmFQRflfI0y /wGqKISTFiZCN3QCcE1bZtsil3U/GgfIrQZVXDwwmXw3E9OAInvmjiedmE0EFS3zg9rKeIP46WE6j AuqMP4qfNMB0J+LVuCFg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLqC-006084-6P; Mon, 06 Dec 2021 21:48:09 +0000 Received: from out2.migadu.com ([188.165.223.204]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLmu-005yo3-P6 for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:44:46 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827083; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=i2FffCVJttNjYU5fhxzifZEfN2PNcPmkAgEoBkLN39I=; b=ki8a4uqZiwX/o6uF82TlRSlizCDnifW77Fpu8VTqd908FDeI+i1+dbsBcCFIYSxQYQKSLq e1kCTsdesyz70eaaUS9SEwh9RnUJRBS29bdYj1y+wucVhuxQmVl5m0s6844MrKgsJ4pZQO Np3RYHeNsMKNqpoYYjN5JcmZPFdV9Co= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 09/34] kasan, page_alloc: refactor init checks in post_alloc_hook Date: Mon, 6 Dec 2021 22:43:46 +0100 Message-Id: <7445f15afeaeffd92956d7093ba6aab62781f637.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134445_018092_60FAC21A X-CRM114-Status: GOOD ( 11.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov This patch separates code for zeroing memory from the code clearing tags in post_alloc_hook(). This patch is not useful by itself but makes the simplifications in the following patches easier to follow. This patch does no functional changes. Signed-off-by: Andrey Konovalov --- mm/page_alloc.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index f70bfa63a374..507004a54f2f 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2405,19 +2405,21 @@ inline void post_alloc_hook(struct page *page, unsigned int order, kasan_alloc_pages(page, order, gfp_flags); } else { bool init = !want_init_on_free() && want_init_on_alloc(gfp_flags); + bool init_tags = init && (gfp_flags & __GFP_ZEROTAGS); kasan_unpoison_pages(page, order, init); - if (init) { - if (gfp_flags & __GFP_ZEROTAGS) { - int i; + if (init_tags) { + int i; - for (i = 0; i < 1 << order; i++) - tag_clear_highpage(page + i); - } else { - kernel_init_free_pages(page, 1 << order); - } + for (i = 0; i < 1 << order; i++) + tag_clear_highpage(page + i); + + init = false; } + + if (init) + kernel_init_free_pages(page, 1 << order); } set_page_owner(page, order, gfp_flags); From patchwork Mon Dec 6 21:43:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12694995 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 261E8C433EF for ; Mon, 6 Dec 2021 21:50:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=whPRlLk0aqDiF08mcYAOGvt/QZfqD5X0UvLVn7krcGI=; b=2laTqBC7vGauZY Z51Fa4KYm84lj8UPj4rzLxiRH48iXkiL0wE8KQ5XT4LBY2Rqz8GX/fCdjKjWBBna1nK0vL/jFfYTa GcD/dEDTlarZGVBwBSotATFndwV8AM8EkuZFReEgidGWOe4hHf67ojiOynMOUCvuBRa+mMhxlaAED CEU71MdW5tBejykZ+SgDB0saTWu++vOiKpv2PG4UZyV5oZLGO0Vo+w2mZBjGOPcyATT1TCz91zn2o /9yrf9QHzygLSLzzJmnNwHwBo7tb2vnv8l1Wrp/ZF4DGgjYJEZ4YzVd5E4MJfbWhWCyGpeTZY2h+f JbX5Z2kPfpg/38bqz58A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLqw-0060Pq-6y; Mon, 06 Dec 2021 21:48:55 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLmy-005ypK-Rg for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:44:51 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827087; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=l+PTHzJSTVa8536ReyOZGg3sJjywrvvVxDxQcr48qWw=; b=JcidHI7fB81ESIfrRwfCBhRIR0b35fCsh0g1jQ9O3eTzupVwmgfGpCTTWygZCz+1HLbpT3 rPuWOFmoZic4PJ9mHQ9CIpudL/fAqQQvWUvs5mxlWxESD7G6w115rhCH5vJ8WPnuQ6iJNO xKksA9RL8x5FQXqmV1/v5u+GhB1TqVw= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 10/34] kasan, page_alloc: merge kasan_alloc_pages into post_alloc_hook Date: Mon, 6 Dec 2021 22:43:47 +0100 Message-Id: <76981f93389088cc08d90801a9754346894bb3b9.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134449_283562_9B7A03A9 X-CRM114-Status: GOOD ( 20.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Currently, the code responsible for initializing and poisoning memory in post_alloc_hook() is scattered across two locations: kasan_alloc_pages() hook for HW_TAGS KASAN and post_alloc_hook() itself. This is confusing. This and a few following patches combine the code from these two locations. Along the way, these patches do a step-by-step restructure the many performed checks to make them easier to follow. This patch replaces the only caller of kasan_alloc_pages() with its implementation. As kasan_has_integrated_init() is only true when CONFIG_KASAN_HW_TAGS is enabled, moving the code does no functional changes. The patch also moves init and init_tags variables definitions out of kasan_has_integrated_init() clause in post_alloc_hook(), as they have the same values regardless of what the if condition evaluates to. This patch is not useful by itself but makes the simplifications in the following patches easier to follow. Signed-off-by: Andrey Konovalov --- include/linux/kasan.h | 9 --------- mm/kasan/common.c | 2 +- mm/kasan/hw_tags.c | 22 ---------------------- mm/page_alloc.c | 20 +++++++++++++++----- 4 files changed, 16 insertions(+), 37 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 89a43d8ae4fe..1031070be3f3 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -94,8 +94,6 @@ static inline bool kasan_hw_tags_enabled(void) return kasan_enabled(); } -void kasan_alloc_pages(struct page *page, unsigned int order, gfp_t flags); - #else /* CONFIG_KASAN_HW_TAGS */ static inline bool kasan_enabled(void) @@ -108,13 +106,6 @@ static inline bool kasan_hw_tags_enabled(void) return false; } -static __always_inline void kasan_alloc_pages(struct page *page, - unsigned int order, gfp_t flags) -{ - /* Only available for integrated init. */ - BUILD_BUG(); -} - #endif /* CONFIG_KASAN_HW_TAGS */ static inline bool kasan_has_integrated_init(void) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 66078cc1b4f0..d7168bfca61a 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -536,7 +536,7 @@ void * __must_check __kasan_kmalloc_large(const void *ptr, size_t size, return NULL; /* - * The object has already been unpoisoned by kasan_alloc_pages() for + * The object has already been unpoisoned by kasan_unpoison_pages() for * alloc_pages() or by kasan_krealloc() for krealloc(). */ diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index c643740b8599..76cf2b6229c7 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -192,28 +192,6 @@ void __init kasan_init_hw_tags(void) kasan_stack_collection_enabled() ? "on" : "off"); } -void kasan_alloc_pages(struct page *page, unsigned int order, gfp_t flags) -{ - /* - * This condition should match the one in post_alloc_hook() in - * page_alloc.c. - */ - bool init = !want_init_on_free() && want_init_on_alloc(flags); - bool init_tags = init && (flags & __GFP_ZEROTAGS); - - if (flags & __GFP_SKIP_KASAN_POISON) - SetPageSkipKASanPoison(page); - - if (init_tags) { - int i; - - for (i = 0; i != 1 << order; ++i) - tag_clear_highpage(page + i); - } else { - kasan_unpoison_pages(page, order, init); - } -} - #if IS_ENABLED(CONFIG_KASAN_KUNIT_TEST) void kasan_enable_tagging_sync(void) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 507004a54f2f..d33e0b0547be 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2383,6 +2383,9 @@ static bool check_new_pages(struct page *page, unsigned int order) inline void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flags) { + bool init = !want_init_on_free() && want_init_on_alloc(gfp_flags); + bool init_tags = init && (gfp_flags & __GFP_ZEROTAGS); + set_page_private(page, 0); set_page_refcounted(page); @@ -2398,15 +2401,22 @@ inline void post_alloc_hook(struct page *page, unsigned int order, /* * As memory initialization might be integrated into KASAN, - * kasan_alloc_pages and kernel_init_free_pages must be + * KASAN unpoisoning and memory initializion code must be * kept together to avoid discrepancies in behavior. */ if (kasan_has_integrated_init()) { - kasan_alloc_pages(page, order, gfp_flags); - } else { - bool init = !want_init_on_free() && want_init_on_alloc(gfp_flags); - bool init_tags = init && (gfp_flags & __GFP_ZEROTAGS); + if (gfp_flags & __GFP_SKIP_KASAN_POISON) + SetPageSkipKASanPoison(page); + + if (init_tags) { + int i; + for (i = 0; i != 1 << order; ++i) + tag_clear_highpage(page + i); + } else { + kasan_unpoison_pages(page, order, init); + } + } else { kasan_unpoison_pages(page, order, init); if (init_tags) { From patchwork Mon Dec 6 21:43:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12694996 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DA6C7C433EF for ; Mon, 6 Dec 2021 21:51:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=g+m1u9hLMXNq+UsRpXa9RmWHG4+oSfpST70rAZGnlkU=; b=UmJTAfTdPyz1G1 dM+dK5c0armDQ7KYiI0Sjht3KcuraSDbQ5+4EcarpysvKhsaMOs0/yTe+LaX6By3Qga2PNwYeUF5n CmgUIpan/xGN2WMoK+qlWc9zaN1mu70mxgGEJqoy/Ed40c6Xjo/qeVDwLU8EX62S0kNlFwxC7IPzD VaGKwDMGlB6LsHxrLEhqYIHyMj59w2lXbvvwunOjIR7n3WcByEqVviOq1svh2tg+GoXpGpa0m6sXm SbfmrJ1WLImMoIUpNKJreOpLIr4zIW68DYRH8X4FBCjDQGxCRMtwlG20G+3FYqDkOXomDy2YH7ZZZ KF9QXSFl2v3KadNwgpMw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLra-0060hm-Mt; Mon, 06 Dec 2021 21:49:35 +0000 Received: from out2.migadu.com ([188.165.223.204]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLnC-005yuI-9W for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:45:04 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827100; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ir/Gb9vyo5VTknMlksNZZRWU049BNvYKgeMRmdqw2Yw=; b=XpQsncOOxcG+6nrBl4/akzj9F0TaG5zHuqwPTROfRBfdqA3gnl7cCTF66gp5NNd5FY4oOM SCsrpzeNu7CVWNxoTU61TjuwuFEYJfADpRXcysY6EBkv63hV0/0Kun3qVA9G/XNX1Z2Fzx UgfLoBM9ztjAPWqZypQ/KRfHpuAX1eI= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 11/34] kasan, page_alloc: combine tag_clear_highpage calls in post_alloc_hook Date: Mon, 6 Dec 2021 22:43:48 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134502_563977_97D59FB0 X-CRM114-Status: GOOD ( 12.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov The patch moves tag_clear_highpage() loops out of the kasan_has_integrated_init() clause as a code simplification. This patch does no functional changes. Signed-off-by: Andrey Konovalov --- mm/page_alloc.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index d33e0b0547be..781b75563276 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2404,30 +2404,30 @@ inline void post_alloc_hook(struct page *page, unsigned int order, * KASAN unpoisoning and memory initializion code must be * kept together to avoid discrepancies in behavior. */ + + /* + * If memory tags should be zeroed (which happens only when memory + * should be initialized as well). + */ + if (init_tags) { + int i; + + /* Initialize both memory and tags. */ + for (i = 0; i != 1 << order; ++i) + tag_clear_highpage(page + i); + + /* Note that memory is already initialized by the loop above. */ + init = false; + } if (kasan_has_integrated_init()) { if (gfp_flags & __GFP_SKIP_KASAN_POISON) SetPageSkipKASanPoison(page); - if (init_tags) { - int i; - - for (i = 0; i != 1 << order; ++i) - tag_clear_highpage(page + i); - } else { + if (!init_tags) kasan_unpoison_pages(page, order, init); - } } else { kasan_unpoison_pages(page, order, init); - if (init_tags) { - int i; - - for (i = 0; i < 1 << order; i++) - tag_clear_highpage(page + i); - - init = false; - } - if (init) kernel_init_free_pages(page, 1 << order); } From patchwork Mon Dec 6 21:43:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12694997 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DC1DAC433F5 for ; Mon, 6 Dec 2021 21:52:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=UthS4gYYkkxJ6nC0ZQWwy0LbBiIkLQhHJg/rEuknZ5k=; b=D4A8ZMJmPpE/2d s7LHnlrcz0tQN9hdmUmksg9xWD3JNWGEbgVPzznYXPXorHmuWJ/qev8AnFPGSzibDuEojdJC5Zjt9 Ey+J/erkxTSeyq33eEQ/TGU3RJoTIl2JhRW9ZrJvvDu//SobgfbmC1YBoB4XmndyOPdOqtrUnYH/j Z5jvME9wHgIBeFJ4h6ZsCwTd3rlR8Mxmhy3FDhVFoPZVKGztg0iJsp8npEIYpoL6Eykpt7NBJAYWl YaH6ybv7TF4subnNurCvS4wU3JdcjwdCjiAuyDOG9tfL0j3OW/eMVBOcdwSqP16ooDKhHYPhEUVrV 7KV6wsYGNYXqLMGvUVkA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLsO-00616k-AP; Mon, 06 Dec 2021 21:50:24 +0000 Received: from out2.migadu.com ([188.165.223.204]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLnM-005yxT-UR for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:45:14 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827111; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sMKllahn3TRwqEUvrph4wKLGXjFwUnG4mJTUv85O/gQ=; b=Ae5GuST6QVTbymZj0jmlxZYedwgab+xmAmhlOD5ni8ahgYTa97idVbc4zdwmHFi4iNFCH1 1gt1Kbd+C9alAZ5RqNSLVaA3bPrANzV9eeaOKHwfks7pGjK+b+/ezp53ix1YKJpFmtG9j9 YetXwC5pomqsxmLWljNCiFfdWybX0ms= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 12/34] kasan, page_alloc: move SetPageSkipKASanPoison in post_alloc_hook Date: Mon, 6 Dec 2021 22:43:49 +0100 Message-Id: <76d3972363dd96b33e6af31ea4332cc63d317837.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134513_195268_77589DAF X-CRM114-Status: GOOD ( 10.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Pull the SetPageSkipKASanPoison() call in post_alloc_hook() out of the big if clause for better code readability. This also allows for more simplifications in the following patches. Also turn the kasan_has_integrated_init() check into the proper CONFIG_KASAN_HW_TAGS one. These checks evaluate to the same value, but logically skipping kasan poisoning has nothing to do with integrated init. Signed-off-by: Andrey Konovalov --- mm/page_alloc.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 781b75563276..cbbaf76db6d9 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2420,9 +2420,6 @@ inline void post_alloc_hook(struct page *page, unsigned int order, init = false; } if (kasan_has_integrated_init()) { - if (gfp_flags & __GFP_SKIP_KASAN_POISON) - SetPageSkipKASanPoison(page); - if (!init_tags) kasan_unpoison_pages(page, order, init); } else { @@ -2431,6 +2428,10 @@ inline void post_alloc_hook(struct page *page, unsigned int order, if (init) kernel_init_free_pages(page, 1 << order); } + /* Propagate __GFP_SKIP_KASAN_POISON to page flags. */ + if (IS_ENABLED(CONFIG_KASAN_HW_TAGS) && + (gfp_flags & __GFP_SKIP_KASAN_POISON)) + SetPageSkipKASanPoison(page); set_page_owner(page, order, gfp_flags); } From patchwork Mon Dec 6 21:43:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12694998 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B08FBC433F5 for ; Mon, 6 Dec 2021 21:53:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ylRR8/XXl5FxqRLlcL7zV/phydpzv4y8tAV2B4uOhto=; b=jijL4mNz9ndiht DFu3HQ7bJ//63vFzbLsCwTUi5NkG8Jisqb3jZ/u2O4bCeY30HfevcuiuBPJNdY/OL4Gqev4oFF07+ EU5UVz4t7t3dW3w3yLSnHrl2utDm7BgDnYEJ3fwutiHzeZH+mivsBNCFVRnZvu4TX6eGnyF9yVQA+ 3T3vYIMvCiHxDtSaIrA00+ANax/ZNXITuLH6RdPHTAlrzdv3RcSqY1b8nKxpAqiNmpkwVk4Rb0PYX 0LnWwB1eNBfQzBsc7F4bgRoYo4Q2trqD2NtH7mRobtAlhTEGO0tNE8UImgGQPgb0SQRibgaIl+l5J JVj7FTHzQDdopf1dnCTw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLtC-0061UC-RX; Mon, 06 Dec 2021 21:51:15 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLnU-005yzZ-4z for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:45:21 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827118; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sq3jTC1ItQcEg8U0H5RA1L+mQBZGObrT47vWcbRK6Ok=; b=ks6MOpDryY6ANscXA0vYO6PTrU6RwmxL6UskNxWm8LAanUMvR4oNITiYqJPOpM6lf6FzEL rrf7kb8PTgrzSldFjjNRdU4zm3h2sUNSRiz/RD5DVTYBoOpsBsoaxSj1NclFAu40AcowCH pMgzFiOHUwMNz0nDBe1Uy5Eqhu+NEBY= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 13/34] kasan, page_alloc: move kernel_init_free_pages in post_alloc_hook Date: Mon, 6 Dec 2021 22:43:50 +0100 Message-Id: <42626baf4ce66be7fb1538c074b6508d9d867312.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134520_397142_C8D57831 X-CRM114-Status: GOOD ( 11.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Pull the kernel_init_free_pages() call in post_alloc_hook() out of the big if clause for better code readability. This also allows for more simplifications in the following patch. This patch does no functional changes. Signed-off-by: Andrey Konovalov --- mm/page_alloc.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index cbbaf76db6d9..5c346375cff9 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2420,14 +2420,18 @@ inline void post_alloc_hook(struct page *page, unsigned int order, init = false; } if (kasan_has_integrated_init()) { - if (!init_tags) + if (!init_tags) { kasan_unpoison_pages(page, order, init); + + /* Note that memory is already initialized by KASAN. */ + init = false; + } } else { kasan_unpoison_pages(page, order, init); - - if (init) - kernel_init_free_pages(page, 1 << order); } + /* If memory is still not initialized, do it now. */ + if (init) + kernel_init_free_pages(page, 1 << order); /* Propagate __GFP_SKIP_KASAN_POISON to page flags. */ if (IS_ENABLED(CONFIG_KASAN_HW_TAGS) && (gfp_flags & __GFP_SKIP_KASAN_POISON)) From patchwork Mon Dec 6 21:43:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12694999 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5F5C5C433F5 for ; Mon, 6 Dec 2021 21:53:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=JQqTw2IJtnFzu7DdO/KEMPH03ms3EcAaoeH3p7ucKTc=; b=dAbIJ8LV7RYVEO KkuT4xMNBSuIlse795Ll4J2Hh/3E1wfPkblj3uGanDRwVEWYrW7ulmQHOa8hJuGb7Mgbv4l5mM3oU Hfb9t/95EBiioX9mnKZDRHlsjiioT/orCkPQvnW0dwPBeM7qSpeDpvAbF6ULBk42xC0oFHMVcJizX uyUgd2Q4izcZV+R/iENzW9RSeidiTi37cKZyrxc69HQlM5Fj5zOhRWyOFkc/gXGIg4wzDFwCQL+GF 2hCTPH09YCwiDTzuycM4fVdV8iXZ+MZ7Xkr8WxqkhODx6DMMu4NTDaE4QVmzPcqmy5AJiuVD9E95c M82aFDrIM+JME0nm69Kg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLu0-0061oj-RK; Mon, 06 Dec 2021 21:52:05 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLnb-005z2R-5e for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:45:28 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827125; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DzChjposR8HVqGI1LyY9RFr0jz+Wu5dx9pG+xQIWYf8=; b=ka0h1Ym03kuH5Ox8LDn59GBrR/hauweOgUKRmsGeAXiqq7096r1v2XFaibQYmWydVy7RF7 h+192nLSScSWnCv4lCj6WsXt+/TMWwpV5IX8YjAcCtNZYBRe3Rfld6BxTzM5I8aAaE4Fsc 7LJiJZ+F7Sro7zz/twsMwtqU9UTjMxY= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 14/34] kasan, page_alloc: simplify kasan_unpoison_pages call site Date: Mon, 6 Dec 2021 22:43:51 +0100 Message-Id: <10ba3c7a7524a912098d3b1747c0ca2e1e626ebc.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134527_408071_4B8C4BB7 X-CRM114-Status: GOOD ( 13.16 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Simplify the checks around kasan_unpoison_pages() call in post_alloc_hook(). The logical condition for calling this function is: - If a software KASAN mode is enabled, we need to mark shadow memory. - Otherwise, HW_TAGS KASAN is enabled, and it only makes sense to set tags if they haven't already been cleared by tag_clear_highpage(), which is indicated by init_tags. This patch concludes the simplifications for post_alloc_hook(). Signed-off-by: Andrey Konovalov --- mm/page_alloc.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 5c346375cff9..73e6500c9767 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2419,15 +2419,18 @@ inline void post_alloc_hook(struct page *page, unsigned int order, /* Note that memory is already initialized by the loop above. */ init = false; } - if (kasan_has_integrated_init()) { - if (!init_tags) { - kasan_unpoison_pages(page, order, init); + /* + * If either a software KASAN mode is enabled, or, + * in the case of hardware tag-based KASAN, + * if memory tags have not been cleared via tag_clear_highpage(). + */ + if (!IS_ENABLED(CONFIG_KASAN_HW_TAGS) || !init_tags) { + /* Mark shadow memory or set memory tags. */ + kasan_unpoison_pages(page, order, init); - /* Note that memory is already initialized by KASAN. */ + /* Note that memory is already initialized by KASAN. */ + if (kasan_has_integrated_init()) init = false; - } - } else { - kasan_unpoison_pages(page, order, init); } /* If memory is still not initialized, do it now. */ if (init) From patchwork Mon Dec 6 21:43:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695000 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7D8EBC433EF for ; Mon, 6 Dec 2021 21:54:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=J6nA+u6rSo4FPi3AG/s6ifpWImkt6d99Qh+wev0paSA=; b=D5YowvTZupfvk8 1q1wINEsMdsOyAYQDjATY9YT4aJZFieYGZusEczY3/yLWRKa0Z6ppFqJACxSPdlx0HBD5YT5Ly+Jk JwvS4IeYLINtECnkyauQqv7n3TcdhjZNyPlmPTZVc8CpVXkAlVmzUBz26mK7lvffekgfDWHzeHtb4 x0ycSFvVPSJ/zQUL2N1sq+602TNKanZAFzOrdNCRU8CbJS2DG19UEifc7hiZ+3InwoPtR3prxEsCs mvyjJlDgPZ8/i6lxAPIvnEEx1yRe2GL7svCm0PfjNOk/uLNJQgZMaknAES9E7lvi/n9FLWkIKU8KK /LDZ1f8iC4RJpZo/fJ+g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLuu-0062Ap-3e; Mon, 06 Dec 2021 21:53:00 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLnh-005z4N-Kl for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:45:35 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827132; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pxjwWPacm+/5s9n0kHhZDBwjPwmcQ9h5CfpbBNedfPY=; b=LcblNL2Dhl2+KRKVhyQIrZvDQLyZ7lVXupbf9rT4WH27jbCvEgalHSsV3fgXLOcYEXLvFv hrBwiTnYG+YRlcs2Nn+4Y94IHjR8mtjxZr7mnJcGVNACoTCNHuRves86h50E2PoHQJGD7Q gYJ/46MhnfEleuzNaCHcBRzyRBmL4NM= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 15/34] kasan: clean up metadata byte definitions Date: Mon, 6 Dec 2021 22:43:52 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134533_889443_71F6A192 X-CRM114-Status: UNSURE ( 9.10 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Most of the metadata byte values are only used for Generic KASAN. Remove KASAN_KMALLOC_FREETRACK definition for !CONFIG_KASAN_GENERIC case, and put it along with other metadata values for the Generic mode under a corresponding ifdef. Signed-off-by: Andrey Konovalov --- mm/kasan/kasan.h | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index aebd8df86a1f..a50450160638 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -71,15 +71,16 @@ static inline bool kasan_sync_fault_possible(void) #define KASAN_PAGE_REDZONE 0xFE /* redzone for kmalloc_large allocations */ #define KASAN_KMALLOC_REDZONE 0xFC /* redzone inside slub object */ #define KASAN_KMALLOC_FREE 0xFB /* object was freed (kmem_cache_free/kfree) */ -#define KASAN_KMALLOC_FREETRACK 0xFA /* object was freed and has free track set */ #else #define KASAN_FREE_PAGE KASAN_TAG_INVALID #define KASAN_PAGE_REDZONE KASAN_TAG_INVALID #define KASAN_KMALLOC_REDZONE KASAN_TAG_INVALID #define KASAN_KMALLOC_FREE KASAN_TAG_INVALID -#define KASAN_KMALLOC_FREETRACK KASAN_TAG_INVALID #endif +#ifdef CONFIG_KASAN_GENERIC + +#define KASAN_KMALLOC_FREETRACK 0xFA /* object was freed and has free track set */ #define KASAN_GLOBAL_REDZONE 0xF9 /* redzone for global variable */ #define KASAN_VMALLOC_INVALID 0xF8 /* unallocated space in vmapped page */ @@ -110,6 +111,8 @@ static inline bool kasan_sync_fault_possible(void) #define KASAN_ABI_VERSION 1 #endif +#endif /* CONFIG_KASAN_GENERIC */ + /* Metadata layout customization. */ #define META_BYTES_PER_BLOCK 1 #define META_BLOCKS_PER_ROW 16 From patchwork Mon Dec 6 21:43:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695001 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EDFF7C433EF for ; Mon, 6 Dec 2021 21:55:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=9LMXwHHyqVvVzv5WkYT61VAjNtWv0JT71wSCmDu6HF0=; b=ICaPp8QQyoglKP ndSmIPq+25ZwpjCuGmVhheSqj1Hpf9zCPalsm6eUmZuEl/EyIXuMAJgnSYO8c9QOQYJW7r4enI174 lMKxYPnZWCxgUTspI/FZEo0eU0GTsFJgeYJWMmrR6JVDWgkpLQhgmj9mSuu+GQbDft68zvYPvtg9L jB/sXSR0oqXgI5C3thIvspK3LsLtDhAwscRmstLD2QE1sM00HcH9fI5babXssBNt+MyKWLYvkmJtw jhjeVSS9UR6ZDC8mGopl2uCdVcKjyADDazcMp+anchDeXJMhwHMuYNXZEzh8GD4fD0H0BtnbONX2C JOmCqaFuMJg0MxRXx+qA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLvz-0062gD-Im; Mon, 06 Dec 2021 21:54:08 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLnp-005z74-0j for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:45:42 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827139; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XbhL8TrmzwL9tq/+XeKdOp7BHc1RVFQ8fUYh+mFdIuI=; b=WPYCRHJuqRxj6DlRzMdcSwAHgTaV+gNwN4tmFLWsvKX36iTE2dLLFEmMBfSi7WM8u/dc/I G0AEdVM0xYRyE79yNt/C908Uv4g2g7h8YrRoxF1fsAbczLeWjRBi42PtQTgec3QCJlVJfp GpwRtgf2TOSJxW9AkLwt7vDAPMBb5k4= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 16/34] kasan: define KASAN_VMALLOC_INVALID for SW_TAGS Date: Mon, 6 Dec 2021 22:43:53 +0100 Message-Id: <599ecad2c26832e053a248737207cb1a1e4ed039.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134541_261596_D23DD0CE X-CRM114-Status: UNSURE ( 9.96 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov In preparation for adding vmalloc support to SW_TAGS KASAN, provide a KASAN_VMALLOC_INVALID definition for it. HW_TAGS KASAN won't be using this value, as it falls back onto page_alloc for poisoning freed vmalloc() memory. Signed-off-by: Andrey Konovalov --- mm/kasan/kasan.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index a50450160638..0827d74d0d87 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -71,18 +71,19 @@ static inline bool kasan_sync_fault_possible(void) #define KASAN_PAGE_REDZONE 0xFE /* redzone for kmalloc_large allocations */ #define KASAN_KMALLOC_REDZONE 0xFC /* redzone inside slub object */ #define KASAN_KMALLOC_FREE 0xFB /* object was freed (kmem_cache_free/kfree) */ +#define KASAN_VMALLOC_INVALID 0xF8 /* unallocated space in vmapped page */ #else #define KASAN_FREE_PAGE KASAN_TAG_INVALID #define KASAN_PAGE_REDZONE KASAN_TAG_INVALID #define KASAN_KMALLOC_REDZONE KASAN_TAG_INVALID #define KASAN_KMALLOC_FREE KASAN_TAG_INVALID +#define KASAN_VMALLOC_INVALID KASAN_TAG_INVALID /* only for SW_TAGS */ #endif #ifdef CONFIG_KASAN_GENERIC #define KASAN_KMALLOC_FREETRACK 0xFA /* object was freed and has free track set */ #define KASAN_GLOBAL_REDZONE 0xF9 /* redzone for global variable */ -#define KASAN_VMALLOC_INVALID 0xF8 /* unallocated space in vmapped page */ /* * Stack redzone shadow values From patchwork Mon Dec 6 21:43:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695002 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F27C4C433EF for ; Mon, 6 Dec 2021 21:57:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ag7qwGQzZEa3X7vxWS7sG0QL4vtnxCDv0xcAhZPazMU=; b=HGmTyGCdvBegHO pYv8iJPcYIo2SzTMfsYACjMT2pTOmSQ3dCVcnXw1uNIMvzA/6jMUUQHMK9w0AKjVHUuLDOMFuwoOZ lVwuJy3QZjP+4BUnP32isIawzQQo4U1W5ITWuhwpOi8s4mEYdH7k+B+gLXMuBfrdwIR1u55UOIuhc orHjtWgC5h/NHdXRy7QHQpNVgWuj7n32ZtWTl1UPZET0XoW47fbkO1HSNPrYicro7Xc6Np/yGwnj0 thhq1PtTLtBgsmWR/E7Zw4/kaJpi6xM0iDY9euByTy0XlcEfZ7lFDaDw+RuF6Q/8tDuGBl+YTmegM bY54H1PCec4iINf5aj3Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLwx-00634T-10; Mon, 06 Dec 2021 21:55:07 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLnu-005z9h-Gg for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:45:48 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827145; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5vrmXGtyQkXmrhoONS3mEv98+EJ5a6iWSXEuOfvldi8=; b=d7P/LxHyM6jtvQS5baA6aRMBUddHSC/GthpLY39qjL5PfBw71gfmfjspU1S2YvHDAy16Bl Ucg99Oq6euUS+VsrWl1+O0ooSG31xZrVY5MgEy4jN1m+XqTp+Eo+buslB8lVSgup+m1mo8 q1fvvr4mCw4Xw2Xh+F9qARf7zpYn1Do= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 17/34] kasan, x86, arm64, s390: rename functions for modules shadow Date: Mon, 6 Dec 2021 22:43:54 +0100 Message-Id: <11f5a6419f8830fdedc84dca5f847543ef7960f4.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134546_941066_B27426E2 X-CRM114-Status: GOOD ( 14.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Rename kasan_free_shadow to kasan_free_module_shadow and kasan_module_alloc to kasan_alloc_module_shadow. These functions are used to allocate/free shadow memory for kernel modules when KASAN_VMALLOC is not enabled. The new names better reflect their purpose. Also reword the comment next to their declaration to improve clarity. Signed-off-by: Andrey Konovalov Acked-by: Catalin Marinas --- arch/arm64/kernel/module.c | 2 +- arch/s390/kernel/module.c | 2 +- arch/x86/kernel/module.c | 2 +- include/linux/kasan.h | 14 +++++++------- mm/kasan/shadow.c | 4 ++-- mm/vmalloc.c | 2 +- 6 files changed, 13 insertions(+), 13 deletions(-) diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c index b5ec010c481f..f8bd5100efb5 100644 --- a/arch/arm64/kernel/module.c +++ b/arch/arm64/kernel/module.c @@ -58,7 +58,7 @@ void *module_alloc(unsigned long size) PAGE_KERNEL, 0, NUMA_NO_NODE, __builtin_return_address(0)); - if (p && (kasan_module_alloc(p, size) < 0)) { + if (p && (kasan_alloc_module_shadow(p, size) < 0)) { vfree(p); return NULL; } diff --git a/arch/s390/kernel/module.c b/arch/s390/kernel/module.c index b01ba460b7ca..a753cebedda9 100644 --- a/arch/s390/kernel/module.c +++ b/arch/s390/kernel/module.c @@ -44,7 +44,7 @@ void *module_alloc(unsigned long size) p = __vmalloc_node_range(size, MODULE_ALIGN, MODULES_VADDR, MODULES_END, GFP_KERNEL, PAGE_KERNEL_EXEC, 0, NUMA_NO_NODE, __builtin_return_address(0)); - if (p && (kasan_module_alloc(p, size) < 0)) { + if (p && (kasan_alloc_module_shadow(p, size) < 0)) { vfree(p); return NULL; } diff --git a/arch/x86/kernel/module.c b/arch/x86/kernel/module.c index 169fb6f4cd2e..dec41d9ba337 100644 --- a/arch/x86/kernel/module.c +++ b/arch/x86/kernel/module.c @@ -77,7 +77,7 @@ void *module_alloc(unsigned long size) MODULES_END, GFP_KERNEL, PAGE_KERNEL, 0, NUMA_NO_NODE, __builtin_return_address(0)); - if (p && (kasan_module_alloc(p, size) < 0)) { + if (p && (kasan_alloc_module_shadow(p, size) < 0)) { vfree(p); return NULL; } diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 1031070be3f3..4eec58e6ef82 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -453,17 +453,17 @@ static inline void kasan_populate_early_vm_area_shadow(void *start, !defined(CONFIG_KASAN_VMALLOC) /* - * These functions provide a special case to support backing module - * allocations with real shadow memory. With KASAN vmalloc, the special - * case is unnecessary, as the work is handled in the generic case. + * These functions allocate and free shadow memory for kernel modules. + * They are only required when KASAN_VMALLOC is not supported, as otherwise + * shadow memory is allocated by the generic vmalloc handlers. */ -int kasan_module_alloc(void *addr, size_t size); -void kasan_free_shadow(const struct vm_struct *vm); +int kasan_alloc_module_shadow(void *addr, size_t size); +void kasan_free_module_shadow(const struct vm_struct *vm); #else /* (CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) && !CONFIG_KASAN_VMALLOC */ -static inline int kasan_module_alloc(void *addr, size_t size) { return 0; } -static inline void kasan_free_shadow(const struct vm_struct *vm) {} +static inline int kasan_alloc_module_shadow(void *addr, size_t size) { return 0; } +static inline void kasan_free_module_shadow(const struct vm_struct *vm) {} #endif /* (CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS) && !CONFIG_KASAN_VMALLOC */ diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index 4a4929b29a23..585c2bf1073b 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -498,7 +498,7 @@ void kasan_release_vmalloc(unsigned long start, unsigned long end, #else /* CONFIG_KASAN_VMALLOC */ -int kasan_module_alloc(void *addr, size_t size) +int kasan_alloc_module_shadow(void *addr, size_t size) { void *ret; size_t scaled_size; @@ -529,7 +529,7 @@ int kasan_module_alloc(void *addr, size_t size) return -ENOMEM; } -void kasan_free_shadow(const struct vm_struct *vm) +void kasan_free_module_shadow(const struct vm_struct *vm) { if (vm->flags & VM_KASAN) vfree(kasan_mem_to_shadow(vm->addr)); diff --git a/mm/vmalloc.c b/mm/vmalloc.c index d2a00ad4e1dd..c5235e3e5857 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2524,7 +2524,7 @@ struct vm_struct *remove_vm_area(const void *addr) va->vm = NULL; spin_unlock(&vmap_area_lock); - kasan_free_shadow(vm); + kasan_free_module_shadow(vm); free_unmap_vmap_area(va); return vm; From patchwork Mon Dec 6 21:43:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 17F59C433FE for ; Mon, 6 Dec 2021 21:57:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=vSZTGQ9QXgMf2UPFtZK1XiG6ol4c7PECZKxviBhRxpU=; b=Q0MiM9St8q7v0l hBRwNmucBakP9FqSsbXafKBF++fk/lrjoryvfhev8RlgHdlvZ8QCX+OJHrC3bD0yOjWzPNwJps69I eROiMtkQojvF0LomUqe00b/3EPfSGNZSwcH2saAL08353T7jNKYomftmsv+uMsmatdF+eyuTAOtuG xOOBZQ36+G4wNkfGim+PoMFFhTwApYZ1luf6R3KFSBMpDpSET/nRAHQe1PDDXdKT5UefHssHhaHNA rjb8xJNI3fRBnZn8hXKsFv4Q21oKko5wSnE9qaB9inLKWK4BNC1/KLlHqpdqPbSjAIPsbDeajnl36 oVB/9yAdmPGRemkhW4DA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLxf-0063Mf-AU; Mon, 06 Dec 2021 21:55:52 +0000 Received: from out2.migadu.com ([188.165.223.204]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLo2-005zCi-FL for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:45:57 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827153; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qxn2Xep6jUSip5qM4WVsL2tCgAIiNA6s93LyLl9ypbE=; b=sBk8L+hA2GSdvOEbvDg2sGEJ6BG9LYDOLDc4ayXuZZpgSc8QoAowuwfLwICf8I+5L0fZt1 0sZSRjpfEUfRz0B/0AHqUiuOsb8YqXEb/7XCgdLHNPIKkk13V+1dlWMM9K7Ru9dUjum3hf NPS8tS9D0hluyCZNY8rV9F1yr1j8xfg= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 18/34] kasan, vmalloc: drop outdated VM_KASAN comment Date: Mon, 6 Dec 2021 22:43:55 +0100 Message-Id: <4d44c09c5999cf4767803724eb47581294f4341c.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134554_718320_ED1D0165 X-CRM114-Status: GOOD ( 11.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov The comment about VM_KASAN in include/linux/vmalloc.c is outdated. VM_KASAN is currently only used to mark vm_areas allocated for kernel modules when CONFIG_KASAN_VMALLOC is disabled. Drop the comment. Signed-off-by: Andrey Konovalov --- include/linux/vmalloc.h | 11 ----------- 1 file changed, 11 deletions(-) diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 6e022cc712e6..b22369f540eb 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -28,17 +28,6 @@ struct notifier_block; /* in notifier.h */ #define VM_MAP_PUT_PAGES 0x00000200 /* put pages and free array in vfree */ #define VM_NO_HUGE_VMAP 0x00000400 /* force PAGE_SIZE pte mapping */ -/* - * VM_KASAN is used slightly differently depending on CONFIG_KASAN_VMALLOC. - * - * If IS_ENABLED(CONFIG_KASAN_VMALLOC), VM_KASAN is set on a vm_struct after - * shadow memory has been mapped. It's used to handle allocation errors so that - * we don't try to poison shadow on free if it was never allocated. - * - * Otherwise, VM_KASAN is set for kasan_module_alloc() allocations and used to - * determine which allocations need the module shadow freed. - */ - /* bits [20..32] reserved for arch specific ioremap internals */ /* From patchwork Mon Dec 6 21:43:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695004 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 72425C433F5 for ; Mon, 6 Dec 2021 21:58:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Ha9y6PS37CqqRmF7DUCOVUWe6ec0Jyelozl0Kso9So0=; b=Su3UXy/xWLh3kp CqfBp9Ympm8MBnSr3Pb4dIkJshHELCXvdw/7K21D/rsRN9WhEaKHg8myhcdlGphi/buEZmGURaHXL iSngXtwO5DYjLwXvbcckYcallChz8vif4wVOFtQLPA7JqTTljTx30eD1tnj1EI3lgw4siGnZLM9LU XguRp6eU38W/yJUUU+eBRFW8FSP+W/DbfKIgrWIaaYYdc8QFNsJi/co3G5yDG/5x4r27ZGAjIN5RX oaNBSIxsE+Uh8/DJDdOp+1daxRLQOomhHjxqySop5cHE8U8Jk15UhGIZ6Vyn2YkB0iyiJmIaSJOof XG9Kjqpuqv4a9Jphmoag==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLyh-0063kg-QW; Mon, 06 Dec 2021 21:56:57 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLo8-005zFT-5H for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:46:01 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827158; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JTNahgB9b8xFEkRfhyeNPJ7ITVoB0ROvdRGcm8t3Kk0=; b=W/pnZKDTzt0ZvxJmo1x5OPPRZRKjSv0T6iFkBTSWoeTmTvq25TfpVEihWtwD4u+LO+iBQK t2EE47Es2sMYVGgGTPKx4r4xbkzyUTO1HCXjADS0Bsu+B4aCYVtzu9ntJV4Lnt4FtSZAxq I57Kt/VAh236UGJJKk+q1QS8Ax5ebzk= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 19/34] kasan: reorder vmalloc hooks Date: Mon, 6 Dec 2021 22:43:56 +0100 Message-Id: <290884a271e5adc79ef0121868558c1e19db1b70.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134600_569980_EB0C1C2C X-CRM114-Status: GOOD ( 12.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Group functions that [de]populate shadow memory for vmalloc. Group functions that [un]poison memory for vmalloc. This patch does no functional changes but prepares KASAN code for adding vmalloc support to HW_TAGS KASAN. Signed-off-by: Andrey Konovalov --- include/linux/kasan.h | 20 +++++++++----------- mm/kasan/shadow.c | 43 ++++++++++++++++++++++--------------------- 2 files changed, 31 insertions(+), 32 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 4eec58e6ef82..af2dd67d2c0e 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -417,34 +417,32 @@ static inline void kasan_init_hw_tags(void) { } #ifdef CONFIG_KASAN_VMALLOC +void kasan_populate_early_vm_area_shadow(void *start, unsigned long size); int kasan_populate_vmalloc(unsigned long addr, unsigned long size); -void kasan_poison_vmalloc(const void *start, unsigned long size); -void kasan_unpoison_vmalloc(const void *start, unsigned long size); void kasan_release_vmalloc(unsigned long start, unsigned long end, unsigned long free_region_start, unsigned long free_region_end); -void kasan_populate_early_vm_area_shadow(void *start, unsigned long size); +void kasan_unpoison_vmalloc(const void *start, unsigned long size); +void kasan_poison_vmalloc(const void *start, unsigned long size); #else /* CONFIG_KASAN_VMALLOC */ +static inline void kasan_populate_early_vm_area_shadow(void *start, + unsigned long size) { } static inline int kasan_populate_vmalloc(unsigned long start, unsigned long size) { return 0; } - -static inline void kasan_poison_vmalloc(const void *start, unsigned long size) -{ } -static inline void kasan_unpoison_vmalloc(const void *start, unsigned long size) -{ } static inline void kasan_release_vmalloc(unsigned long start, unsigned long end, unsigned long free_region_start, - unsigned long free_region_end) {} + unsigned long free_region_end) { } -static inline void kasan_populate_early_vm_area_shadow(void *start, - unsigned long size) +static inline void kasan_unpoison_vmalloc(const void *start, unsigned long size) +{ } +static inline void kasan_poison_vmalloc(const void *start, unsigned long size) { } #endif /* CONFIG_KASAN_VMALLOC */ diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index 585c2bf1073b..49a3660e111a 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -345,27 +345,6 @@ int kasan_populate_vmalloc(unsigned long addr, unsigned long size) return 0; } -/* - * Poison the shadow for a vmalloc region. Called as part of the - * freeing process at the time the region is freed. - */ -void kasan_poison_vmalloc(const void *start, unsigned long size) -{ - if (!is_vmalloc_or_module_addr(start)) - return; - - size = round_up(size, KASAN_GRANULE_SIZE); - kasan_poison(start, size, KASAN_VMALLOC_INVALID, false); -} - -void kasan_unpoison_vmalloc(const void *start, unsigned long size) -{ - if (!is_vmalloc_or_module_addr(start)) - return; - - kasan_unpoison(start, size, false); -} - static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr, void *unused) { @@ -496,6 +475,28 @@ void kasan_release_vmalloc(unsigned long start, unsigned long end, } } + +void kasan_unpoison_vmalloc(const void *start, unsigned long size) +{ + if (!is_vmalloc_or_module_addr(start)) + return; + + kasan_unpoison(start, size, false); +} + +/* + * Poison the shadow for a vmalloc region. Called as part of the + * freeing process at the time the region is freed. + */ +void kasan_poison_vmalloc(const void *start, unsigned long size) +{ + if (!is_vmalloc_or_module_addr(start)) + return; + + size = round_up(size, KASAN_GRANULE_SIZE); + kasan_poison(start, size, KASAN_VMALLOC_INVALID, false); +} + #else /* CONFIG_KASAN_VMALLOC */ int kasan_alloc_module_shadow(void *addr, size_t size) From patchwork Mon Dec 6 21:43:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EC230C4332F for ; Mon, 6 Dec 2021 21:59:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Z+MtvONSCRaUUJtwsAwIdjFzFfwt2+QxCwk98eX5wmY=; b=LdXjoFCh90ayS5 Xi6Avd7On3wVFyNNO3vSNtjIZREWwzwfeqlYu6d6E2yGZv/MsKm0iLe7AzwbvYg1+2Ka4e1hYR5Ko FHTFnl7eEq9l5XjkGdO96cR+paoNQn4/3G7z8LtNoqMtpRiiapR+cdo7hRMbFx4rEsiOmNWVyymSA Q3YqeNwg9Naula8bUn89drvas+dtZaYi9440ULRjoZ2uWHpvIy8qMn4schO1vMbtxZOO0SUpGGhdb dlzv0Xvob4VqPSF9ZUtUdOw3CFg+Kuiqr04Jigyp92LR2c61oDY3o/CaP8OM72thKO/a/lUTgMO2G xBTSsIj0ew9wkle7KAKg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLzS-00643N-Dw; Mon, 06 Dec 2021 21:57:42 +0000 Received: from out2.migadu.com ([188.165.223.204]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLoC-005zHb-Mx for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:46:06 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827163; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=n4gFh8zlpMsEgMrbMj+8ISQjGusuzX7sFY8iv+MHRNk=; b=BAU7QhMXad4k+Bs4uuFgfTy8Osx4N2eU9XW5HUcmBIoE2tHu5eEqChynIkRi7h/UPjmAyp Wh2B2BXG5PE6OPKyfga9Z+TUt2aCvkh3SuR6hB0eOuIfTirt7GBbqWO5YePYcEjhP0v39c o+j3IODMSj+AEtGoOy8M746E7XjM/j0= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 20/34] kasan: add wrappers for vmalloc hooks Date: Mon, 6 Dec 2021 22:43:57 +0100 Message-Id: <8944b0d772ed776b3d79eb26ed1bcf6888b0f378.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134604_962071_032BFC7B X-CRM114-Status: GOOD ( 11.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Add wrappers around functions that [un]poison memory for vmalloc allocations. These functions will be used by HW_TAGS KASAN and therefore need to be disabled when kasan=off command line argument is provided. This patch does no functional changes for software KASAN modes. Signed-off-by: Andrey Konovalov --- include/linux/kasan.h | 17 +++++++++++++++-- mm/kasan/shadow.c | 5 ++--- 2 files changed, 17 insertions(+), 5 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index af2dd67d2c0e..ad4798e77f60 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -423,8 +423,21 @@ void kasan_release_vmalloc(unsigned long start, unsigned long end, unsigned long free_region_start, unsigned long free_region_end); -void kasan_unpoison_vmalloc(const void *start, unsigned long size); -void kasan_poison_vmalloc(const void *start, unsigned long size); +void __kasan_unpoison_vmalloc(const void *start, unsigned long size); +static __always_inline void kasan_unpoison_vmalloc(const void *start, + unsigned long size) +{ + if (kasan_enabled()) + __kasan_unpoison_vmalloc(start, size); +} + +void __kasan_poison_vmalloc(const void *start, unsigned long size); +static __always_inline void kasan_poison_vmalloc(const void *start, + unsigned long size) +{ + if (kasan_enabled()) + __kasan_poison_vmalloc(start, size); +} #else /* CONFIG_KASAN_VMALLOC */ diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index 49a3660e111a..fa0c8a750d09 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -475,8 +475,7 @@ void kasan_release_vmalloc(unsigned long start, unsigned long end, } } - -void kasan_unpoison_vmalloc(const void *start, unsigned long size) +void __kasan_unpoison_vmalloc(const void *start, unsigned long size) { if (!is_vmalloc_or_module_addr(start)) return; @@ -488,7 +487,7 @@ void kasan_unpoison_vmalloc(const void *start, unsigned long size) * Poison the shadow for a vmalloc region. Called as part of the * freeing process at the time the region is freed. */ -void kasan_poison_vmalloc(const void *start, unsigned long size) +void __kasan_poison_vmalloc(const void *start, unsigned long size) { if (!is_vmalloc_or_module_addr(start)) return; From patchwork Mon Dec 6 21:43:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695006 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E3B70C433F5 for ; Mon, 6 Dec 2021 22:00:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=yt/maOHPvH56sKvOLcb13BVhHW59EUxCCZM56s7y4pw=; b=r91vju4I1tzO0J peo1LEzqTgXkR2drw81GHupaEBgg5eftfJmkNJRSzTMPvxLsYf79y1LZaWCBQG/uMtJiSEj+OWOfe Ld558tGty7JxkZCmWXZ/KZZlDqpwo3V3mvnn5gD2GT5ltrdkRROHYzNFgUR6QyY7CGbJjzStEJlEv CYHrgab1+F8D5HJrfpslFYtb+zrneoKXvIXi9ld+8t7JMuWkyj7gkTF42y63hxK2JUe9NSBplbB1K uzqu5aFRzAHqnc9GfdPqt2JWuGiSqGYyNk+KfVTh3dfhoIsjDaLecXwTqmGJRxz8qOSBCDKueHxGe 8zLjbqg7DtrkI1b4ULeA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muM0F-0064Ss-8g; Mon, 06 Dec 2021 21:58:31 +0000 Received: from out2.migadu.com ([188.165.223.204]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLoH-005zJT-J2 for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:46:11 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827168; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DG1z80n+FdxKIjaCouQa8EFEG7uKXNAbYhiqtcBDOzg=; b=vxro8/A3f6u+g9tCqrxJ0dPCIULo7w6FPsYXp/AX31m8If40/krW8khAxSScfLQhWgb8Eb YLQDToXJ/36hUhhZu/93oF+LNBqkmvSzLeIefrwDvKmBnQAgQXuP6tJXKAeV/VY01tLdfV PxQG8+Q+2QyGq/Fbt10fhQrtjPyL0ds= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 21/34] kasan, vmalloc: reset tags in vmalloc functions Date: Mon, 6 Dec 2021 22:43:58 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134609_825221_4CDD4174 X-CRM114-Status: GOOD ( 12.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov In preparation for adding vmalloc support to SW/HW_TAGS KASAN, reset pointer tags in functions that use pointer values in range checks. vread() is a special case here. Despite the untagging of the addr pointer in its prologue, the accesses performed by vread() are checked. Instead of accessing the virtual mappings though addr directly, vread() recovers the physical address via page_address(vmalloc_to_page()) and acceses that. And as page_address() recovers the pointer tag, the accesses get checked. Signed-off-by: Andrey Konovalov --- Changes v1->v2: - Clarified the description of untagging in vread(). --- mm/vmalloc.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index c5235e3e5857..a059b3100c0a 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -72,7 +72,7 @@ static const bool vmap_allow_huge = false; bool is_vmalloc_addr(const void *x) { - unsigned long addr = (unsigned long)x; + unsigned long addr = (unsigned long)kasan_reset_tag(x); return addr >= VMALLOC_START && addr < VMALLOC_END; } @@ -630,7 +630,7 @@ int is_vmalloc_or_module_addr(const void *x) * just put it in the vmalloc space. */ #if defined(CONFIG_MODULES) && defined(MODULES_VADDR) - unsigned long addr = (unsigned long)x; + unsigned long addr = (unsigned long)kasan_reset_tag(x); if (addr >= MODULES_VADDR && addr < MODULES_END) return 1; #endif @@ -804,6 +804,8 @@ static struct vmap_area *find_vmap_area_exceed_addr(unsigned long addr) struct vmap_area *va = NULL; struct rb_node *n = vmap_area_root.rb_node; + addr = (unsigned long)kasan_reset_tag((void *)addr); + while (n) { struct vmap_area *tmp; @@ -825,6 +827,8 @@ static struct vmap_area *__find_vmap_area(unsigned long addr) { struct rb_node *n = vmap_area_root.rb_node; + addr = (unsigned long)kasan_reset_tag((void *)addr); + while (n) { struct vmap_area *va; @@ -2143,7 +2147,7 @@ EXPORT_SYMBOL_GPL(vm_unmap_aliases); void vm_unmap_ram(const void *mem, unsigned int count) { unsigned long size = (unsigned long)count << PAGE_SHIFT; - unsigned long addr = (unsigned long)mem; + unsigned long addr = (unsigned long)kasan_reset_tag(mem); struct vmap_area *va; might_sleep(); @@ -3361,6 +3365,8 @@ long vread(char *buf, char *addr, unsigned long count) unsigned long buflen = count; unsigned long n; + addr = kasan_reset_tag(addr); + /* Don't allow overflow */ if ((unsigned long) addr + count < count) count = -(unsigned long) addr; From patchwork Mon Dec 6 21:43:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4AC22C433EF for ; Mon, 6 Dec 2021 22:00:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=1sCxx1yBjLq+1Z/DTzVuzYfudyzk72l59sAFekNH2mw=; b=ektjyp3fqjyruO vqFrupf92OMQEUbEWc6Gcuy9Dz68RtnAnb05zdEPOXH2IH+HF7K7ZeTn+ukGS+bvCTLiWSUJuFNmN hPKj6ZLfqGdJnKh5klK1sDuUIb6MOcOvlCIDP8T4oAa7Ofi8RRcnuJcjQMsDKK7H89zJmukrean3h u6+6bXrRrJHeYgO844tN5hKKQayXvEub/4xfOpJPRvNICwBT/sNI8/ANgPp51330nYFonBy7GqN6x 7Okn47rFzdZLvM8HZ/kxQP4c7RRMK7rJDEA4q2czF6f3Up7TzBbAaN8BzkuAku6tWf6vNC8na057L D5oikc4ZcUfiMtb/ks7g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muM0v-0064l8-Ls; Mon, 06 Dec 2021 21:59:13 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLoM-005zLh-Aw for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:46:15 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827172; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nHVI23+3UF63ZVUN3WDdsPua2nJbVrQ0CU8a80/ebmM=; b=ujYKIeajFpP/PIrY0lYCB4MCZN3i1pSExHZJXIcrpXgbnsJzMD4EgICHob7YlR6kne7Tiz gbfuUVKVkF9Q1Cy4b9XwYxyjRHGj4uvef0TupKTFXNk0N5EnuAzM6PwXz1nXkBVCiiv3mg UwHPb9ND+kz9nrNsKy7uzIxwP7b++e0= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 22/34] kasan, fork: don't tag stacks allocated with vmalloc Date: Mon, 6 Dec 2021 22:43:59 +0100 Message-Id: <92424a5bd4ceaabe6412da558624f2340d107756.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134614_586811_8A6D35A8 X-CRM114-Status: UNSURE ( 9.69 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Once tag-based KASAN modes start tagging vmalloc() allocations, kernel stacks will start getting tagged if CONFIG_VMAP_STACK is enabled. Reset the tag of kernel stack pointers after allocation. For SW_TAGS KASAN, when CONFIG_KASAN_STACK is enabled, the instrumentation can't handle the sp register being tagged. For HW_TAGS KASAN, there's no instrumentation-related issues. However, the impact of having a tagged SP pointer needs to be properly evaluated, so keep it non-tagged for now. Note, that the memory for the stack allocation still gets tagged to catch vmalloc-into-stack out-of-bounds accesses. Signed-off-by: Andrey Konovalov --- kernel/fork.c | 1 + 1 file changed, 1 insertion(+) diff --git a/kernel/fork.c b/kernel/fork.c index 3244cc56b697..062d1484ef42 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -253,6 +253,7 @@ static unsigned long *alloc_thread_stack_node(struct task_struct *tsk, int node) * so cache the vm_struct. */ if (stack) { + stack = kasan_reset_tag(stack); tsk->stack_vm_area = find_vm_area(stack); tsk->stack = stack; } From patchwork Mon Dec 6 21:44:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695008 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EAA3AC433EF for ; Mon, 6 Dec 2021 22:01:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=1xnLWUFRTzBX10TgnhCBK/+okayWMAVNKc2Zp6Da3gs=; b=bu/uDtTY2IKCGX /JXTHvq97I5kmDIsS42jb6OCi5AurNMTRg4wfwjRi3sUVhKBWn3tYW5TglTsIfMg0A2qWEF1FyIKG fDHQBC9mXaoG+hUSDqlqsp50ZjIxFwDGbZNbMMAS78FikH62y1K0/70Rd00P/Te1MX1/7Su+HGg1X NSZXjCVmbDhMM4HY0Elwlc5Sc1wmuvzLJ7bZ1G6NNY9XKWVamKjNsS3+ZS+FX9sUkmrMiio0sZaTf c/iOQ3XFCiBAnvQXlcxFT0EJ/58RghL3K1amYBzAQavxGfu0coDQ/Hpri1zJqdxixPyGi17R27Tqa YznFGIPXTB9gi6D56HQw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muM1g-00658u-SA; Mon, 06 Dec 2021 22:00:01 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLoR-005zNx-CP for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:46:21 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827178; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4ip+mfiCrUsqaU/ZPq/AUCt3O9J1b1z5H3QxOY+5r30=; b=jpdwDylT/uL4URAlxZowoEzRKnBJtPW77wI4HMlU8g00LVAfw9613B3AicbBk1BI2NRIot K32IXPXhnspxnmJrDkOlWVoY6BtqWM1Svq63bBuOezsDU3FrWLWkFutkZemteUnJR2HKED gwUE9lG6TiioKxiDfBP4kpfsNzfPwXg= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 23/34] kasan, vmalloc: add vmalloc support to SW_TAGS Date: Mon, 6 Dec 2021 22:44:00 +0100 Message-Id: <666b9e932dde24df6e1b02493a04530b99ace697.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134619_788569_9F316AA6 X-CRM114-Status: GOOD ( 14.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov This patch adds vmalloc tagging support to SW_TAGS KASAN. The changes include: - __kasan_unpoison_vmalloc() now assigns a random pointer tag, poisons the virtual mapping accordingly, and embeds the tag into the returned pointer. - __get_vm_area_node() (used by vmalloc() and vmap()) and pcpu_get_vm_areas() save the tagged pointer into vm_struct->addr (note: not into vmap_area->addr). This requires putting kasan_unpoison_vmalloc() after setup_vmalloc_vm[_locked](); otherwise the latter will overwrite the tagged pointer. The tagged pointer then is naturally propagateed to vmalloc() and vmap(). - vm_map_ram() returns the tagged pointer directly. - Allow enabling KASAN_VMALLOC with SW_TAGS. Signed-off-by: Andrey Konovalov --- Changes v1->v2: - Allow enabling KASAN_VMALLOC with SW_TAGS in this patch. --- include/linux/kasan.h | 17 +++++++++++------ lib/Kconfig.kasan | 2 +- mm/kasan/shadow.c | 6 ++++-- mm/vmalloc.c | 14 ++++++++------ 4 files changed, 24 insertions(+), 15 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index ad4798e77f60..6a2619759e93 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -423,12 +423,14 @@ void kasan_release_vmalloc(unsigned long start, unsigned long end, unsigned long free_region_start, unsigned long free_region_end); -void __kasan_unpoison_vmalloc(const void *start, unsigned long size); -static __always_inline void kasan_unpoison_vmalloc(const void *start, - unsigned long size) +void * __must_check __kasan_unpoison_vmalloc(const void *start, + unsigned long size); +static __always_inline void * __must_check kasan_unpoison_vmalloc( + const void *start, unsigned long size) { if (kasan_enabled()) - __kasan_unpoison_vmalloc(start, size); + return __kasan_unpoison_vmalloc(start, size); + return (void *)start; } void __kasan_poison_vmalloc(const void *start, unsigned long size); @@ -453,8 +455,11 @@ static inline void kasan_release_vmalloc(unsigned long start, unsigned long free_region_start, unsigned long free_region_end) { } -static inline void kasan_unpoison_vmalloc(const void *start, unsigned long size) -{ } +static inline void *kasan_unpoison_vmalloc(const void *start, + unsigned long size, bool unique) +{ + return (void *)start; +} static inline void kasan_poison_vmalloc(const void *start, unsigned long size) { } diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan index cdc842d090db..3f144a87f8a3 100644 --- a/lib/Kconfig.kasan +++ b/lib/Kconfig.kasan @@ -179,7 +179,7 @@ config KASAN_TAGS_IDENTIFY config KASAN_VMALLOC bool "Back mappings in vmalloc space with real shadow memory" - depends on KASAN_GENERIC && HAVE_ARCH_KASAN_VMALLOC + depends on (KASAN_GENERIC || KASAN_SW_TAGS) && HAVE_ARCH_KASAN_VMALLOC help By default, the shadow region for vmalloc space is the read-only zero page. This means that KASAN cannot detect errors involving diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index fa0c8a750d09..4ca280a96fbc 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -475,12 +475,14 @@ void kasan_release_vmalloc(unsigned long start, unsigned long end, } } -void __kasan_unpoison_vmalloc(const void *start, unsigned long size) +void *__kasan_unpoison_vmalloc(const void *start, unsigned long size) { if (!is_vmalloc_or_module_addr(start)) - return; + return (void *)start; + start = set_tag(start, kasan_random_tag()); kasan_unpoison(start, size, false); + return (void *)start; } /* diff --git a/mm/vmalloc.c b/mm/vmalloc.c index a059b3100c0a..7be18b292679 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2208,7 +2208,7 @@ void *vm_map_ram(struct page **pages, unsigned int count, int node) mem = (void *)addr; } - kasan_unpoison_vmalloc(mem, size); + mem = kasan_unpoison_vmalloc(mem, size); if (vmap_pages_range(addr, addr + size, PAGE_KERNEL, pages, PAGE_SHIFT) < 0) { @@ -2441,10 +2441,10 @@ static struct vm_struct *__get_vm_area_node(unsigned long size, return NULL; } - kasan_unpoison_vmalloc((void *)va->va_start, requested_size); - setup_vmalloc_vm(area, va, flags, caller); + area->addr = kasan_unpoison_vmalloc(area->addr, requested_size); + return area; } @@ -3752,9 +3752,6 @@ struct vm_struct **pcpu_get_vm_areas(const unsigned long *offsets, for (area = 0; area < nr_vms; area++) { if (kasan_populate_vmalloc(vas[area]->va_start, sizes[area])) goto err_free_shadow; - - kasan_unpoison_vmalloc((void *)vas[area]->va_start, - sizes[area]); } /* insert all vm's */ @@ -3767,6 +3764,11 @@ struct vm_struct **pcpu_get_vm_areas(const unsigned long *offsets, } spin_unlock(&vmap_area_lock); + /* mark allocated areas as accessible */ + for (area = 0; area < nr_vms; area++) + vms[area]->addr = kasan_unpoison_vmalloc(vms[area]->addr, + vms[area]->size); + kfree(vas); return vms; From patchwork Mon Dec 6 21:44:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695009 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0F1CCC433F5 for ; Mon, 6 Dec 2021 22:02:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=15jRECNd/K1hhwRrL1VBn5ULNJPkMVGdvb2W0Bsw8Oc=; b=Bu4DlMlN/uUgwL FLipAPUeS6JqxG21aw9cNrNR/4jxxoGrWFUPgOPD+yNxs/UbDoBa6Ks/BL1iWxPHiUPf2v8lH+HZy Mmn0vLwGre4XOuJiOXjO5NAymmWTj7ua9v9ZNayZyrCjBH+0acesjqP6JPbDEy0awaaWqFS1HeJVt 9Pj+cLrV4isJzppQldfZ1WPCdbLlquJhejv/jXkdhxKR2NSAataAqTNED4AGXLcZ1YQNTv27YcIob jQkZQfNSaP8DjpUYq2uBRASOt+Awxz9nKrKxzE2yfcXJybw0TGl6g+zn6VlY+aUijLjuQEIJwyAOQ 1X3JdWj+C2xVOadRrc1A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muM2P-0065T0-7A; Mon, 06 Dec 2021 22:00:46 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLoY-005zQz-8s for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:46:27 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827183; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kpH+33k/GfROSS5YE3KBLshkOEFVObuKhNsDDFbF7nk=; b=AJN2EJAaDdUYomCCG15rAW68PpaWtux6TdzBqwlVip4UT4P2FCHgshWjhjUQEu14AbIBc9 WAouu69YfoInvm0k3MYZvbV6mVH5PbIMIJH0E/kSYdArl7NH3cgxLONQ4fvAJDF8/3VWvT eIFFtCTeWtCVQkyGgIJ6R9B2yOwt5qs= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 24/34] kasan, vmalloc, arm64: mark vmalloc mappings as pgprot_tagged Date: Mon, 6 Dec 2021 22:44:01 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134626_520981_C74B3FFA X-CRM114-Status: UNSURE ( 9.79 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov HW_TAGS KASAN relies on ARM Memory Tagging Extension (MTE). With MTE, a memory region must be mapped as MT_NORMAL_TAGGED to allow setting memory tags via MTE-specific instructions. This change adds proper protection bits to vmalloc() allocations. These allocations are always backed by page_alloc pages, so the tags will actually be getting set on the corresponding physical memory. Signed-off-by: Andrey Konovalov Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino --- arch/arm64/include/asm/vmalloc.h | 10 ++++++++++ include/linux/vmalloc.h | 7 +++++++ mm/vmalloc.c | 2 ++ 3 files changed, 19 insertions(+) diff --git a/arch/arm64/include/asm/vmalloc.h b/arch/arm64/include/asm/vmalloc.h index b9185503feae..3d35adf365bf 100644 --- a/arch/arm64/include/asm/vmalloc.h +++ b/arch/arm64/include/asm/vmalloc.h @@ -25,4 +25,14 @@ static inline bool arch_vmap_pmd_supported(pgprot_t prot) #endif +#define arch_vmalloc_pgprot_modify arch_vmalloc_pgprot_modify +static inline pgprot_t arch_vmalloc_pgprot_modify(pgprot_t prot) +{ + if (IS_ENABLED(CONFIG_KASAN_HW_TAGS) && + (pgprot_val(prot) == pgprot_val(PAGE_KERNEL))) + prot = pgprot_tagged(prot); + + return prot; +} + #endif /* _ASM_ARM64_VMALLOC_H */ diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index b22369f540eb..965c4bf475f1 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -108,6 +108,13 @@ static inline int arch_vmap_pte_supported_shift(unsigned long size) } #endif +#ifndef arch_vmalloc_pgprot_modify +static inline pgprot_t arch_vmalloc_pgprot_modify(pgprot_t prot) +{ + return prot; +} +#endif + /* * Highlevel APIs for driver use */ diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 7be18b292679..f37d0ed99bf9 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -3033,6 +3033,8 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align, return NULL; } + prot = arch_vmalloc_pgprot_modify(prot); + if (vmap_allow_huge && !(vm_flags & VM_NO_HUGE_VMAP)) { unsigned long size_per_node; From patchwork Mon Dec 6 21:44:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695010 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 23E62C433F5 for ; Mon, 6 Dec 2021 22:03:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=/6v1WX73ZqPcywhE0+3fjIIJPZ/CyJE/ibcLmOLYEms=; b=h4t5keqkXzz8bZ JGR0s6v/v5bXkwRj56niiwh3w+gU7lEP5tNtBJOUXAu1HVPp4B/GSoCfzHjaUW+YOoLEjbb2fMnm4 ETc7tv1ujxbZfyUMdQZyS92Hm+eATtsMmFyN0hMxi1qz3WaMVZZhilffj+Asi8qopcdeOc42fa574 lkzGf+a33KOtKfGmMjRaGjacGSCmye0J6/Nr7OCA5E3mHxm+cO2GMGsGI/qAM2R1rmvl2U6rdFelT DiXPOuVgwVkuEFJoA4YGNmT1D55f0ZNS3qq/ggPl7GLm1rSwG+S67jlcQrlHNYyk04ZlJdf0Y+Etb XiJT6so88kR+EEKhsJrg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muM3K-0065ng-PM; Mon, 06 Dec 2021 22:01:43 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLoe-005zUA-43 for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:46:33 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827190; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CceoAuXc8ztt6c/41GK5sR1YQRKzgrFVIEpBFl753js=; b=M9OKKoZeQ9QP7msj10cO90weY6/5kMTde7yVB3UKk03n8tkNr3GzxhxMhOwyTPDJ3eYzFP c//+1GM2DRuo/ZAQo8QWKTzW4/gPIXUbKXbm+qCzXBgo0y6nEomsoaSQ6VtyzAaAumgxNt ya1i1WBCg5uK+FnuH9zrz5God4EzzGg= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 25/34] kasan, vmalloc: don't unpoison VM_ALLOC pages before mapping Date: Mon, 6 Dec 2021 22:44:02 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134632_366511_1C3587EF X-CRM114-Status: GOOD ( 13.99 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov This patch makes KASAN unpoison vmalloc mappings after that have been mapped in when it's possible: for vmalloc() (indentified via VM_ALLOC) and vm_map_ram(). The reasons for this are: - For vmalloc() and vm_map_ram(): pages don't get unpoisoned in case mapping them fails. - For vmalloc(): HW_TAGS KASAN needs pages to be mapped to set tags via kasan_unpoison_vmalloc(). Signed-off-by: Andrey Konovalov --- mm/vmalloc.c | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index f37d0ed99bf9..82ef1e27e2e4 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2208,14 +2208,15 @@ void *vm_map_ram(struct page **pages, unsigned int count, int node) mem = (void *)addr; } - mem = kasan_unpoison_vmalloc(mem, size); - if (vmap_pages_range(addr, addr + size, PAGE_KERNEL, pages, PAGE_SHIFT) < 0) { vm_unmap_ram(mem, count); return NULL; } + /* Mark the pages as accessible after they were mapped in. */ + mem = kasan_unpoison_vmalloc(mem, size); + return mem; } EXPORT_SYMBOL(vm_map_ram); @@ -2443,7 +2444,14 @@ static struct vm_struct *__get_vm_area_node(unsigned long size, setup_vmalloc_vm(area, va, flags, caller); - area->addr = kasan_unpoison_vmalloc(area->addr, requested_size); + /* + * For VM_ALLOC mappings, __vmalloc_node_range() mark the pages as + * accessible after they are mapped in. + * Otherwise, as the pages can be mapped outside of vmalloc code, + * mark them now as a best-effort approach. + */ + if (!(flags & VM_ALLOC)) + area->addr = kasan_unpoison_vmalloc(area->addr, requested_size); return area; } @@ -3072,6 +3080,12 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align, if (!addr) goto fail; + /* + * Mark the pages for VM_ALLOC mappings as accessible after they were + * mapped in. + */ + addr = kasan_unpoison_vmalloc(addr, real_size); + /* * In this function, newly allocated vm_struct has VM_UNINITIALIZED * flag. It means that vm_struct is not fully initialized. @@ -3766,7 +3780,11 @@ struct vm_struct **pcpu_get_vm_areas(const unsigned long *offsets, } spin_unlock(&vmap_area_lock); - /* mark allocated areas as accessible */ + /* + * Mark allocated areas as accessible. + * As the pages are mapped outside of vmalloc code, + * mark them now as a best-effort approach. + */ for (area = 0; area < nr_vms; area++) vms[area]->addr = kasan_unpoison_vmalloc(vms[area]->addr, vms[area]->size); From patchwork Mon Dec 6 21:44:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695011 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 84710C433EF for ; Mon, 6 Dec 2021 22:04:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=S4ssxB8wGpnAN5+g4KCxxx3wJ8EmzzbEIzRG3S9cglU=; b=QcPPUlba3n4u5q xdHizPdDxNzvRNQa31TJTqI1NyNjBBqLS3CyJvnLAOKemio7Sk7rCCt4KA/Ta1uPApQGrV2lssP6J axu/NA9k5Xl19mDlaDN2xNZh+dkwjYOqFUygI5UdOLbrD8dcX6fl3mBwK+jnHIu1cXUDX160LLjeK 4xcalWjnIX13sq99KMWFtJKOV6BTfxIyI44MpqFOZ1d4/0Mo2cir6ijHgYMFNoSmnFceGjmlRQ6iQ XwRVVfeXb2P9Ss8AfGc5QFZZ6oJ6G7Ldk2UbmY/DiBwvRxzNuV8qthjhww/vElqV45xRHOLOyGJuu wdg3KxhctdDqxkg9HpLQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muM47-006693-D2; Mon, 06 Dec 2021 22:02:31 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLoj-005zWl-KJ for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:46:39 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827196; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Psw3MnbE/X6is67FfglkhKemZX8bH5/LHATd5xYq6jA=; b=diPSXiCfLACAd0DqsXiD0bXyUQiNhxdRmeZoiW+g4q/SdEsANmZ0nh9ThUNWZxKx1eOO8M 64851O9H7oodtk6lJMDcs1AvOgM8GQpc3dd27i0hq08gmW8FJMFEfppcEfNJNEgWjoASjJ GtwTE7ttRrZfJ2FE1pHUF7BVIJbRi6A= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 26/34] kasan, page_alloc: allow skipping unpoisoning for HW_TAGS Date: Mon, 6 Dec 2021 22:44:03 +0100 Message-Id: <694654c29f4dddb3e927c264f71d032df6d906cd.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134638_033322_90BEA6F3 X-CRM114-Status: GOOD ( 17.99 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov This patch adds a new GFP flag __GFP_SKIP_KASAN_UNPOISON that allows skipping KASAN poisoning for page_alloc allocations. The flag is only effective with HW_TAGS KASAN. This flag will be used by vmalloc code for page_alloc allocations backing vmalloc() mappings in a following patch. The reason to skip KASAN poisoning for these pages in page_alloc is because vmalloc code will be poisoning them instead. This patch also rewords the comment for __GFP_SKIP_KASAN_POISON. Signed-off-by: Andrey Konovalov --- include/linux/gfp.h | 18 +++++++++++------- mm/page_alloc.c | 24 +++++++++++++++++------- 2 files changed, 28 insertions(+), 14 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index dddd7597689f..8a3083d4cbbe 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -54,9 +54,10 @@ struct vm_area_struct; #define ___GFP_THISNODE 0x200000u #define ___GFP_ACCOUNT 0x400000u #define ___GFP_ZEROTAGS 0x800000u -#define ___GFP_SKIP_KASAN_POISON 0x1000000u +#define ___GFP_SKIP_KASAN_UNPOISON 0x1000000u +#define ___GFP_SKIP_KASAN_POISON 0x2000000u #ifdef CONFIG_LOCKDEP -#define ___GFP_NOLOCKDEP 0x2000000u +#define ___GFP_NOLOCKDEP 0x4000000u #else #define ___GFP_NOLOCKDEP 0 #endif @@ -235,21 +236,24 @@ struct vm_area_struct; * %__GFP_ZEROTAGS zeroes memory tags at allocation time if the memory itself * is being zeroed (either via __GFP_ZERO or via init_on_alloc). * - * %__GFP_SKIP_KASAN_POISON returns a page which does not need to be poisoned - * on deallocation. Typically used for userspace pages. Currently only has an - * effect in HW tags mode. + * %__GFP_SKIP_KASAN_UNPOISON makes KASAN skip unpoisoning on page allocation. + * Only effective in HW_TAGS mode. + * + * %__GFP_SKIP_KASAN_POISON makes KASAN skip poisoning on page deallocation. + * Typically, used for userspace pages. Only effective in HW_TAGS mode. */ #define __GFP_NOWARN ((__force gfp_t)___GFP_NOWARN) #define __GFP_COMP ((__force gfp_t)___GFP_COMP) #define __GFP_ZERO ((__force gfp_t)___GFP_ZERO) #define __GFP_ZEROTAGS ((__force gfp_t)___GFP_ZEROTAGS) -#define __GFP_SKIP_KASAN_POISON ((__force gfp_t)___GFP_SKIP_KASAN_POISON) +#define __GFP_SKIP_KASAN_UNPOISON ((__force gfp_t)___GFP_SKIP_KASAN_UNPOISON) +#define __GFP_SKIP_KASAN_POISON ((__force gfp_t)___GFP_SKIP_KASAN_POISON) /* Disable lockdep for GFP context tracking */ #define __GFP_NOLOCKDEP ((__force gfp_t)___GFP_NOLOCKDEP) /* Room for N __GFP_FOO bits */ -#define __GFP_BITS_SHIFT (25 + IS_ENABLED(CONFIG_LOCKDEP)) +#define __GFP_BITS_SHIFT (26 + IS_ENABLED(CONFIG_LOCKDEP)) #define __GFP_BITS_MASK ((__force gfp_t)((1 << __GFP_BITS_SHIFT) - 1)) /** diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 73e6500c9767..7065d0e763e9 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2380,6 +2380,21 @@ static bool check_new_pages(struct page *page, unsigned int order) return false; } +static inline bool should_skip_kasan_unpoison(gfp_t flags, bool init_tags) +{ + /* Don't skip if a software KASAN mode is enabled. */ + if (!IS_ENABLED(CONFIG_KASAN_HW_TAGS)) + return false; + + /* + * For hardware tag-based KASAN, skip if either: + * + * 1. Memory tags have already been cleared via tag_clear_highpage(). + * 2. Skipping has been requested via __GFP_SKIP_KASAN_UNPOISON. + */ + return init_tags || (flags & __GFP_SKIP_KASAN_UNPOISON); +} + inline void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flags) { @@ -2419,13 +2434,8 @@ inline void post_alloc_hook(struct page *page, unsigned int order, /* Note that memory is already initialized by the loop above. */ init = false; } - /* - * If either a software KASAN mode is enabled, or, - * in the case of hardware tag-based KASAN, - * if memory tags have not been cleared via tag_clear_highpage(). - */ - if (!IS_ENABLED(CONFIG_KASAN_HW_TAGS) || !init_tags) { - /* Mark shadow memory or set memory tags. */ + if (!should_skip_kasan_unpoison(gfp_flags, init_tags)) { + /* Unpoison shadow memory or set memory tags. */ kasan_unpoison_pages(page, order, init); /* Note that memory is already initialized by KASAN. */ From patchwork Mon Dec 6 21:44:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695012 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 20A65C433EF for ; Mon, 6 Dec 2021 22:05:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=MSgQQ+XL7Yc5+CiZPiOYqwb3clQ22SNR8m5G6PO/SpE=; b=FeXSx3d8z2ltoc gQ6HOvTt993COuqf3qfV+8b79xOCPm1fkdp8kOrvHzO6C3EJ3bOgA2e1BnkRnT0bimZ3Nwv/4QzOB lV4ONb1mxvUZqPRwVT5wNuBtwS3myxTWjbf2QtwI3e6cg8oDvWSA/8CRkQ4rDY8H/+t3cLy/kS7R1 WLY3b7quCeWN24Mfq/dDuxHwSo43Z/gU4a/EJQ3JAne0s6mLoIF9D3aF4mKMCpz3JcMgU2xMwnSMi yyd21BTs9V1ATJAMz0Z2Wp45w+74RWt4J7V1UVkPxLrIEpxkr6tmKl6fcwEbI7/98K34k9CYgD9DT itlUZdznRQX1vvrgQpNg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muM4o-0066Sa-Bm; Mon, 06 Dec 2021 22:03:16 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLop-005zZT-EU for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:46:45 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827202; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MyllHPB10Kx40eayZCcKDlRSKUaneC5WeW2VZ41vX/A=; b=dv+XWoVf6gjaCwO2DL4qZFt6eP8mFDzhuetkuwBy1ChYmJnnbWiZiTclE5urBgNyU+T04P shbTjgddT63AYOPW39jx2q7K2ltD/3tisyOK40V1N2V6GzgfUB5DrXE93psv0u5AD5a+Aq sXFB6iFIRmHt3GwoeV5F8ZRskWozokc= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 27/34] kasan, page_alloc: allow skipping memory init for HW_TAGS Date: Mon, 6 Dec 2021 22:44:04 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134643_850787_59AE2467 X-CRM114-Status: GOOD ( 17.94 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov This patch adds a new GFP flag __GFP_SKIP_ZERO that allows to skip memory initialization. The flag is only effective with HW_TAGS KASAN. This flag will be used by vmalloc code for page_alloc allocations backing vmalloc() mappings in a following patch. The reason to skip memory initialization for these pages in page_alloc is because vmalloc code will be initializing them instead. With the current implementation, when __GFP_SKIP_ZERO is provided, __GFP_ZEROTAGS is ignored. This doesn't matter, as these two flags are never provided at the same time. However, if this is changed in the future, this particular implementation detail can be changed as well. Signed-off-by: Andrey Konovalov --- Changes v1->v2: - This is a new patch. --- include/linux/gfp.h | 16 +++++++++++----- mm/page_alloc.c | 13 ++++++++++++- 2 files changed, 23 insertions(+), 6 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 8a3083d4cbbe..5dbde04e8e7b 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -54,10 +54,11 @@ struct vm_area_struct; #define ___GFP_THISNODE 0x200000u #define ___GFP_ACCOUNT 0x400000u #define ___GFP_ZEROTAGS 0x800000u -#define ___GFP_SKIP_KASAN_UNPOISON 0x1000000u -#define ___GFP_SKIP_KASAN_POISON 0x2000000u +#define ___GFP_SKIP_ZERO 0x1000000u +#define ___GFP_SKIP_KASAN_UNPOISON 0x2000000u +#define ___GFP_SKIP_KASAN_POISON 0x4000000u #ifdef CONFIG_LOCKDEP -#define ___GFP_NOLOCKDEP 0x4000000u +#define ___GFP_NOLOCKDEP 0x8000000u #else #define ___GFP_NOLOCKDEP 0 #endif @@ -234,7 +235,11 @@ struct vm_area_struct; * %__GFP_ZERO returns a zeroed page on success. * * %__GFP_ZEROTAGS zeroes memory tags at allocation time if the memory itself - * is being zeroed (either via __GFP_ZERO or via init_on_alloc). + * is being zeroed (either via __GFP_ZERO or via init_on_alloc, provided that + * __GFP_SKIP_ZERO is not set). + * + * %__GFP_SKIP_ZERO makes page_alloc skip zeroing memory. + * Only effective when HW_TAGS KASAN is enabled. * * %__GFP_SKIP_KASAN_UNPOISON makes KASAN skip unpoisoning on page allocation. * Only effective in HW_TAGS mode. @@ -246,6 +251,7 @@ struct vm_area_struct; #define __GFP_COMP ((__force gfp_t)___GFP_COMP) #define __GFP_ZERO ((__force gfp_t)___GFP_ZERO) #define __GFP_ZEROTAGS ((__force gfp_t)___GFP_ZEROTAGS) +#define __GFP_SKIP_ZERO ((__force gfp_t)___GFP_SKIP_ZERO) #define __GFP_SKIP_KASAN_UNPOISON ((__force gfp_t)___GFP_SKIP_KASAN_UNPOISON) #define __GFP_SKIP_KASAN_POISON ((__force gfp_t)___GFP_SKIP_KASAN_POISON) @@ -253,7 +259,7 @@ struct vm_area_struct; #define __GFP_NOLOCKDEP ((__force gfp_t)___GFP_NOLOCKDEP) /* Room for N __GFP_FOO bits */ -#define __GFP_BITS_SHIFT (26 + IS_ENABLED(CONFIG_LOCKDEP)) +#define __GFP_BITS_SHIFT (27 + IS_ENABLED(CONFIG_LOCKDEP)) #define __GFP_BITS_MASK ((__force gfp_t)((1 << __GFP_BITS_SHIFT) - 1)) /** diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 7065d0e763e9..366b08b761ee 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2395,10 +2395,21 @@ static inline bool should_skip_kasan_unpoison(gfp_t flags, bool init_tags) return init_tags || (flags & __GFP_SKIP_KASAN_UNPOISON); } +static inline bool should_skip_init(gfp_t flags) +{ + /* Don't skip if a software KASAN mode is enabled. */ + if (!IS_ENABLED(CONFIG_KASAN_HW_TAGS)) + return false; + + /* For hardware tag-based KASAN, skip if requested. */ + return (flags & __GFP_SKIP_ZERO); +} + inline void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flags) { - bool init = !want_init_on_free() && want_init_on_alloc(gfp_flags); + bool init = !want_init_on_free() && want_init_on_alloc(gfp_flags) && + !should_skip_init(gfp_flags); bool init_tags = init && (gfp_flags & __GFP_ZEROTAGS); set_page_private(page, 0); From patchwork Mon Dec 6 21:44:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695013 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 60630C433F5 for ; Mon, 6 Dec 2021 22:05:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=aG04ODlS0DFd2W7nZ/fm5HXF37MMI7qHV2ffG4InqP8=; b=Xty1GTM1k2SRiz WR70r73ZUsWRBKiZLB/8GAY47l05bx+veM9AhYqgB6bSw/owIMsYaGazispX7RRyE8OrUMkPa9FIy DpugKozfH1z/n8Xplqc3K/cbGbhSlw8zxn3JMHI4iV8rrucwCHhtccjNfmk8V9eLcMFBePqlHw8pi U8rcfObNQvMpcutdlXeNJ0MBtpm4hcGQcua9uRqxgdRMlMf4k8VrJbZ1aBz2LqPlW68I9ZTS+OWZz DSDrORASSuX1+zhF7K5D6VfUnzPHj5IWGfBXPkJLlFve2H7Y4mchZT5XuZ4+Yc2sCoU/DPL6zBP20 3bBtY8ayx14AFaK1UFYQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muM5g-0066qv-Ji; Mon, 06 Dec 2021 22:04:09 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLot-005zbU-Fi for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:46:50 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827206; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=74CNvwwh68OzFRGu8FIPa2yQlRoKG2WJZYC0+RUr6EA=; b=DPBuz+qZ6VGFSO38WzJsbibEAHaOipRDeIIBWN50AAXPfCFSPZvTf67UtobREPFYVOGdr4 gTJrEKR0T7ctUjT8Rn0yQ9KJt1e30ou9T9QfInGFeFtMhnJHG62Fo6G2vPHgCCoy0xtVOP sFujYSJ7hokqav2cpoeXyhXgHglid1k= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 28/34] kasan, vmalloc: add vmalloc support to HW_TAGS Date: Mon, 6 Dec 2021 22:44:05 +0100 Message-Id: <72a8a7aa09eb279d7eabf7ea1101556d13360950.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134647_905254_06C2914D X-CRM114-Status: GOOD ( 30.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov This patch adds vmalloc tagging support to HW_TAGS KASAN. The key difference between HW_TAGS and the other two KASAN modes when it comes to vmalloc: HW_TAGS KASAN can only assign tags to physical memory. The other two modes have shadow memory covering every mapped virtual memory region. This patch makes __kasan_unpoison_vmalloc() for HW_TAGS KASAN: - Skip non-VM_ALLOC mappings as HW_TAGS KASAN can only tag a single mapping of normal physical memory; see the comment in the function. - Generate a random tag, tag the returned pointer and the allocation, and initialize the allocation at the same time. - Propagate the tag into the page stucts to allow accesses through page_address(vmalloc_to_page()). The rest of vmalloc-related KASAN hooks are not needed: - The shadow-related ones are fully skipped. - __kasan_poison_vmalloc() is kept as a no-op with a comment. Poisoning and zeroing of physical pages that are backing vmalloc() allocations are skipped via __GFP_SKIP_KASAN_UNPOISON and __GFP_SKIP_ZERO: __kasan_unpoison_vmalloc() does that instead. This patch allows enabling CONFIG_KASAN_VMALLOC with HW_TAGS and adjusts CONFIG_KASAN_VMALLOC description: - Mention HW_TAGS support. - Remove unneeded internal details: they have no place in Kconfig description and are already explained in the documentation. Signed-off-by: Andrey Konovalov Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino --- Changes v1->v2: - Allow enabling CONFIG_KASAN_VMALLOC with HW_TAGS in this patch. - Move memory init for page_alloc pages backing vmalloc() into kasan_unpoison_vmalloc(). --- include/linux/kasan.h | 30 +++++++++++++-- lib/Kconfig.kasan | 20 +++++----- mm/kasan/hw_tags.c | 89 +++++++++++++++++++++++++++++++++++++++++++ mm/kasan/shadow.c | 11 +++++- mm/vmalloc.c | 32 +++++++++++++--- 5 files changed, 162 insertions(+), 20 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 6a2619759e93..0bdc2b824b9c 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -417,19 +417,40 @@ static inline void kasan_init_hw_tags(void) { } #ifdef CONFIG_KASAN_VMALLOC +#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) + void kasan_populate_early_vm_area_shadow(void *start, unsigned long size); int kasan_populate_vmalloc(unsigned long addr, unsigned long size); void kasan_release_vmalloc(unsigned long start, unsigned long end, unsigned long free_region_start, unsigned long free_region_end); +#else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ + +static inline void kasan_populate_early_vm_area_shadow(void *start, + unsigned long size) +{ } +static inline int kasan_populate_vmalloc(unsigned long start, + unsigned long size) +{ + return 0; +} +static inline void kasan_release_vmalloc(unsigned long start, + unsigned long end, + unsigned long free_region_start, + unsigned long free_region_end) { } + +#endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ + void * __must_check __kasan_unpoison_vmalloc(const void *start, - unsigned long size); + unsigned long size, + bool vm_alloc, bool init); static __always_inline void * __must_check kasan_unpoison_vmalloc( - const void *start, unsigned long size) + const void *start, unsigned long size, + bool vm_alloc, bool init) { if (kasan_enabled()) - return __kasan_unpoison_vmalloc(start, size); + return __kasan_unpoison_vmalloc(start, size, vm_alloc, init); return (void *)start; } @@ -456,7 +477,8 @@ static inline void kasan_release_vmalloc(unsigned long start, unsigned long free_region_end) { } static inline void *kasan_unpoison_vmalloc(const void *start, - unsigned long size, bool unique) + unsigned long size, + bool vm_alloc, bool init) { return (void *)start; } diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan index 3f144a87f8a3..7834c35a7964 100644 --- a/lib/Kconfig.kasan +++ b/lib/Kconfig.kasan @@ -178,17 +178,17 @@ config KASAN_TAGS_IDENTIFY memory consumption. config KASAN_VMALLOC - bool "Back mappings in vmalloc space with real shadow memory" - depends on (KASAN_GENERIC || KASAN_SW_TAGS) && HAVE_ARCH_KASAN_VMALLOC + bool "Check accesses to vmalloc allocations" + depends on HAVE_ARCH_KASAN_VMALLOC help - By default, the shadow region for vmalloc space is the read-only - zero page. This means that KASAN cannot detect errors involving - vmalloc space. - - Enabling this option will hook in to vmap/vmalloc and back those - mappings with real shadow memory allocated on demand. This allows - for KASAN to detect more sorts of errors (and to support vmapped - stacks), but at the cost of higher memory usage. + This mode makes KASAN check accesses to vmalloc allocations for + validity. + + With software KASAN modes, checking is done for all types of vmalloc + allocations. Enabling this option leads to higher memory usage. + + With hardware tag-based KASAN, only VM_ALLOC mappings are checked. + There is no additional memory usage. config KASAN_KUNIT_TEST tristate "KUnit-compatible tests of KASAN bug detection capabilities" if !KUNIT_ALL_TESTS diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 76cf2b6229c7..837c260beec6 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -192,6 +192,95 @@ void __init kasan_init_hw_tags(void) kasan_stack_collection_enabled() ? "on" : "off"); } +#ifdef CONFIG_KASAN_VMALLOC + +static void unpoison_vmalloc_pages(const void *addr, u8 tag) +{ + struct vm_struct *area; + int i; + + /* + * As hardware tag-based KASAN only tags VM_ALLOC vmalloc allocations + * (see the comment in __kasan_unpoison_vmalloc), all of the pages + * should belong to a single area. + */ + area = find_vm_area((void *)addr); + if (WARN_ON(!area)) + return; + + for (i = 0; i < area->nr_pages; i++) { + struct page *page = area->pages[i]; + + page_kasan_tag_set(page, tag); + } +} + +void *__kasan_unpoison_vmalloc(const void *start, unsigned long size, + bool vm_alloc, bool init) +{ + u8 tag; + unsigned long redzone_start, redzone_size; + + if (!is_vmalloc_or_module_addr(start)) + return (void *)start; + + /* Unpoisoning and pointer tag assignment is skipped for non-VM_ALLOC + * mappings as: + * + * 1. Unlike the software KASAN modes, hardware tag-based KASAN only + * supports tagging physical memory. Therefore, it can only tag a + * single mapping of normal physical pages. + * 2. Hardware tag-based KASAN can only tag memory mapped with special + * mapping protection bits, see arch_vmalloc_pgprot_modify(). + * As non-VM_ALLOC mappings can be mapped outside of vmalloc code, + * providing these bits would require tracking all non-VM_ALLOC + * mappers. + * + * Thus, for VM_ALLOC mappings, hardware tag-based KASAN only tags + * the first virtual mapping, which is created by vmalloc(). + * Tagging the page_alloc memory backing that vmalloc() allocation is + * skipped, see ___GFP_SKIP_KASAN_UNPOISON. + * + * For non-VM_ALLOC allocations, page_alloc memory is tagged as usual. + */ + if (!vm_alloc) + return (void *)start; + + tag = kasan_random_tag(); + start = set_tag(start, tag); + + /* Unpoison and initialize memory up to size. */ + kasan_unpoison(start, size, init); + + /* + * Explicitly poison and initialize the in-page vmalloc() redzone. + * Unlike software KASAN modes, hardware tag-based KASAN doesn't + * unpoison memory when populating shadow for vmalloc() space. + */ + redzone_start = round_up((unsigned long)start + size, KASAN_GRANULE_SIZE); + redzone_size = round_up(redzone_start, PAGE_SIZE) - redzone_start; + kasan_poison((void *)redzone_start, redzone_size, KASAN_TAG_INVALID, init); + + /* + * Set per-page tag flags to allow accessing physical memory for the + * vmalloc() mapping through page_address(vmalloc_to_page()). + */ + unpoison_vmalloc_pages(start, tag); + + return (void *)start; +} + +void __kasan_poison_vmalloc(const void *start, unsigned long size) +{ + /* + * No tagging here. + * The physical pages backing the vmalloc() allocation are poisoned + * through the usual page_alloc paths. + */ +} + +#endif + #if IS_ENABLED(CONFIG_KASAN_KUNIT_TEST) void kasan_enable_tagging_sync(void) diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index 4ca280a96fbc..8600dd925f35 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -475,8 +475,17 @@ void kasan_release_vmalloc(unsigned long start, unsigned long end, } } -void *__kasan_unpoison_vmalloc(const void *start, unsigned long size) +void *__kasan_unpoison_vmalloc(const void *start, unsigned long size, + bool vm_alloc, bool init) { + /* + * Software tag-based KASAN tags both VM_ALLOC and non-VM_ALLOC + * mappings, so the vm_alloc argument is ignored. + * Software tag-based KASAN can't optimize zeroing memory by combining + * it with setting memory tags, so the init argument is ignored; + * vmalloc() memory is poisoned via page_alloc. + */ + if (!is_vmalloc_or_module_addr(start)) return (void *)start; diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 82ef1e27e2e4..d48db7cc3358 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2214,8 +2214,12 @@ void *vm_map_ram(struct page **pages, unsigned int count, int node) return NULL; } - /* Mark the pages as accessible after they were mapped in. */ - mem = kasan_unpoison_vmalloc(mem, size); + /* + * Mark the pages as accessible after they were mapped in. + * With hardware tag-based KASAN, marking is skipped for + * non-VM_ALLOC mappings, see __kasan_unpoison_vmalloc(). + */ + mem = kasan_unpoison_vmalloc(mem, size, false, false); return mem; } @@ -2449,9 +2453,12 @@ static struct vm_struct *__get_vm_area_node(unsigned long size, * accessible after they are mapped in. * Otherwise, as the pages can be mapped outside of vmalloc code, * mark them now as a best-effort approach. + * With hardware tag-based KASAN, marking is skipped for + * non-VM_ALLOC mappings, see __kasan_unpoison_vmalloc(). */ if (!(flags & VM_ALLOC)) - area->addr = kasan_unpoison_vmalloc(area->addr, requested_size); + area->addr = kasan_unpoison_vmalloc(area->addr, requested_size, + false, false); return area; } @@ -2849,6 +2856,12 @@ vm_area_alloc_pages(gfp_t gfp, int nid, struct page *page; int i; + /* + * Skip page_alloc poisoning and zeroing for pages backing VM_ALLOC + * mappings. Only effective in HW_TAGS mode. + */ + gfp &= __GFP_SKIP_KASAN_UNPOISON & __GFP_SKIP_ZERO; + /* * For order-0 pages we make use of bulk allocator, if * the page array is partly or not at all populated due @@ -3027,6 +3040,7 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align, { struct vm_struct *area; void *addr; + bool init; unsigned long real_size = size; unsigned long real_align = align; unsigned int shift = PAGE_SHIFT; @@ -3083,8 +3097,13 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align, /* * Mark the pages for VM_ALLOC mappings as accessible after they were * mapped in. + * The init condition should match the one in post_alloc_hook() + * (except for the should_skip_init() check) to make sure that memory + * is initialized under the same conditions regardless of the enabled + * KASAN mode. */ - addr = kasan_unpoison_vmalloc(addr, real_size); + init = !want_init_on_free() && want_init_on_alloc(gfp_mask); + addr = kasan_unpoison_vmalloc(addr, real_size, true, init); /* * In this function, newly allocated vm_struct has VM_UNINITIALIZED @@ -3784,10 +3803,13 @@ struct vm_struct **pcpu_get_vm_areas(const unsigned long *offsets, * Mark allocated areas as accessible. * As the pages are mapped outside of vmalloc code, * mark them now as a best-effort approach. + * With hardware tag-based KASAN, marking is skipped for + * non-VM_ALLOC mappings, see __kasan_unpoison_vmalloc(). */ for (area = 0; area < nr_vms; area++) vms[area]->addr = kasan_unpoison_vmalloc(vms[area]->addr, - vms[area]->size); + vms[area]->size, + false, false); kfree(vas); return vms; From patchwork Mon Dec 6 21:44:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E0414C433F5 for ; Mon, 6 Dec 2021 22:06:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=gYiz8oDkLgyAOSW/w6ONM2BpVH+oW07DWXFTVauhtsA=; b=yFQhgzty9opfmn 5Pu0Y8WiIs5ItVZqLHPTiCZB6aSx5Hc1iA5JSGfa0NMJyS/VQyMgpKTcOFw6CrcB8MNtnTg6cStNg xGDrx8wzbzSMmVH6G5JXYLbuQ/H5ZovZ6aG5JPHCzNoxI6AWJJmJRB3mPa8w8Wp9EvAmqy/Qc86vc ZrJ34o5G59IIL9xPKHOYuh9x1cp39PA5/fMynkMZg3r+GLHXgCGSFukH6efr6mMIPYboxUE7EKSLh jN+Tx+bHDhYyMS7Z1ra8DLxsQqxCM7gyemYG1D2p0pTPFMFHF/ER2q0KA17KMFJTNnAbIOMzIz06v vWZRAbY5P/MFMcn8JDCA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muM6Z-0067D1-D5; Mon, 06 Dec 2021 22:05:03 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLox-005zdm-UL for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:46:53 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827210; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=L9vUKL8lfozP3PwlMvZv+xcgDLHHdbGfzGfMbX3NgM0=; b=WPzPRi878VdXo+EXxktw0jjLigh3pY4a3cVf5B+ywmR5yqCFLTOz5aF89G+9Np2w077oJp NyXVIx6bVYtxzJlcBgjm1ZnFuh3cG12pnwAXY9xBzCh7kqm3VSZBFAyByXCbIDqeznX1QP YUqTLQBg+5C7OFQRwiJQ06L740TxsS8= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 29/34] kasan: mark kasan_arg_stacktrace as __initdata Date: Mon, 6 Dec 2021 22:44:06 +0100 Message-Id: <31889507c217774d2b24fd45c63fdc4855a98c76.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134652_187789_034E29DE X-CRM114-Status: GOOD ( 11.32 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov As kasan_arg_stacktrace is only used in __init functions, mark it as __initdata instead of __ro_after_init to allow it be freed after boot. The other enums for KASAN args are used in kasan_init_hw_tags_cpu(), which is not marked as __init as a CPU can be hot-plugged after boot. Clarify this in a comment. Signed-off-by: Andrey Konovalov Suggested-by: Marco Elver --- Changes v1->v2: - Add this patch. --- mm/kasan/hw_tags.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 837c260beec6..983ae15ed4f0 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -40,7 +40,7 @@ enum kasan_arg_stacktrace { static enum kasan_arg kasan_arg __ro_after_init; static enum kasan_arg_mode kasan_arg_mode __ro_after_init; -static enum kasan_arg_stacktrace kasan_arg_stacktrace __ro_after_init; +static enum kasan_arg_stacktrace kasan_arg_stacktrace __initdata; /* Whether KASAN is enabled at all. */ DEFINE_STATIC_KEY_FALSE(kasan_flag_enabled); @@ -116,7 +116,10 @@ static inline const char *kasan_mode_info(void) return "sync"; } -/* kasan_init_hw_tags_cpu() is called for each CPU. */ +/* + * kasan_init_hw_tags_cpu() is called for each CPU. + * Not marked as __init as a CPU can be hot-plugged after boot. + */ void kasan_init_hw_tags_cpu(void) { /* From patchwork Mon Dec 6 21:44:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695015 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0B3DDC433EF for ; Mon, 6 Dec 2021 22:07:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=1M3l4M08u7THwjDikfLfcmABnqg0iU73HyMgN+o7l94=; b=ExGnTk8t5B45rq DtpS0bgu1v0CF1WiRlOE0qmelmEDdkaoLbJs6BGwBs2e+ujdJYFMGDTQV05j82jwNmwOeqBy8EBj+ cXDjxTF3h8ogGd+9KpujZ/UiOiB7oEEMiIeUQZM+Po96nyV81Xov5xYe57i+vmO8dkTwLrew5hcHw 08GLSYw8gfT4R64WAa6ajTejIUQpMHqSRXZ9yOUdH3rcvxC89C+t0Pdp4CEYzPptMQNcLxZjgm8yf jk1MSC5hFZzOeWsK0327CTKnDe63bpKWf9WS5OgL4tSsFMn5+E7kgRQB+FybG67PwhU7ZVO6dg6wt Dn1oTA8KfKKuYZrNwAZw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muM7Q-0067c6-D1; Mon, 06 Dec 2021 22:05:57 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLp5-005zh5-8w for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:47:00 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827217; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H+x4/GNYyuJ4olLesp51NHArgpf+Wp1NvuW2iXEozGc=; b=mAzNzcsSBYCdZLIQLvxhgKJyEv+94RCDE0NW/I8ThfTBTWQcoPwEV4ejLqmoraovWDrHXN QG4PRjKrnYOllVYAWn2+ezBBymFuz/h/YuzxEU5k0u0tprs2rmuL+VOuSYpfyXGkYHvRzF Vn3XzrJp1oVgI+s0T6UvX/AGAQAemqk= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 30/34] kasan: simplify kasan_init_hw_tags Date: Mon, 6 Dec 2021 22:44:07 +0100 Message-Id: <1de4bcc3b3f7da3574a2e8e3f6dab48f47aa03b5.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134659_518230_1BAE2337 X-CRM114-Status: GOOD ( 10.10 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Simplify kasan_init_hw_tags(): - Remove excessive comments in kasan_arg_mode switch. - Combine DEFAULT and ON cases in kasan_arg_stacktrace switch. Signed-off-by: Andrey Konovalov --- Changes v1->v2: - Add this patch. --- mm/kasan/hw_tags.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 983ae15ed4f0..e12f2d195cc9 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -159,20 +159,15 @@ void __init kasan_init_hw_tags(void) switch (kasan_arg_mode) { case KASAN_ARG_MODE_DEFAULT: - /* - * Default to sync mode. - */ + /* Default to sync mode. */ fallthrough; case KASAN_ARG_MODE_SYNC: - /* Sync mode enabled. */ kasan_mode = KASAN_MODE_SYNC; break; case KASAN_ARG_MODE_ASYNC: - /* Async mode enabled. */ kasan_mode = KASAN_MODE_ASYNC; break; case KASAN_ARG_MODE_ASYMM: - /* Asymm mode enabled. */ kasan_mode = KASAN_MODE_ASYMM; break; } @@ -180,14 +175,13 @@ void __init kasan_init_hw_tags(void) switch (kasan_arg_stacktrace) { case KASAN_ARG_STACKTRACE_DEFAULT: /* Default to enabling stack trace collection. */ + fallthrough; + case KASAN_ARG_STACKTRACE_ON: static_branch_enable(&kasan_flag_stacktrace); break; case KASAN_ARG_STACKTRACE_OFF: /* Do nothing, kasan_flag_stacktrace keeps its default value. */ break; - case KASAN_ARG_STACKTRACE_ON: - static_branch_enable(&kasan_flag_stacktrace); - break; } pr_info("KernelAddressSanitizer initialized (hw-tags, mode=%s, stacktrace=%s)\n", From patchwork Mon Dec 6 21:44:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6F2A7C433F5 for ; Mon, 6 Dec 2021 22:08:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=lcUhWhRt1a2U/3WI6bySCHxXG7FICV9pwkZuH1FdOLw=; b=myI5GV/488LeDS lA4ZtQTHViqBwC21yyChhE8qcgw2USU3CpQe4X2qcRZTkGXIv1Xwt2m/rO+046dwBWQP8CYFcLBCh fV2CmVNVs9cisgEzS0To5U5LLuoAp1n60CTzonkYuxKK4Nnf4uNi+B2pX+bsp+OJrcs9AGic3i4oq 9jQGiIrlr/Ja3J6n6shRvmVL77f2S0GIr6h3Q5S7trm3G2HzvgoacOybxnxqMC30K8o06+Fe18tCF 1rw21GgETZOUJW/Mi8bAbr7YNtqx42Xfr4jET0yiyR48RF32erhxvi+ZylWf5RBklpWOen/eewVAC 7pXG1Ff9oNF0NeLgBvwA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muM83-0067tM-Ve; Mon, 06 Dec 2021 22:06:36 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLpB-005zjg-2V for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:47:06 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827223; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1+9hzOYxgqQnSoACXB+891e/bOFhsq8xWG50JY0SyCM=; b=m1+VKGi09B9F3WZg+qNVG9hH1CuZ648d9lE+Ee2iDv0LVZ9dWAG6+b+ecOO7GttkfmII3b ybXaTYNa5/IBMevPeLGfgV2v56/fON6Vm0NWJ240PM2rYQKn5MP6vRheVEQyiFpJikqFRP GZXYsaE3GSXhu8NChYktpax9/wLgcro= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 31/34] kasan: add kasan.vmalloc command line flag Date: Mon, 6 Dec 2021 22:44:08 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134705_475393_6DC5F032 X-CRM114-Status: GOOD ( 14.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Allow disabling vmalloc() tagging for HW_TAGS KASAN via a kasan.vmalloc command line switch. This is a fail-safe switch intended for production systems that enable HW_TAGS KASAN. In case vmalloc() tagging ends up having an issue not detected during testing but that manifests in production, kasan.vmalloc allows to turn vmalloc() tagging off while leaving page_alloc/slab tagging on. Signed-off-by: Andrey Konovalov --- Changes v1->v2: - Mark kasan_arg_stacktrace as __initdata instead of __ro_after_init. - Combine KASAN_ARG_VMALLOC_DEFAULT and KASAN_ARG_VMALLOC_ON switch cases. --- mm/kasan/hw_tags.c | 45 ++++++++++++++++++++++++++++++++++++++++++++- mm/kasan/kasan.h | 6 ++++++ 2 files changed, 50 insertions(+), 1 deletion(-) diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index e12f2d195cc9..5683eeac7348 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -32,6 +32,12 @@ enum kasan_arg_mode { KASAN_ARG_MODE_ASYMM, }; +enum kasan_arg_vmalloc { + KASAN_ARG_VMALLOC_DEFAULT, + KASAN_ARG_VMALLOC_OFF, + KASAN_ARG_VMALLOC_ON, +}; + enum kasan_arg_stacktrace { KASAN_ARG_STACKTRACE_DEFAULT, KASAN_ARG_STACKTRACE_OFF, @@ -40,6 +46,7 @@ enum kasan_arg_stacktrace { static enum kasan_arg kasan_arg __ro_after_init; static enum kasan_arg_mode kasan_arg_mode __ro_after_init; +static enum kasan_arg_vmalloc kasan_arg_vmalloc __initdata; static enum kasan_arg_stacktrace kasan_arg_stacktrace __initdata; /* Whether KASAN is enabled at all. */ @@ -50,6 +57,9 @@ EXPORT_SYMBOL(kasan_flag_enabled); enum kasan_mode kasan_mode __ro_after_init; EXPORT_SYMBOL_GPL(kasan_mode); +/* Whether to enable vmalloc tagging. */ +DEFINE_STATIC_KEY_FALSE(kasan_flag_vmalloc); + /* Whether to collect alloc/free stack traces. */ DEFINE_STATIC_KEY_FALSE(kasan_flag_stacktrace); @@ -89,6 +99,23 @@ static int __init early_kasan_mode(char *arg) } early_param("kasan.mode", early_kasan_mode); +/* kasan.vmalloc=off/on */ +static int __init early_kasan_flag_vmalloc(char *arg) +{ + if (!arg) + return -EINVAL; + + if (!strcmp(arg, "off")) + kasan_arg_vmalloc = KASAN_ARG_VMALLOC_OFF; + else if (!strcmp(arg, "on")) + kasan_arg_vmalloc = KASAN_ARG_VMALLOC_ON; + else + return -EINVAL; + + return 0; +} +early_param("kasan.vmalloc", early_kasan_flag_vmalloc); + /* kasan.stacktrace=off/on */ static int __init early_kasan_flag_stacktrace(char *arg) { @@ -172,6 +199,18 @@ void __init kasan_init_hw_tags(void) break; } + switch (kasan_arg_vmalloc) { + case KASAN_ARG_VMALLOC_DEFAULT: + /* Default to enabling vmalloc tagging. */ + fallthrough; + case KASAN_ARG_VMALLOC_ON: + static_branch_enable(&kasan_flag_vmalloc); + break; + case KASAN_ARG_VMALLOC_OFF: + /* Do nothing, kasan_flag_vmalloc keeps its default value. */ + break; + } + switch (kasan_arg_stacktrace) { case KASAN_ARG_STACKTRACE_DEFAULT: /* Default to enabling stack trace collection. */ @@ -184,8 +223,9 @@ void __init kasan_init_hw_tags(void) break; } - pr_info("KernelAddressSanitizer initialized (hw-tags, mode=%s, stacktrace=%s)\n", + pr_info("KernelAddressSanitizer initialized (hw-tags, mode=%s, vmalloc=%s, stacktrace=%s)\n", kasan_mode_info(), + kasan_vmalloc_enabled() ? "on" : "off", kasan_stack_collection_enabled() ? "on" : "off"); } @@ -218,6 +258,9 @@ void *__kasan_unpoison_vmalloc(const void *start, unsigned long size, u8 tag; unsigned long redzone_start, redzone_size; + if (!kasan_vmalloc_enabled()) + return (void *)start; + if (!is_vmalloc_or_module_addr(start)) return (void *)start; diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 0827d74d0d87..b58a4547ec5a 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -12,6 +12,7 @@ #include #include "../slab.h" +DECLARE_STATIC_KEY_FALSE(kasan_flag_vmalloc); DECLARE_STATIC_KEY_FALSE(kasan_flag_stacktrace); enum kasan_mode { @@ -22,6 +23,11 @@ enum kasan_mode { extern enum kasan_mode kasan_mode __ro_after_init; +static inline bool kasan_vmalloc_enabled(void) +{ + return static_branch_likely(&kasan_flag_vmalloc); +} + static inline bool kasan_stack_collection_enabled(void) { return static_branch_unlikely(&kasan_flag_stacktrace); From patchwork Mon Dec 6 21:44:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695017 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1F2B2C433F5 for ; Mon, 6 Dec 2021 22:08:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=GE6kxmdXrCebR53TP64e4YknBhuFxULuFG06P/TCZik=; b=CAztIYDSlJjyFm lIfhlBOGidFr2yiFojnZDo2iicvSusL4QmgAnIf+NtizW0abIAiovvcXatTkiVbTv//CfJ4QEn4Qr anmZFFrR9aS91K+ksjlgjzFOzceoR/GujtH+HGoX3b6D0uN0rWaVIsyQkRmB7KwLKmSWRBv07oJ5G rY3/TjPKn1kli15JH48XTRtQaHbbx1WcdfV/aAtE/wMgCuisp9AD7Oqw30lXSlXz5kuLb0452ePCJ jg4JUknN3AVQ3OUNeLI42RLCJ4E0LyCJ2uesUJMwdzvOUdCoW7wqnHxvoqkAdJCiioH5/4bXlGiLB 2uPvmfmRsuy1S6pl8B9Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muM8c-00686C-1w; Mon, 06 Dec 2021 22:07:10 +0000 Received: from out2.migadu.com ([188.165.223.204]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLpI-005zmk-2S for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:47:13 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827230; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vhe5kEMNFRSnWjEMXar5wkGHtuo9NDKTQhUT0l8T7fU=; b=jRv2K2Ltk3F5wz4BzV2fccJSdYyc31RJmDOteBv/6VztOq/T+Cdioxa94cDFBW6Ax15zwl dVVKoIobsAYHqwIlB1Dmjg/h38gXHcZ0QytRMSJhPsqglzRtdH8RcYcuutAAQvS41BUqyy ndBkslAgLs45ddwWu5tiYcbgf/8lzzY= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 32/34] arm64: select KASAN_VMALLOC for SW/HW_TAGS modes Date: Mon, 6 Dec 2021 22:44:09 +0100 Message-Id: <4f56dd2bfaf945032a226f90141bb4f8e73959b7.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134712_310180_EA0FD9F6 X-CRM114-Status: GOOD ( 11.59 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Generic KASAN already selects KASAN_VMALLOC to allow VMAP_STACK to be selected unconditionally, see commit acc3042d62cb9 ("arm64: Kconfig: select KASAN_VMALLOC if KANSAN_GENERIC is enabled"). The same change is needed for SW_TAGS KASAN. HW_TAGS KASAN does not require enabling KASAN_VMALLOC for VMAP_STACK, they already work together as is. Still, selecting KASAN_VMALLOC still makes sense to make vmalloc() always protected. In case any bugs in KASAN's vmalloc() support are discovered, the command line kasan.vmalloc flag can be used to disable vmalloc() checking. This patch selects KASAN_VMALLOC for all KASAN modes for arm64. Signed-off-by: Andrey Konovalov Acked-by: Catalin Marinas --- Changes v1->v2: - Split out this patch. --- arch/arm64/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index c4207cf9bb17..f0aa434e3b7a 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -205,7 +205,7 @@ config ARM64 select IOMMU_DMA if IOMMU_SUPPORT select IRQ_DOMAIN select IRQ_FORCED_THREADING - select KASAN_VMALLOC if KASAN_GENERIC + select KASAN_VMALLOC if KASAN select MODULES_USE_ELF_RELA select NEED_DMA_MAP_STATE select NEED_SG_DMA_LENGTH From patchwork Mon Dec 6 21:44:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695018 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9E92DC433EF for ; Mon, 6 Dec 2021 22:09:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=vFgvXgXLTtcLlmP2hR0OpHBlXswc3uB8SzJMIJm3Y4g=; b=uOCR5VhX9z2/Ba fDtKr47w+6hmCcopzzrDMUjYRielOxzNVfUD27Sbc1XSafGOsPavKQdHkX/fWNU88kcqRRrjgsnn/ vvTEk6l78sZNqJtR3UF71I+SHQ1k00zPIaF5Iil1NNHbrhVPys9vjFWiKCKSOMeZhsyOlLtZCemUW DjTYz0vGzp8Qg7Gj7XVgHhMsS/a8DIi9Md4tWE9qEQVPdacpgdC10+hOiyKBfEhPw5XCckc8PV/b5 xMqqBdmQruw2leMvV0EngYCQoV/3yejDQjhifB55olFaKHtnYMeWyRkaZjWvdk4J/tIsMidGAS2F+ veBhuRpXShCxR89AMubA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muM98-0068KL-JX; Mon, 06 Dec 2021 22:07:42 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLpN-005zpa-SU for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:47:19 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827236; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AV0M9szKvSlfXhwybxJe1jA9V5abexpCELFOGxjWDjg=; b=AbHNSooVjpyjTe3n28e2qeKWOSnqnQZToUN0NB3Ok/5lzPMsEokA5mj+OvjlVTA12oKMj2 /MHikLO0YXIdSqWmN8jOTpSl3elzNCdpfkuhh7vXYb0fQ0qQK4rfsOSqI7fLnZqic2wDYx A/0fT96PN8xSzDrK7+RCWJFJh2DXKMo= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 33/34] kasan: documentation updates Date: Mon, 6 Dec 2021 22:44:10 +0100 Message-Id: In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134718_134192_EDBD026C X-CRM114-Status: GOOD ( 14.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Update KASAN documentation: - Bump Clang version requirement for HW_TAGS as ARM64_MTE depends on AS_HAS_LSE_ATOMICS as of commit 2decad92f4731 ("arm64: mte: Ensure TIF_MTE_ASYNC_FAULT is set atomically"), which requires Clang 12. - Add description of the new kasan.vmalloc command line flag. - Mention that SW_TAGS and HW_TAGS modes now support vmalloc tagging. - Explicitly say that the "Shadow memory" section is only applicable to software KASAN modes. - Mention that shadow-based KASAN_VMALLOC is supported on arm64. Signed-off-by: Andrey Konovalov --- Documentation/dev-tools/kasan.rst | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/Documentation/dev-tools/kasan.rst b/Documentation/dev-tools/kasan.rst index 8089c559d339..7614a1fc30fa 100644 --- a/Documentation/dev-tools/kasan.rst +++ b/Documentation/dev-tools/kasan.rst @@ -30,7 +30,7 @@ Software tag-based KASAN mode is only supported in Clang. The hardware KASAN mode (#3) relies on hardware to perform the checks but still requires a compiler version that supports memory tagging instructions. -This mode is supported in GCC 10+ and Clang 11+. +This mode is supported in GCC 10+ and Clang 12+. Both software KASAN modes work with SLUB and SLAB memory allocators, while the hardware tag-based KASAN currently only supports SLUB. @@ -206,6 +206,9 @@ additional boot parameters that allow disabling KASAN or controlling features: Asymmetric mode: a bad access is detected synchronously on reads and asynchronously on writes. +- ``kasan.vmalloc=off`` or ``=on`` disables or enables tagging of vmalloc + allocations (default: ``on``). + - ``kasan.stacktrace=off`` or ``=on`` disables or enables alloc and free stack traces collection (default: ``on``). @@ -279,8 +282,8 @@ Software tag-based KASAN uses 0xFF as a match-all pointer tag (accesses through pointers with the 0xFF pointer tag are not checked). The value 0xFE is currently reserved to tag freed memory regions. -Software tag-based KASAN currently only supports tagging of slab and page_alloc -memory. +Software tag-based KASAN currently only supports tagging of slab, page_alloc, +and vmalloc memory. Hardware tag-based KASAN ~~~~~~~~~~~~~~~~~~~~~~~~ @@ -303,8 +306,8 @@ Hardware tag-based KASAN uses 0xFF as a match-all pointer tag (accesses through pointers with the 0xFF pointer tag are not checked). The value 0xFE is currently reserved to tag freed memory regions. -Hardware tag-based KASAN currently only supports tagging of slab and page_alloc -memory. +Hardware tag-based KASAN currently only supports tagging of slab, page_alloc, +and VM_ALLOC-based vmalloc memory. If the hardware does not support MTE (pre ARMv8.5), hardware tag-based KASAN will not be enabled. In this case, all KASAN boot parameters are ignored. @@ -319,6 +322,8 @@ checking gets disabled. Shadow memory ------------- +The contents of this section are only applicable to software KASAN modes. + The kernel maps memory in several different parts of the address space. The range of kernel virtual addresses is large: there is not enough real memory to support a real shadow region for every address that could be @@ -349,7 +354,7 @@ CONFIG_KASAN_VMALLOC With ``CONFIG_KASAN_VMALLOC``, KASAN can cover vmalloc space at the cost of greater memory usage. Currently, this is supported on x86, -riscv, s390, and powerpc. +arm64, riscv, s390, and powerpc. This works by hooking into vmalloc and vmap and dynamically allocating real shadow memory to back the mappings. From patchwork Mon Dec 6 21:44:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 12695019 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 10CDFC433EF for ; Mon, 6 Dec 2021 22:10:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=vkQ2Nww9Z0GMtx382Am3ZVak5IxVxQpxT8oX38UoFm8=; b=Pm4poshAkmX1GJ +PqdIJAb+Y9hk+Kr1Xcxbgszqk+wHfkldlnxZNJzCbMdi3ErG/F3LK7v2RvFGlW2ddQmrG7/WVP47 UkAwlsZBjsFyynHp7rY8ivyRVSYBKAJtoOKpOHS2cOKjujJ5NepItUXQEEUfQLuo9BTYgjJe+wPHn FLUpdXQxNlTlMeZx3gDOQvRwylMt7eOPYJbSdSn1AsTzPl3fDXxacCFdTrrt0ZxNtemfrX/lsKU2G nAzWSJxO0glf8DMigMnH6h6guBGDcM6R/uZpPuAsFsxYPlpigEp8nXFMklmt4cVDEvEu14bnpEgCt QOZ1SMKmXr5fQV9XbNbA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1muMAP-0068sG-QE; Mon, 06 Dec 2021 22:09:04 +0000 Received: from out2.migadu.com ([2001:41d0:2:aacc::]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1muLpT-005zrt-3N for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 21:47:25 +0000 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1638827241; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3pytIz/mVv/PIB06AmhSlrnOdE2zgWaSkR0qkks0t+o=; b=I2Ewj6wAxAKScF6XzdWWRK1WyMk8nttk0qAXnuLWNx6Ahb/S3M7TVmP3vwhWK4FA9kObVN NPhBlzB8Turix4wc4cRoC6img98Yo7bwKwUjjeICkJZk1WGIJleDPDM93IdkONF5ipIXg8 KhfQcNnHQLNegSXDpvAW1kTvdC+juVI= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko , Vincenzo Frascino , Catalin Marinas , Peter Collingbourne Cc: Andrey Konovalov , Dmitry Vyukov , Andrey Ryabinin , kasan-dev@googlegroups.com, Andrew Morton , linux-mm@kvack.org, Will Deacon , Mark Rutland , linux-arm-kernel@lists.infradead.org, Evgenii Stepanov , linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH v2 34/34] kasan: improve vmalloc tests Date: Mon, 6 Dec 2021 22:44:11 +0100 Message-Id: <1780c3aae4f143d4bd2137cb0d2e3a137a680664.1638825394.git.andreyknvl@google.com> In-Reply-To: References: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211206_134723_506741_D24FEC82 X-CRM114-Status: GOOD ( 18.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Andrey Konovalov Update the existing vmalloc_oob() test to account for the specifics of the tag-based modes. Also add a few new checks and comments. Add new vmalloc-related tests: - vmalloc_helpers_tags() to check that exported vmalloc helpers can handle tagged pointers. - vmap_tags() to check that SW_TAGS mode properly tags vmap() mappings. - vm_map_ram_tags() to check that SW_TAGS mode properly tags vm_map_ram() mappings. - vmalloc_percpu() to check that SW_TAGS mode tags regions allocated for __alloc_percpu(). The tagging of per-cpu mappings is best-effort; proper tagging is tracked in [1]. [1] https://bugzilla.kernel.org/show_bug.cgi?id=215019 Signed-off-by: Andrey Konovalov --- lib/test_kasan.c | 181 +++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 175 insertions(+), 6 deletions(-) diff --git a/lib/test_kasan.c b/lib/test_kasan.c index 0643573f8686..44875356278a 100644 --- a/lib/test_kasan.c +++ b/lib/test_kasan.c @@ -1025,21 +1025,174 @@ static void kmalloc_double_kzfree(struct kunit *test) KUNIT_EXPECT_KASAN_FAIL(test, kfree_sensitive(ptr)); } +static void vmalloc_helpers_tags(struct kunit *test) +{ + void *ptr; + + /* This test is intended for tag-based modes. */ + KASAN_TEST_NEEDS_CONFIG_OFF(test, CONFIG_KASAN_GENERIC); + + KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_KASAN_VMALLOC); + + ptr = vmalloc(PAGE_SIZE); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr); + + /* Check that the returned pointer is tagged. */ + KUNIT_EXPECT_GE(test, (u8)get_tag(ptr), (u8)KASAN_TAG_MIN); + KUNIT_EXPECT_LT(test, (u8)get_tag(ptr), (u8)KASAN_TAG_KERNEL); + + /* Make sure exported vmalloc helpers handle tagged pointers. */ + KUNIT_ASSERT_TRUE(test, is_vmalloc_addr(ptr)); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, vmalloc_to_page(ptr)); + + vfree(ptr); +} + static void vmalloc_oob(struct kunit *test) { - void *area; + char *v_ptr, *p_ptr; + struct page *page; + size_t size = PAGE_SIZE / 2 - KASAN_GRANULE_SIZE - 5; KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_KASAN_VMALLOC); + v_ptr = vmalloc(size); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, v_ptr); + /* - * We have to be careful not to hit the guard page. + * We have to be careful not to hit the guard page in vmalloc tests. * The MMU will catch that and crash us. */ - area = vmalloc(3000); - KUNIT_ASSERT_NOT_ERR_OR_NULL(test, area); - KUNIT_EXPECT_KASAN_FAIL(test, ((volatile char *)area)[3100]); - vfree(area); + /* Make sure in-bounds accesses are valid. */ + v_ptr[0] = 0; + v_ptr[size - 1] = 0; + + /* + * An unaligned access past the requested vmalloc size. + * Only generic KASAN can precisely detect these. + */ + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) + KUNIT_EXPECT_KASAN_FAIL(test, ((volatile char *)v_ptr)[size]); + + /* An aligned access into the first out-of-bounds granule. */ + KUNIT_EXPECT_KASAN_FAIL(test, ((volatile char *)v_ptr)[size + 5]); + + /* Check that in-bounds accesses to the physical page are valid. */ + page = vmalloc_to_page(v_ptr); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, page); + p_ptr = page_address(page); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, p_ptr); + p_ptr[0] = 0; + + vfree(v_ptr); + + /* + * We can't check for use-after-unmap bugs in this nor in the following + * vmalloc tests, as the page might be fully unmapped and accessing it + * will crash the kernel. + */ +} + +static void vmap_tags(struct kunit *test) +{ + char *p_ptr, *v_ptr; + struct page *p_page, *v_page; + size_t order = 1; + + /* + * This test is specifically crafted for the software tag-based mode, + * the only tag-based mode that poisons vmap mappings. + */ + KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_KASAN_SW_TAGS); + + KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_KASAN_VMALLOC); + + p_page = alloc_pages(GFP_KERNEL, order); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, p_page); + p_ptr = page_address(p_page); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, p_ptr); + + v_ptr = vmap(&p_page, 1 << order, VM_MAP, PAGE_KERNEL); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, v_ptr); + + /* + * We can't check for out-of-bounds bugs in this nor in the following + * vmalloc tests, as allocations have page granularity and accessing + * the guard page will crash the kernel. + */ + + KUNIT_EXPECT_GE(test, (u8)get_tag(v_ptr), (u8)KASAN_TAG_MIN); + KUNIT_EXPECT_LT(test, (u8)get_tag(v_ptr), (u8)KASAN_TAG_KERNEL); + + /* Make sure that in-bounds accesses through both pointers work. */ + *p_ptr = 0; + *v_ptr = 0; + + /* Make sure vmalloc_to_page() correctly recovers the page pointer. */ + v_page = vmalloc_to_page(v_ptr); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, v_page); + KUNIT_EXPECT_PTR_EQ(test, p_page, v_page); + + vunmap(v_ptr); + free_pages((unsigned long)p_ptr, order); +} + +static void vm_map_ram_tags(struct kunit *test) +{ + char *p_ptr, *v_ptr; + struct page *page; + size_t order = 1; + + /* + * This test is specifically crafted for the software tag-based mode, + * the only tag-based mode that poisons vm_map_ram mappings. + */ + KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_KASAN_SW_TAGS); + + page = alloc_pages(GFP_KERNEL, order); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, page); + p_ptr = page_address(page); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, p_ptr); + + v_ptr = vm_map_ram(&page, 1 << order, -1); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, v_ptr); + + KUNIT_EXPECT_GE(test, (u8)get_tag(v_ptr), (u8)KASAN_TAG_MIN); + KUNIT_EXPECT_LT(test, (u8)get_tag(v_ptr), (u8)KASAN_TAG_KERNEL); + + /* Make sure that in-bounds accesses through both pointers work. */ + *p_ptr = 0; + *v_ptr = 0; + + vm_unmap_ram(v_ptr, 1 << order); + free_pages((unsigned long)p_ptr, order); +} + +static void vmalloc_percpu(struct kunit *test) +{ + char __percpu *ptr; + int cpu; + + /* + * This test is specifically crafted for the software tag-based mode, + * the only tag-based mode that poisons percpu mappings. + */ + KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_KASAN_SW_TAGS); + + ptr = __alloc_percpu(PAGE_SIZE, PAGE_SIZE); + + for_each_possible_cpu(cpu) { + char *c_ptr = per_cpu_ptr(ptr, cpu); + + KUNIT_EXPECT_GE(test, (u8)get_tag(c_ptr), (u8)KASAN_TAG_MIN); + KUNIT_EXPECT_LT(test, (u8)get_tag(c_ptr), (u8)KASAN_TAG_KERNEL); + + /* Make sure that in-bounds accesses don't crash the kernel. */ + *c_ptr = 0; + } + + free_percpu(ptr); } /* @@ -1073,6 +1226,18 @@ static void match_all_not_assigned(struct kunit *test) KUNIT_EXPECT_LT(test, (u8)get_tag(ptr), (u8)KASAN_TAG_KERNEL); free_pages((unsigned long)ptr, order); } + + if (!IS_ENABLED(CONFIG_KASAN_VMALLOC)) + return; + + for (i = 0; i < 256; i++) { + size = (get_random_int() % 1024) + 1; + ptr = vmalloc(size); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr); + KUNIT_EXPECT_GE(test, (u8)get_tag(ptr), (u8)KASAN_TAG_MIN); + KUNIT_EXPECT_LT(test, (u8)get_tag(ptr), (u8)KASAN_TAG_KERNEL); + vfree(ptr); + } } /* Check that 0xff works as a match-all pointer tag for tag-based modes. */ @@ -1176,7 +1341,11 @@ static struct kunit_case kasan_kunit_test_cases[] = { KUNIT_CASE(kasan_bitops_generic), KUNIT_CASE(kasan_bitops_tags), KUNIT_CASE(kmalloc_double_kzfree), + KUNIT_CASE(vmalloc_helpers_tags), KUNIT_CASE(vmalloc_oob), + KUNIT_CASE(vmap_tags), + KUNIT_CASE(vm_map_ram_tags), + KUNIT_CASE(vmalloc_percpu), KUNIT_CASE(match_all_not_assigned), KUNIT_CASE(match_all_ptr_tag), KUNIT_CASE(match_all_mem_tag),