From patchwork Fri Nov 13 22:19:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904951 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EEF18697 for ; Fri, 13 Nov 2020 22:20:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A9CC0206F9 for ; Fri, 13 Nov 2020 22:20:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="sv8Z/Kpl" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A9CC0206F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C51836B0068; Fri, 13 Nov 2020 17:20:19 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C010F6B006C; Fri, 13 Nov 2020 17:20:19 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B172B6B006E; Fri, 13 Nov 2020 17:20:19 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0140.hostedemail.com [216.40.44.140]) by kanga.kvack.org (Postfix) with ESMTP id 842F66B0068 for ; Fri, 13 Nov 2020 17:20:19 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 3928D181AEF07 for ; Fri, 13 Nov 2020 22:20:19 +0000 (UTC) X-FDA: 77480814558.15.way70_1a0cd1527312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin15.hostedemail.com (Postfix) with ESMTP id 1150E1814B0C7 for ; Fri, 13 Nov 2020 22:20:19 +0000 (UTC) X-Spam-Summary: 1,0,0,8e4ad7a57e634523,d41d8cd98f00b204,3oqavxwokcg4mzpdqkwzhxsaasxq.oayxuzgj-yywhmow.ads@flex--andreyknvl.bounces.google.com,,RULES_HIT:41:152:355:379:541:800:960:966:967:968:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1542:1593:1594:1711:1730:1747:1777:1792:1981:2194:2196:2199:2200:2393:2525:2559:2563:2682:2685:2859:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3353:3865:3867:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4425:4605:5007:6261:6653:6742:7901:8660:9025:9969:10004:10400:11026:11473:11658:11914:12043:12296:12297:12438:12555:12679:12895:12986:13148:13230:14096:14097:14181:14394:14659:14721:21080:21212:21365:21444:21451:21627:21939:21966:21990:30054:30075,0,RBL:209.85.221.73:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100;04yrhykkj7eh4f3s9eucffktnai9yypfut7xejzkudzask68dpp6b4zjcburbw1.bn6owbqjzpzez4hxu4qr4bu6h7n6kcgxkddsfx1aerh7qurd6jzagdu zicsapmn X-HE-Tag: way70_1a0cd1527312 X-Filterd-Recvd-Size: 5834 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:18 +0000 (UTC) Received: by mail-wr1-f73.google.com with SMTP id l5so4659800wrn.18 for ; Fri, 13 Nov 2020 14:20:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=yJWxegPyzZmdx3tmzwd/mzW6YwRRuhTO52iQtSKYSdE=; b=sv8Z/Kpl8DPqS1Sq+9LSV8BzlnIXBEtlOFEuhbGnHPOXO55RZ0oPEEP5JKnN7fT+Ak fKVXOWxiDqG1juvq/4nEvVop/KyMwRKCDawX2Wcfc9QLjTsbEjqvJ47W3sld96b2x+Ku u4CNde58Ko+XriUl+s1iPD2ydInydYVLqXLRBCG2enfSsNWyw21fJAJrOdEDsYO2Z8HX hR3X4h0AllRSuOJUSwRLDw1SY8aOm1MsvLgNxcz27AOTj51eguTLz0FmMKiIK5CCEf+t gYZjbtRgH58KjocgQo0Ok1aEKIWTwfTFJND37dE2bp/Fxk5hMPsf96NpO4P7Pkn2t4Xo V2zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=yJWxegPyzZmdx3tmzwd/mzW6YwRRuhTO52iQtSKYSdE=; b=aj5NWX2y8BRLpxyIVubsy1v+6R3SIxdsmfNF9r2h7JvQSPIOT5FrTsLIOw+5aGHx87 52a6TjjnQnDsodFZJleEwNJ8hcpHAJ3vSRBFXoMjENlel5II81S7fuK1+lwMy70oZMcf wjpX5lIjFPSB2lYAKmZxcujbcbfC47h6a7b2GKhbf/i2vQYbOheIoiV6Eqo2mO7v8gmZ z/NUgBIu5rAK7zhopUOK6uhI1RZQ3pq+xwnVUpBQP0ZdqDAifu/WiT36PgB71jvNE3D9 DvuE4qeRLvwuSoB3c+ZgrKZ4Ydo78CBJ/Ot6sT9/joJNiDFoBKJjYQOu/CKAP+RNNwkH QoRw== X-Gm-Message-State: AOAM531K5OdPXoT5m5IN0QbXjV36q4iyKwBrZJq0lZAyeyPTFQmosG46 pW1h/2UMKuIEgcpO5DGshfstSaXmf+6P6CzI X-Google-Smtp-Source: ABdhPJztvZOjYQgEdQq2vWkh9fZERHuoYj0nV+zcLr3AnX/+IEHgi9Qpjls1gxC875+5jFe6XwSJ/NKXDEkUQdd4 X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a1c:658b:: with SMTP id z133mr4378653wmb.1.1605306017372; Fri, 13 Nov 2020 14:20:17 -0800 (PST) Date: Fri, 13 Nov 2020 23:19:51 +0100 In-Reply-To: Message-Id: <0eeeec8ecbf877e526ea43808e40e9062550217b.1605305978.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 01/19] kasan: simplify quarantine_put call site From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Move get_free_info() call into quarantine_put() to simplify the call site. No functional changes. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Reviewed-by: Marco Elver Link: https://linux-review.googlesource.com/id/Iab0f04e7ebf8d83247024b7190c67c3c34c7940f --- mm/kasan/common.c | 2 +- mm/kasan/kasan.h | 5 ++--- mm/kasan/quarantine.c | 3 ++- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 998aede4d172..e11fac2ee30c 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -317,7 +317,7 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, kasan_set_free_info(cache, object, tag); - quarantine_put(get_free_info(cache, object), cache); + quarantine_put(cache, object); return IS_ENABLED(CONFIG_KASAN_GENERIC); } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 64560cc71191..13c511e85d5f 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -216,12 +216,11 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, #if defined(CONFIG_KASAN_GENERIC) && \ (defined(CONFIG_SLAB) || defined(CONFIG_SLUB)) -void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache); +void quarantine_put(struct kmem_cache *cache, void *object); void quarantine_reduce(void); void quarantine_remove_cache(struct kmem_cache *cache); #else -static inline void quarantine_put(struct kasan_free_meta *info, - struct kmem_cache *cache) { } +static inline void quarantine_put(struct kmem_cache *cache, void *object) { } static inline void quarantine_reduce(void) { } static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #endif diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c index 580ff5610fc1..a0792f0d6d0f 100644 --- a/mm/kasan/quarantine.c +++ b/mm/kasan/quarantine.c @@ -161,11 +161,12 @@ static void qlist_free_all(struct qlist_head *q, struct kmem_cache *cache) qlist_init(q); } -void quarantine_put(struct kasan_free_meta *info, struct kmem_cache *cache) +void quarantine_put(struct kmem_cache *cache, void *object) { unsigned long flags; struct qlist_head *q; struct qlist_head temp = QLIST_INIT; + struct kasan_free_meta *info = get_free_info(cache, object); /* * Note: irq must be disabled until after we move the batch to the From patchwork Fri Nov 13 22:19:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904953 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9E877697 for ; Fri, 13 Nov 2020 22:20:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 54B78206F9 for ; Fri, 13 Nov 2020 22:20:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="SAqovLo3" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 54B78206F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 65DB46B006E; Fri, 13 Nov 2020 17:20:22 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 60C3E6B0070; Fri, 13 Nov 2020 17:20:22 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 523B56B0071; Fri, 13 Nov 2020 17:20:22 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0196.hostedemail.com [216.40.44.196]) by kanga.kvack.org (Postfix) with ESMTP id 25B916B006E for ; Fri, 13 Nov 2020 17:20:22 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id BFD022493 for ; Fri, 13 Nov 2020 22:20:21 +0000 (UTC) X-FDA: 77480814642.26.hook81_5716b3327312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin26.hostedemail.com (Postfix) with ESMTP id 9A0641804B668 for ; Fri, 13 Nov 2020 22:20:21 +0000 (UTC) X-Spam-Summary: 1,0,0,8321a46a4c9e5fc5,d41d8cd98f00b204,3owavxwokchaobrfsmybjzuccuzs.qcazwbil-aayjoqy.cfu@flex--andreyknvl.bounces.google.com,,RULES_HIT:1:2:41:152:355:379:541:800:960:965:966:967:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1593:1594:1605:1730:1747:1777:1792:1981:2194:2196:2198:2199:2200:2201:2393:2525:2559:2563:2682:2685:2859:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3865:3867:3868:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4052:4321:4385:4390:4395:4425:4605:5007:6261:6653:6742:7901:8603:8660:8957:9025:9969:10004:11026:11473:11658:11914:12043:12296:12297:12438:12555:12679:12895:12986:13148:13230:14096:14097:14394:14659:21080:21365:21444:21451:21627:21939:21966:21990:30003:30054:30075,0,RBL:209.85.128.74:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-66.100.201.100 62.18.0.100;04yfzhg6gh93nkiuwbw3uqc87ih4bycmnyi1j6897uin5oumhhmamuc65dgwmbh.xi4mqnffqograriqdb35r3mi4x7pyncq6qdz5h4iz9zw qtedp589 X-HE-Tag: hook81_5716b3327312 X-Filterd-Recvd-Size: 12806 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf32.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:21 +0000 (UTC) Received: by mail-wm1-f74.google.com with SMTP id 14so4721271wmg.1 for ; Fri, 13 Nov 2020 14:20:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=P/VONEfbusEo0xeHDJ2NdS5sIzPWe1VMLcnIssonnS8=; b=SAqovLo37QQ2NB0aWReXa7jImijWxTZnuFwwtpHpcE9Jy3Ti9kErTdC0pkh5jo4Bnz +WqtAKPoUQzb6V5oAwZa34fYqrWPzMVNc7g4a4QgAN/Im2MBZ+ccH8GpccckQZDV87TG v3HcsPfOeHSd5+BkCJaSTq56znwPG8S8hcvuvI8CP4Usu5oSQ19Owl4EbOsBXbFzZB2m 6/k6rdUtMzyDp/HKKyv8rPMGygiXdibU79dhJJwUMjG4qU67DZp8cszaHZM4nn1tGbDE 0OgET3lVqtyFprs7VxmZVg8xnjhh1Zm4PYtlHmcqKbvI7rNdLRwTM8C/Ze6pfvBXwUW8 hW0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=P/VONEfbusEo0xeHDJ2NdS5sIzPWe1VMLcnIssonnS8=; b=sHIFXZvP9tBJUh6rQP9WgIecXblxf/bi/fjFaTPyyMaO0vu6l6+0FLh7O5EL2fr/L+ HVeRWp2FLg3mNIBAjqy0zUppP1dQnb6x9p9w9g67gPw7pTjTnnoChDErJ/Bwsin7AEcK rLBzunS1wg3dduqILIvvOfnL3z/KKf80B3co3KPkLBjkNtWO2M85xMtPfCFaSiN6Z2RI smv6fYf+UZ++Es0EIpGiJDEdVoLEG4CJexvoRGBOICDIP1AiTbBGJR+2/8QFrFCa4Tqw +Ycnb9E8axNDRigUJL39vdX2BfuKjNkRUPNLI2EV+HGRDCBYJBVwnlsi8ksLLGxBBJAG oLVw== X-Gm-Message-State: AOAM530pXKktu8hB795kYxpuZh4+qEVMkmpin3Iy76tuHcgR6gIDAfOR 10Ik4Naf/Z1HzmGcQ0VLdUUIV6O+8dIh8415 X-Google-Smtp-Source: ABdhPJyuI8ihad4A/Fvw0cT1leHjWshG3CP2UBNMIFnD16Ns/Uvb/aqbft2aTvZwCvVu+YGfwnZ7WuMqiZhVrNmH X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:adf:8382:: with SMTP id 2mr6018992wre.227.1605306019928; Fri, 13 Nov 2020 14:20:19 -0800 (PST) Date: Fri, 13 Nov 2020 23:19:52 +0100 In-Reply-To: Message-Id: <54a24c9db9d2c5dd7e0c268f19a693077adf6fd9.1605305978.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 02/19] kasan: rename get_alloc/free_info From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Rename get_alloc_info() and get_free_info() to kasan_get_alloc_meta() and kasan_get_free_meta() to better reflect what those do and avoid confusion with kasan_set_free_info(). No functional changes. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Reviewed-by: Marco Elver Link: https://linux-review.googlesource.com/id/Ib6e4ba61c8b12112b403d3479a9799ac8fff8de1 --- mm/kasan/common.c | 16 ++++++++-------- mm/kasan/generic.c | 12 ++++++------ mm/kasan/hw_tags.c | 4 ++-- mm/kasan/kasan.h | 8 ++++---- mm/kasan/quarantine.c | 4 ++-- mm/kasan/report.c | 12 ++++++------ mm/kasan/report_sw_tags.c | 2 +- mm/kasan/sw_tags.c | 4 ++-- 8 files changed, 31 insertions(+), 31 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index e11fac2ee30c..8197399b0a1f 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -181,14 +181,14 @@ size_t kasan_metadata_size(struct kmem_cache *cache) sizeof(struct kasan_free_meta) : 0); } -struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, - const void *object) +struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, + const void *object) { return (void *)reset_tag(object) + cache->kasan_info.alloc_meta_offset; } -struct kasan_free_meta *get_free_info(struct kmem_cache *cache, - const void *object) +struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, + const void *object) { BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); return (void *)reset_tag(object) + cache->kasan_info.free_meta_offset; @@ -265,13 +265,13 @@ static u8 assign_tag(struct kmem_cache *cache, const void *object, void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, const void *object) { - struct kasan_alloc_meta *alloc_info; + struct kasan_alloc_meta *alloc_meta; if (!(cache->flags & SLAB_KASAN)) return (void *)object; - alloc_info = get_alloc_info(cache, object); - __memset(alloc_info, 0, sizeof(*alloc_info)); + alloc_meta = kasan_get_alloc_meta(cache, object); + __memset(alloc_meta, 0, sizeof(*alloc_meta)); if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) object = set_tag(object, assign_tag(cache, object, true, false)); @@ -357,7 +357,7 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, KASAN_KMALLOC_REDZONE); if (cache->flags & SLAB_KASAN) - kasan_set_track(&get_alloc_info(cache, object)->alloc_track, flags); + kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); return set_tag(object, tag); } diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index da3608187c25..9c6b77f8c4a4 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -329,7 +329,7 @@ void kasan_record_aux_stack(void *addr) { struct page *page = kasan_addr_to_page(addr); struct kmem_cache *cache; - struct kasan_alloc_meta *alloc_info; + struct kasan_alloc_meta *alloc_meta; void *object; if (is_kfence_address(addr) || !(page && PageSlab(page))) @@ -337,13 +337,13 @@ void kasan_record_aux_stack(void *addr) cache = page->slab_cache; object = nearest_obj(cache, page, addr); - alloc_info = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); /* * record the last two call_rcu() call stacks. */ - alloc_info->aux_stack[1] = alloc_info->aux_stack[0]; - alloc_info->aux_stack[0] = kasan_save_stack(GFP_NOWAIT); + alloc_meta->aux_stack[1] = alloc_meta->aux_stack[0]; + alloc_meta->aux_stack[0] = kasan_save_stack(GFP_NOWAIT); } void kasan_set_free_info(struct kmem_cache *cache, @@ -351,7 +351,7 @@ void kasan_set_free_info(struct kmem_cache *cache, { struct kasan_free_meta *free_meta; - free_meta = get_free_info(cache, object); + free_meta = kasan_get_free_meta(cache, object); kasan_set_track(&free_meta->free_track, GFP_NOWAIT); /* @@ -365,5 +365,5 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, { if (*(u8 *)kasan_mem_to_shadow(object) != KASAN_KMALLOC_FREETRACK) return NULL; - return &get_free_info(cache, object)->free_track; + return &kasan_get_free_meta(cache, object)->free_track; } diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 3f9232464ed4..68e77363e58b 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -75,7 +75,7 @@ void kasan_set_free_info(struct kmem_cache *cache, { struct kasan_alloc_meta *alloc_meta; - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT); } @@ -84,6 +84,6 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, { struct kasan_alloc_meta *alloc_meta; - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); return &alloc_meta->free_track[0]; } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 13c511e85d5f..0eab7e4cecb8 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -149,10 +149,10 @@ struct kasan_free_meta { #endif }; -struct kasan_alloc_meta *get_alloc_info(struct kmem_cache *cache, - const void *object); -struct kasan_free_meta *get_free_info(struct kmem_cache *cache, - const void *object); +struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, + const void *object); +struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, + const void *object); void poison_range(const void *address, size_t size, u8 value); void unpoison_range(const void *address, size_t size); diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c index a0792f0d6d0f..0da3d37e1589 100644 --- a/mm/kasan/quarantine.c +++ b/mm/kasan/quarantine.c @@ -166,7 +166,7 @@ void quarantine_put(struct kmem_cache *cache, void *object) unsigned long flags; struct qlist_head *q; struct qlist_head temp = QLIST_INIT; - struct kasan_free_meta *info = get_free_info(cache, object); + struct kasan_free_meta *meta = kasan_get_free_meta(cache, object); /* * Note: irq must be disabled until after we move the batch to the @@ -179,7 +179,7 @@ void quarantine_put(struct kmem_cache *cache, void *object) local_irq_save(flags); q = this_cpu_ptr(&cpu_quarantine); - qlist_put(q, &info->quarantine_link, cache->size); + qlist_put(q, &meta->quarantine_link, cache->size); if (unlikely(q->bytes > QUARANTINE_PERCPU_SIZE)) { qlist_move_all(q, &temp); diff --git a/mm/kasan/report.c b/mm/kasan/report.c index a69c2827a125..df16bef0d810 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -164,12 +164,12 @@ static void describe_object_addr(struct kmem_cache *cache, void *object, static void describe_object(struct kmem_cache *cache, void *object, const void *addr, u8 tag) { - struct kasan_alloc_meta *alloc_info = get_alloc_info(cache, object); + struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object); if (cache->flags & SLAB_KASAN) { struct kasan_track *free_track; - print_track(&alloc_info->alloc_track, "Allocated"); + print_track(&alloc_meta->alloc_track, "Allocated"); pr_err("\n"); free_track = kasan_get_free_track(cache, object, tag); if (free_track) { @@ -178,14 +178,14 @@ static void describe_object(struct kmem_cache *cache, void *object, } #ifdef CONFIG_KASAN_GENERIC - if (alloc_info->aux_stack[0]) { + if (alloc_meta->aux_stack[0]) { pr_err("Last call_rcu():\n"); - print_stack(alloc_info->aux_stack[0]); + print_stack(alloc_meta->aux_stack[0]); pr_err("\n"); } - if (alloc_info->aux_stack[1]) { + if (alloc_meta->aux_stack[1]) { pr_err("Second to last call_rcu():\n"); - print_stack(alloc_info->aux_stack[1]); + print_stack(alloc_meta->aux_stack[1]); pr_err("\n"); } #endif diff --git a/mm/kasan/report_sw_tags.c b/mm/kasan/report_sw_tags.c index aebc44a29e83..317100fd95b9 100644 --- a/mm/kasan/report_sw_tags.c +++ b/mm/kasan/report_sw_tags.c @@ -46,7 +46,7 @@ const char *get_bug_type(struct kasan_access_info *info) if (page && PageSlab(page)) { cache = page->slab_cache; object = nearest_obj(cache, page, (void *)addr); - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); for (i = 0; i < KASAN_NR_FREE_STACKS; i++) if (alloc_meta->free_pointer_tag[i] == tag) diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c index a518483f3965..6d7648cc3b98 100644 --- a/mm/kasan/sw_tags.c +++ b/mm/kasan/sw_tags.c @@ -174,7 +174,7 @@ void kasan_set_free_info(struct kmem_cache *cache, struct kasan_alloc_meta *alloc_meta; u8 idx = 0; - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY idx = alloc_meta->free_track_idx; @@ -191,7 +191,7 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, struct kasan_alloc_meta *alloc_meta; int i = 0; - alloc_meta = get_alloc_info(cache, object); + alloc_meta = kasan_get_alloc_meta(cache, object); #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY for (i = 0; i < KASAN_NR_FREE_STACKS; i++) { From patchwork Fri Nov 13 22:19:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904955 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 39673697 for ; Fri, 13 Nov 2020 22:20:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E6D852223F for ; Fri, 13 Nov 2020 22:20:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="NbjGOqNw" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E6D852223F Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E72D76B0071; Fri, 13 Nov 2020 17:20:24 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E236B6B0072; Fri, 13 Nov 2020 17:20:24 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3AD96B0073; Fri, 13 Nov 2020 17:20:24 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0030.hostedemail.com [216.40.44.30]) by kanga.kvack.org (Postfix) with ESMTP id A67E76B0071 for ; Fri, 13 Nov 2020 17:20:24 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 5886A180AD807 for ; Fri, 13 Nov 2020 22:20:24 +0000 (UTC) X-FDA: 77480814768.16.birth51_040d25f27312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id 31D9E100E690C for ; Fri, 13 Nov 2020 22:20:24 +0000 (UTC) X-Spam-Summary: 1,0,0,bfb3e20af96d2f42,d41d8cd98f00b204,3pgavxwokchmreuivpbemcxffxcv.tfdczelo-ddbmrtb.fix@flex--andreyknvl.bounces.google.com,,RULES_HIT:41:152:355:379:541:800:960:966:967:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1534:1541:1593:1594:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2693:2859:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3352:3865:3866:3868:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4425:5007:6261:6653:6742:7901:8603:9025:9969:10004:10400:11026:11473:11658:11914:12043:12296:12297:12438:12555:12679:12895:13069:13311:13357:14096:14097:14181:14394:14659:14721:21080:21365:21444:21451:21627:21939:21990:30012:30054,0,RBL:209.85.128.74:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-66.100.201.100 62.18.0.100;04yfkmwqm7jarwqdw16g5dzczewo8ypqjub19hz1ow18mp45wbyndka9wu6y7a1.symanrs7rc5ki4dpnpa74oeioiwuky3c33eyfeqapkcj6trp5pr8dasn15mmo1s.a-lbl8.mails hell.net X-HE-Tag: birth51_040d25f27312 X-Filterd-Recvd-Size: 4787 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf46.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:23 +0000 (UTC) Received: by mail-wm1-f74.google.com with SMTP id y26so4001406wmj.7 for ; Fri, 13 Nov 2020 14:20:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=mI/2rW8u6CFpBd7nyudEpTk9pqQDS3YjqZmg/d3PQMM=; b=NbjGOqNwl3sICIUuy2Dx6Xzsk+mkCs6fm3XUGKRsfG3RlwtzwJeMtGqzhnOtjIOoiT Rot4BuMouS9KdzhIncvVUDcdcTBFMKkaN6qwjznaCmoM7krl5OFJ6ix+0sWy8GuPLay+ e2mJaLuz2BFxMMdHZ8SVcxa4QWYAge/CciV1HtckmdeKroa7ui8YIubpomJfLEoJ1dEc GBNCFWI8S9GHN0gntUUsZaqfFg/Mpc6BhsEB7G2dkF8gCf7kEbANocv7Q+U3YZWQavL/ CxUEXJS9haZSNAG33cDsoNwaEc22R7xi0HycIMDSfOILK2oAZVPrHH2nAhLqdyaCGBkG gbdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=mI/2rW8u6CFpBd7nyudEpTk9pqQDS3YjqZmg/d3PQMM=; b=Ef8RiA9vfQL34Pgep5Cujd8UG1lP+8prFt2o04etKMthBNxOpVMbL7uII0+21CGwZG t50FrSQbXIZeBDRYONAC7DqXzQyS3twQ4Klvb9O4/IuwbJRYAGOEneXRHIy2F1xBzN5t LzTgQOWTDV1VXEQ7Q8Oxit2vmldytvUsIydNPhAQJAogjZRXe5XUMtpzO9YX56aEIOej jbICB2z7B1b3MG+SBvlah8QpzhzN+Pa0TVOR0OHQkinlkYBPGwJuBDlkCiZLUbJWoYn3 XG10AXqEA+gHBvyQodkMaIcuK/+4Kpfedv5gQnsyiJjoVKJKvwW93vDDdeL1KknlV15F yMAQ== X-Gm-Message-State: AOAM530/FO9jjL2Lg4vArNj+m0MEUS6u7GaI9HyQwaQvCXpmeZPO9BL8 tuDbiouTACoHpkS+WEGPQuRXEjw2B25DNGr7 X-Google-Smtp-Source: ABdhPJxYtR4bh2WHC0w/07Gb/OQVXh3sZOJbZ2iTC6HhuummnwnQy7wvSSFekrmaH6T8oUMdO+1HYMOCfV40XpEX X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:adf:f04b:: with SMTP id t11mr5710125wro.147.1605306022529; Fri, 13 Nov 2020 14:20:22 -0800 (PST) Date: Fri, 13 Nov 2020 23:19:53 +0100 In-Reply-To: Message-Id: <31f5366d3245f5405185a0b4057b305613ce60ee.1605305978.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 03/19] kasan: introduce set_alloc_info From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add set_alloc_info() helper and move kasan_set_track() into it. This will simplify the code for one of the upcoming changes. No functional changes. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Reviewed-by: Marco Elver Link: https://linux-review.googlesource.com/id/I0316193cbb4ecc9b87b7c2eee0dd79f8ec908c1a --- mm/kasan/common.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 8197399b0a1f..0a420f1dbc54 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -327,6 +327,11 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) return __kasan_slab_free(cache, object, ip, true); } +static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) +{ + kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); +} + static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, gfp_t flags, bool keep_tag) { @@ -357,7 +362,7 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, KASAN_KMALLOC_REDZONE); if (cache->flags & SLAB_KASAN) - kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); + set_alloc_info(cache, (void *)object, flags); return set_tag(object, tag); } From patchwork Fri Nov 13 22:19:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904957 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 57F05697 for ; Fri, 13 Nov 2020 22:20:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 176872223F for ; Fri, 13 Nov 2020 22:20:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="r1t8fKl4" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 176872223F Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 020596B0073; Fri, 13 Nov 2020 17:20:27 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F39F76B0074; Fri, 13 Nov 2020 17:20:26 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D1ADB6B0075; Fri, 13 Nov 2020 17:20:26 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0054.hostedemail.com [216.40.44.54]) by kanga.kvack.org (Postfix) with ESMTP id A26CC6B0073 for ; Fri, 13 Nov 2020 17:20:26 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 5402D180AD806 for ; Fri, 13 Nov 2020 22:20:26 +0000 (UTC) X-FDA: 77480814852.09.map42_2d1149d27312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin09.hostedemail.com (Postfix) with ESMTP id 3B6CA180AD804 for ; Fri, 13 Nov 2020 22:20:26 +0000 (UTC) X-Spam-Summary: 1,0,0,5a55da3603dd1d85,d41d8cd98f00b204,3qaavxwokchutgwkxrdgoezhhzex.vhfebgnq-ffdotvd.hkz@flex--andreyknvl.bounces.google.com,,RULES_HIT:41:152:355:379:541:800:960:966:967:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1543:1593:1594:1711:1730:1747:1777:1792:1981:2194:2196:2198:2199:2200:2201:2393:2525:2553:2559:2563:2682:2685:2731:2859:2896:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3354:3865:3866:3867:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4118:4250:4321:4385:4425:4605:5007:6261:6653:6742:7875:7901:8603:8660:9025:9592:9969:10004:10400:11026:11473:11657:11658:11914:12043:12295:12296:12297:12438:12555:12679:12895:12986:13148:13161:13180:13229:13230:14096:14097:14181:14394:14659:14721:21080:21365:21444:21451:21627:21939:21990:30003:30054:30090,0,RBL:209.85.219.74:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-66.100.201.100 62.18.0.100;04yf9yjit9aqah7axomw6skb u9qa6yp9 X-HE-Tag: map42_2d1149d27312 X-Filterd-Recvd-Size: 7235 Received: from mail-qv1-f74.google.com (mail-qv1-f74.google.com [209.85.219.74]) by imf21.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:25 +0000 (UTC) Received: by mail-qv1-f74.google.com with SMTP id s8so2718286qvr.20 for ; Fri, 13 Nov 2020 14:20:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=u0fQxCBxmUxhMOfySImDDJJwP6bRFOpgmkJnBahaSB0=; b=r1t8fKl4uuOcWcbL/ueqRvVuuGcEPqzKT8WznSgyrrTLeFP98TCZYmgIsxP7hJRUIL UA2CQ6RqwhXxL5pbj5UZBnxHKj7OqZYU4/ykEQ2vFSNoEairvQ5DffreXNn7ZA2OblFy QESP9jI3hM9x5J9EegrsAz0SQtx2s5rUZfj8PzVPUbblnIVhT9ZUQxrQqGv9gkaVNgN4 FoHyoW9HuPpwVeg7Pseve3oFgp2LEz2IWTH0SpHtt2ViCmh+xEMyYIdlpbP6QcyHVd4l Q3Oork8GgswLd4lgBz2p5tzr6rmuu2fvDobA96l+nwV1GonWfuKd/exDswWmQ0+mJ1rw 1u+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=u0fQxCBxmUxhMOfySImDDJJwP6bRFOpgmkJnBahaSB0=; b=XL6wWawVykh1Rg6oDquh2li1d9wZ91C/hGEpDNbuJmNkaXagKcMu0C9z9LUclZ8Pax /hfz16P/NaOTa0VjKbNCdqn1TblBxRDeWVY08Rb0tAJvBBn+nOz1UL+vY+IijI8lY5E9 jlmHWRyE53dK/RUw0y1W15nH2Uow/1/UDuP+2BrZc5qVSNQUvwR0wHlAsoPNvxsogMUZ 8KMYijf4zUsG7zGNbjIPCutM2mChPcXef+F/PVsPozlimx8jtCsRvpKRq3nWvD19Ec64 uElp0F3xWEJkQOcMxHRICMzugVnEYF6OHVDwJ/BgVBa2wY9btj4D+qr6SySN4hgp+0JI XSIA== X-Gm-Message-State: AOAM531aLtal6gTnNRfkWffroIworjRIivM6mrold9JJJPedCRtzAiCW W1O63rMhqcjStvNoHDmwGn87P5B7UWkA/fr6 X-Google-Smtp-Source: ABdhPJzDRObTBajDl9iT1PbNpJcek6LGpoJ9nw4zlOW0zvL70FxU0UTFPIEWguVxCkQFV3wqiBJkCh4vS4JR1wzJ X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a0c:f9c8:: with SMTP id j8mr4873782qvo.17.1605306024802; Fri, 13 Nov 2020 14:20:24 -0800 (PST) Date: Fri, 13 Nov 2020 23:19:54 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 04/19] kasan, arm64: unpoison stack only with CONFIG_KASAN_STACK From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There's a config option CONFIG_KASAN_STACK that has to be enabled for KASAN to use stack instrumentation and perform validity checks for stack variables. There's no need to unpoison stack when CONFIG_KASAN_STACK is not enabled. Only call kasan_unpoison_task_stack[_below]() when CONFIG_KASAN_STACK is enabled. Note, that CONFIG_KASAN_STACK is an option that is currently always defined when CONFIG_KASAN is enabled, and therefore has to be tested with #if instead of #ifdef. Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Acked-by: Catalin Marinas Link: https://linux-review.googlesource.com/id/If8a891e9fe01ea543e00b576852685afec0887e3 Reviewed-by: Dmitry Vyukov --- arch/arm64/kernel/sleep.S | 2 +- arch/x86/kernel/acpi/wakeup_64.S | 2 +- include/linux/kasan.h | 10 ++++++---- mm/kasan/common.c | 2 ++ 4 files changed, 10 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/sleep.S b/arch/arm64/kernel/sleep.S index ba40d57757d6..bdadfa56b40e 100644 --- a/arch/arm64/kernel/sleep.S +++ b/arch/arm64/kernel/sleep.S @@ -133,7 +133,7 @@ SYM_FUNC_START(_cpu_resume) */ bl cpu_do_resume -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN) && CONFIG_KASAN_STACK mov x0, sp bl kasan_unpoison_task_stack_below #endif diff --git a/arch/x86/kernel/acpi/wakeup_64.S b/arch/x86/kernel/acpi/wakeup_64.S index c8daa92f38dc..5d3a0b8fd379 100644 --- a/arch/x86/kernel/acpi/wakeup_64.S +++ b/arch/x86/kernel/acpi/wakeup_64.S @@ -112,7 +112,7 @@ SYM_FUNC_START(do_suspend_lowlevel) movq pt_regs_r14(%rax), %r14 movq pt_regs_r15(%rax), %r15 -#ifdef CONFIG_KASAN +#if defined(CONFIG_KASAN) && CONFIG_KASAN_STACK /* * The suspend path may have poisoned some areas deeper in the stack, * which we now need to unpoison. diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 0c89e6fdd29e..f2109bf0c5f9 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -76,8 +76,6 @@ static inline void kasan_disable_current(void) {} void kasan_unpoison_range(const void *address, size_t size); -void kasan_unpoison_task_stack(struct task_struct *task); - void kasan_alloc_pages(struct page *page, unsigned int order); void kasan_free_pages(struct page *page, unsigned int order); @@ -122,8 +120,6 @@ void kasan_restore_multi_shot(bool enabled); static inline void kasan_unpoison_range(const void *address, size_t size) {} -static inline void kasan_unpoison_task_stack(struct task_struct *task) {} - static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} @@ -175,6 +171,12 @@ static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #endif /* CONFIG_KASAN */ +#if defined(CONFIG_KASAN) && CONFIG_KASAN_STACK +void kasan_unpoison_task_stack(struct task_struct *task); +#else +static inline void kasan_unpoison_task_stack(struct task_struct *task) {} +#endif + #ifdef CONFIG_KASAN_GENERIC void kasan_cache_shrink(struct kmem_cache *cache); diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 0a420f1dbc54..7648a2452a01 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -64,6 +64,7 @@ void kasan_unpoison_range(const void *address, size_t size) unpoison_range(address, size); } +#if CONFIG_KASAN_STACK static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) { void *base = task_stack_page(task); @@ -90,6 +91,7 @@ asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) unpoison_range(base, watermark - base); } +#endif /* CONFIG_KASAN_STACK */ void kasan_alloc_pages(struct page *page, unsigned int order) { From patchwork Fri Nov 13 22:19:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904959 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D57341391 for ; Fri, 13 Nov 2020 22:20:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 98D1E206F9 for ; Fri, 13 Nov 2020 22:20:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="n+QZ43qt" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 98D1E206F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8804E6B0075; Fri, 13 Nov 2020 17:20:29 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 832AB6B0078; Fri, 13 Nov 2020 17:20:29 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F8276B007B; Fri, 13 Nov 2020 17:20:29 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0111.hostedemail.com [216.40.44.111]) by kanga.kvack.org (Postfix) with ESMTP id 3EA786B0075 for ; Fri, 13 Nov 2020 17:20:29 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id DC8951DF8 for ; Fri, 13 Nov 2020 22:20:28 +0000 (UTC) X-FDA: 77480814936.28.chess85_5506e3d27312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin28.hostedemail.com (Postfix) with ESMTP id B62326C1A for ; Fri, 13 Nov 2020 22:20:28 +0000 (UTC) X-Spam-Summary: 1,0,0,4823c4bf181c91fa,d41d8cd98f00b204,3qwavxwokchgwjznaugjrhckkcha.ykihejqt-iigrwyg.knc@flex--andreyknvl.bounces.google.com,,RULES_HIT:41:152:355:379:541:800:960:967:973:988:989:1042:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1534:1541:1593:1594:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3352:3865:3867:3868:3871:3872:3873:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4425:5007:6261:6653:6742:7901:9025:9969:10004:10400:11026:11232:11473:11658:11914:12043:12296:12297:12438:12555:12679:12895:13069:13311:13357:14096:14097:14181:14394:14659:14721:21080:21365:21444:21451:21627:21740:21772:21939:30054:30070:30091,0,RBL:209.85.208.74:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-62.18.84.100 66.100.201.100;04yguthwyoxbewptazbijdxeas79uocjg8qp6dy79g4zajwceiszyfjiqifw593.wkdezrhnbz976k8fg9hhciu8c8gus8foww195j47r36k7rgc4nzwutekxjxuhog.1- lbl8.mai X-HE-Tag: chess85_5506e3d27312 X-Filterd-Recvd-Size: 4853 Received: from mail-ed1-f74.google.com (mail-ed1-f74.google.com [209.85.208.74]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:28 +0000 (UTC) Received: by mail-ed1-f74.google.com with SMTP id g1so5579429edk.0 for ; Fri, 13 Nov 2020 14:20:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=240iZR3W5frW7OjJkRyo/pFDEniTKiELI7K8tAXDIKI=; b=n+QZ43qtzUyU4WD1xwf080S4imk4yVUJ39FqVQaMSbKzWQ1vUS9xJOJ7s9fUUchALN Jfhs3F7pt9JDglHOLKlY1vawlOGZiB0bvCey0t3dKR6PvUhqL2jTPJ5r28bbCUYJmjjD oqM2PlG8ZS+T6DFa42NZJZZ11NN6A/flIcRnW7bQuT52DHg6F0Qg+zLlyJSc32lyOqEK BesN4mhSUaneb6ZqvX7VkXknlff7kAkLZfrRbfPq4TVOk+bRGC951i/ze0YSbDafVUWf A7LAIMkwIE40dAHbPLl7O39FAlQh5ICWuDPXIrVHeGUVDZvxrh34y8RXepDucQAOvNfq wFNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=240iZR3W5frW7OjJkRyo/pFDEniTKiELI7K8tAXDIKI=; b=lqTetFTVqWRkMagLpG3rDUk760fqAUMGx9+gtOr3dLmX6zuI2jG8TXpjEr2wXZfq3p 7eMHOSVzbuPwKsczKAnagUefO2DqfsBg4Cp1RfPxDHT4SF/lDkHsAqUz1hVuh2QF3X6l BhLyG7YDizWmrRG0qd+fagMeFZZWEUyFT5zEFWA714WC1O8M2tfu2fUPTghs6ucPvZwy Uwt9WP2uFyKsWLcZLSTK1GnZUN2PhGVGZpph4MVcH9c3VGkFdpCJQG6OEl//AjKJ1ncH 2w9vs+pm+5YSXrN9iwPV+4r5MfAXY5XmauURIKPsEK08mSCQ17scWermtbMIsw7MoUXi bMgQ== X-Gm-Message-State: AOAM533wIt8htZLxUaUUptM3u/5ETcYvX1FrVqAHQHhOp8jTZdaIwCWL cBHJerBqnohpsLBPzSvUwwOWqMbbhz4cMZlb X-Google-Smtp-Source: ABdhPJwYKwCfjodYVPcg8DGTSw+yIIvWG6u+2g7e8Io9PpYG0k1E0y52dOywGgbe9Ddkwr0W191sdaasLisaIioe X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a17:906:b043:: with SMTP id bj3mr4104115ejb.543.1605306027092; Fri, 13 Nov 2020 14:20:27 -0800 (PST) Date: Fri, 13 Nov 2020 23:19:55 +0100 In-Reply-To: Message-Id: <89bf275f233121fc0ad695693a072872d4deda5d.1605305978.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 05/19] kasan: allow VMAP_STACK for HW_TAGS mode From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Even though hardware tag-based mode currently doesn't support checking vmalloc allocations, it doesn't use shadow memory and works with VMAP_STACK as is. Change VMAP_STACK definition accordingly. Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Acked-by: Catalin Marinas Link: https://linux-review.googlesource.com/id/I3552cbc12321dec82cd7372676e9372a2eb452ac Reviewed-by: Dmitry Vyukov --- arch/Kconfig | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index 9ebdab3d0ca2..546869c3269d 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -921,16 +921,16 @@ config VMAP_STACK default y bool "Use a virtually-mapped stack" depends on HAVE_ARCH_VMAP_STACK - depends on !KASAN || KASAN_VMALLOC + depends on !KASAN || KASAN_HW_TAGS || KASAN_VMALLOC help Enable this if you want the use virtually-mapped kernel stacks with guard pages. This causes kernel stack overflows to be caught immediately rather than causing difficult-to-diagnose corruption. - To use this with KASAN, the architecture must support backing - virtual mappings with real shadow memory, and KASAN_VMALLOC must - be enabled. + To use this with software KASAN modes, the architecture must support + backing virtual mappings with real shadow memory, and KASAN_VMALLOC + must be enabled. config ARCH_OPTIONAL_KERNEL_RWX def_bool n From patchwork Fri Nov 13 22:19:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904961 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1628D1391 for ; Fri, 13 Nov 2020 22:20:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C29D0206F9 for ; Fri, 13 Nov 2020 22:20:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="Mr40u2Na" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C29D0206F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5EFEC6B007B; Fri, 13 Nov 2020 17:20:31 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 576286B007D; Fri, 13 Nov 2020 17:20:31 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3EAD36B007E; Fri, 13 Nov 2020 17:20:31 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0112.hostedemail.com [216.40.44.112]) by kanga.kvack.org (Postfix) with ESMTP id 1264B6B007B for ; Fri, 13 Nov 2020 17:20:31 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id B09AB181AEF07 for ; Fri, 13 Nov 2020 22:20:30 +0000 (UTC) X-FDA: 77480815020.26.time43_6006bb327312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin26.hostedemail.com (Postfix) with ESMTP id 8D5D91804B668 for ; Fri, 13 Nov 2020 22:20:30 +0000 (UTC) X-Spam-Summary: 1,0,0,8e22cefbfb6e3b0b,d41d8cd98f00b204,3rqavxwokchoylbpcwiltjemmejc.amkjglsv-kkityai.mpe@flex--andreyknvl.bounces.google.com,,RULES_HIT:41:152:355:379:541:800:960:967:968:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1534:1541:1593:1594:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3352:3865:3870:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4425:5007:6261:6653:6742:7875:7901:7903:8603:9025:9592:9969:10004:10400:11026:11473:11658:11914:12043:12114:12296:12297:12438:12555:12679:12895:13069:13161:13229:13311:13357:14096:14097:14181:14394:14659:14721:14877:21080:21365:21444:21451:21627:21939:21990:30003:30054,0,RBL:209.85.222.202:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-66.100.201.100 62.18.0.100;04ygpbihs8x7iahpmerpfo9a3cxreoc56o45wpbi1x4mcfx3jawnmefz3gg5rxi.fqtwkhryo6nye6omiyjdxsqgefz8mcjaiaq5ib6t4kj16q1kdpnim7n6j8bgi 5i.n-lbl X-HE-Tag: time43_6006bb327312 X-Filterd-Recvd-Size: 4676 Received: from mail-qk1-f202.google.com (mail-qk1-f202.google.com [209.85.222.202]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:29 +0000 (UTC) Received: by mail-qk1-f202.google.com with SMTP id o25so7630773qkj.1 for ; Fri, 13 Nov 2020 14:20:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=Jsr+yOOqZi2Nj6qjC5GqIPggpAtw8z6+rbSLs9IYMt8=; b=Mr40u2Na+744Lu2xB5XH5N7NJUu5zkCz5bvYutyv8bprHVKibdWrcZ08RTuHeheEFy 9jmFVE9u4pMUBRSsH8duKKAxLckAMS81HXO4Xd+IF5ViTs3z+IiCHYitEFwvlrygLVdl Fp1Iu8qVyEn4/oBWnVHCDlVMIVq/wT8DxVhflITdtyJe4/pBd2b+AM+ccSDdOjDiwdhT xJhX33OueoxZPgKqC96ik1I2rYb4dt4KvgmOopH5k0UnP/KslDnr1Hsw0cCUNHD6tkYZ pie5A+DraMOwWtJRPgmHqz7/llq9yX68G1qf4bknJXWyJ3BKIpXkFHmhJnfNhv7ht2kH CfAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Jsr+yOOqZi2Nj6qjC5GqIPggpAtw8z6+rbSLs9IYMt8=; b=mt1mFIRSPNyNjyU1qJDgKPiY2dG6+X7Gw4EjEci4Pc1gM1EPGBYWlMij9xecwdPvVF 5DIgJmR2TH1gXRKiIFRAYkDkkMzSqJxFEQBrim6YFDG5Cmx75ncbQIA+Kuy+qCRQP3Ke rjz7erhOKpAlau4Tku4b82SsRdPs2YzOaRnjLMDMyapwzWvWWuR2AXftbKCllyeTzYy0 s4Jj50n4sb6YCQhjIgy2A7Sz2uQF2yDa0NynESFSo6vjFBiNgi2wCaclu4pE3STB/vmn s/ctz3KS39STA9vcezTWmD32lpZejrod43+JgUAz2vbECv2taprpmJ/lbYamShWzJnGl 8Cvw== X-Gm-Message-State: AOAM532MWoH+GK2XmZT6uW7pr8EEcHLKXTTMbEgBxafEek15rHELPrF2 I8IrMRygdcMDD8YH68uo8N1w+2EMtN+tcIkn X-Google-Smtp-Source: ABdhPJwVIBicKScDqVn4tuFRRg0l3bRUAKZbS6vROY4eFdr5r/DaOJjnoyZljU2l4klUlopSKmUGVNXS2wDcCsNI X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:ad4:4633:: with SMTP id x19mr4837837qvv.11.1605306029294; Fri, 13 Nov 2020 14:20:29 -0800 (PST) Date: Fri, 13 Nov 2020 23:19:56 +0100 In-Reply-To: Message-Id: <157e9dfe43c5612ab028638c39ed5774b613449d.1605305978.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 06/19] kasan: remove __kasan_unpoison_stack From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There's no need for __kasan_unpoison_stack() helper, as it's only currently used in a single place. Removing it also removes unneeded arithmetic. No functional changes. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Reviewed-by: Marco Elver Link: https://linux-review.googlesource.com/id/Ie5ba549d445292fe629b4a96735e4034957bcc50 --- mm/kasan/common.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 7648a2452a01..fabd843eff3d 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -65,18 +65,12 @@ void kasan_unpoison_range(const void *address, size_t size) } #if CONFIG_KASAN_STACK -static void __kasan_unpoison_stack(struct task_struct *task, const void *sp) -{ - void *base = task_stack_page(task); - size_t size = sp - base; - - unpoison_range(base, size); -} - /* Unpoison the entire stack for a task. */ void kasan_unpoison_task_stack(struct task_struct *task) { - __kasan_unpoison_stack(task, task_stack_page(task) + THREAD_SIZE); + void *base = task_stack_page(task); + + unpoison_range(base, THREAD_SIZE); } /* Unpoison the stack for the current task beyond a watermark sp value. */ From patchwork Fri Nov 13 22:19:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904963 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7D729697 for ; Fri, 13 Nov 2020 22:20:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 31AF52223F for ; Fri, 13 Nov 2020 22:20:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="FZDE1cHf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 31AF52223F Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0906A6B007E; Fri, 13 Nov 2020 17:20:34 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 018E06B0080; Fri, 13 Nov 2020 17:20:33 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E4B666B0081; Fri, 13 Nov 2020 17:20:33 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0171.hostedemail.com [216.40.44.171]) by kanga.kvack.org (Postfix) with ESMTP id B8C0C6B007E for ; Fri, 13 Nov 2020 17:20:33 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 5DEFA8249980 for ; Fri, 13 Nov 2020 22:20:33 +0000 (UTC) X-FDA: 77480815146.22.fang73_0d1379227312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin22.hostedemail.com (Postfix) with ESMTP id 2B1AE18038E68 for ; Fri, 13 Nov 2020 22:20:33 +0000 (UTC) X-Spam-Summary: 1,0,0,842bcf405138350b,d41d8cd98f00b204,3rwavxwokchwfsiwj3ps0qlttlqj.htrqnsz2-rrp0fhp.twl@flex--andreyknvl.bounces.google.com,,RULES_HIT:1:2:41:152:355:379:541:800:960:966:967:968:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1593:1594:1605:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2859:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3865:3866:3867:3868:3870:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4051:4250:4321:4385:4425:4605:5007:6261:6653:6742:7901:7904:8603:8660:8957:9025:9592:9969:10004:11026:11473:11657:11658:11914:12043:12296:12297:12438:12555:12679:12895:12986:13148:13230:14096:14097:14394:14659:21067:21080:21324:21365:21444:21451:21611:21627:21796:21939:21990:30003:30012:30036:30045:30054:30055:30075,0,RBL:209.85.222.202:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100;04y86z6x6m6itjh9zmrrsj1nrzbwiycym3z35r5qnqayxb14zpygjyd55q47ypo. cw74if73 X-HE-Tag: fang73_0d1379227312 X-Filterd-Recvd-Size: 11664 Received: from mail-qk1-f202.google.com (mail-qk1-f202.google.com [209.85.222.202]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:32 +0000 (UTC) Received: by mail-qk1-f202.google.com with SMTP id v134so7587602qka.19 for ; Fri, 13 Nov 2020 14:20:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=0OoqMffsjTTjhNupRd8vITAFnXjp5GgZqecD32hcplM=; b=FZDE1cHfOmQH1YZ0oOuJY3WlcugPrZbEyGzYQnbBkc0IT7XwtcLMCjRyRdH8ImmzbI 2ocq7RBkVv0T7TZ/ItJjt3h3KQRB0IHBrbddDM9bFsr11+98UW1k9dmhda+JmYfs1Kp8 fZAqGERt02DDohNi3YXRNdYrB7CB0UMeRAfIAdIIviGWokrAidrBjArp+x9WQD78/1RV 35bpHCDJWfjRqal1SZ/i8ugrTnvHwj2sjFNoiFGL4Xy9cNl94xb6x2HYVUt2HgvwcT23 p50qE9wOzHhGqUbIl548ljaTXZ3q8gYTvA5ewEWeMuVkGPQTSmioHFPjDZAF28MSR4aU ilgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=0OoqMffsjTTjhNupRd8vITAFnXjp5GgZqecD32hcplM=; b=YBnIEuKLSmnPKlHnKGqVfYcq4VXZ2/VFLD4XTBnMY9vA/MgcCTbirdX+WAjaRaZn3u 1eFp3/hs4c0VQHwjxHaeHODVk06mpWdwJA2OPrchuJZ6HORLQEikvfLXkr1NprMMPMi8 2WokDo7nib0sc2RoCCrZSNUbE8o2toP3eV8teEO1O/g9bId3Aks5ZicecMKm6Q73wPPN sRjKCuRYSHz77zaez3msQGXPg7oYns2VfGoG3cWErE3AtPMDCch5KlVFuGYmiRQqN+H0 aij6dYlE+/a7prau6WebmeudbuwR5JS2JhOmdFdxbkgBUqjMsYa4+m+Xqjkk7rlXvSXz EZbw== X-Gm-Message-State: AOAM533It7i1C6Y+iP8ZQoGOHOgoVhGUcjwvHhj/NA1JMZFvBtqrozTd NUALyjVNxtTJKWLxllsvMF0UBGAcM912Fd3s X-Google-Smtp-Source: ABdhPJzDRFa4RTm2nDemL1ACuZ/b+tyJABImv2YFhSvfbxTLtvqZK6x+uT+0itTrLD1GAF2tY8oJ/5dvraqEEPYP X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a0c:b418:: with SMTP id u24mr4838113qve.4.1605306031906; Fri, 13 Nov 2020 14:20:31 -0800 (PST) Date: Fri, 13 Nov 2020 23:19:57 +0100 In-Reply-To: Message-Id: <4c2a23ccb3572459da7585a776d2d45f6e8b8580.1605305978.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 07/19] kasan: inline kasan_reset_tag for tag-based modes From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Using kasan_reset_tag() currently results in a function call. As it's called quite often from the allocator code, this leads to a noticeable slowdown. Move it to include/linux/kasan.h and turn it into a static inline function. Also remove the now unneeded reset_tag() internal KASAN macro and use kasan_reset_tag() instead. Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Link: https://linux-review.googlesource.com/id/I4d2061acfe91d480a75df00b07c22d8494ef14b5 Reviewed-by: Dmitry Vyukov --- include/linux/kasan.h | 5 ++++- mm/kasan/common.c | 6 +++--- mm/kasan/hw_tags.c | 9 ++------- mm/kasan/kasan.h | 4 ---- mm/kasan/report.c | 4 ++-- mm/kasan/report_hw_tags.c | 2 +- mm/kasan/report_sw_tags.c | 4 ++-- mm/kasan/shadow.c | 4 ++-- mm/kasan/sw_tags.c | 9 ++------- 9 files changed, 18 insertions(+), 29 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index f2109bf0c5f9..1594177f86bb 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -193,7 +193,10 @@ static inline void kasan_record_aux_stack(void *ptr) {} #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) -void *kasan_reset_tag(const void *addr); +static inline void *kasan_reset_tag(const void *addr) +{ + return (void *)arch_kasan_reset_tag(addr); +} bool kasan_report(unsigned long addr, size_t size, bool is_write, unsigned long ip); diff --git a/mm/kasan/common.c b/mm/kasan/common.c index fabd843eff3d..1ac4f435c679 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -180,14 +180,14 @@ size_t kasan_metadata_size(struct kmem_cache *cache) struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, const void *object) { - return (void *)reset_tag(object) + cache->kasan_info.alloc_meta_offset; + return kasan_reset_tag(object) + cache->kasan_info.alloc_meta_offset; } struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, const void *object) { BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); - return (void *)reset_tag(object) + cache->kasan_info.free_meta_offset; + return kasan_reset_tag(object) + cache->kasan_info.free_meta_offset; } void kasan_poison_slab(struct page *page) @@ -284,7 +284,7 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, tag = get_tag(object); tagged_object = object; - object = reset_tag(object); + object = kasan_reset_tag(object); if (is_kfence_address(object)) return false; diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 68e77363e58b..a34476764f1d 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -31,18 +31,13 @@ void __init kasan_init_hw_tags(void) pr_info("KernelAddressSanitizer initialized\n"); } -void *kasan_reset_tag(const void *addr) -{ - return reset_tag(addr); -} - void poison_range(const void *address, size_t size, u8 value) { /* Skip KFENCE memory if called explicitly outside of sl*b. */ if (is_kfence_address(address)) return; - hw_set_mem_tag_range(reset_tag(address), + hw_set_mem_tag_range(kasan_reset_tag(address), round_up(size, KASAN_GRANULE_SIZE), value); } @@ -52,7 +47,7 @@ void unpoison_range(const void *address, size_t size) if (is_kfence_address(address)) return; - hw_set_mem_tag_range(reset_tag(address), + hw_set_mem_tag_range(kasan_reset_tag(address), round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 0eab7e4cecb8..5e8cd2080369 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -248,15 +248,11 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) return addr; } #endif -#ifndef arch_kasan_reset_tag -#define arch_kasan_reset_tag(addr) ((void *)(addr)) -#endif #ifndef arch_kasan_get_tag #define arch_kasan_get_tag(addr) 0 #endif #define set_tag(addr, tag) ((void *)arch_kasan_set_tag((addr), (tag))) -#define reset_tag(addr) ((void *)arch_kasan_reset_tag(addr)) #define get_tag(addr) arch_kasan_get_tag(addr) #ifdef CONFIG_KASAN_HW_TAGS diff --git a/mm/kasan/report.c b/mm/kasan/report.c index df16bef0d810..76a0e3ae2049 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -328,7 +328,7 @@ void kasan_report_invalid_free(void *object, unsigned long ip) unsigned long flags; u8 tag = get_tag(object); - object = reset_tag(object); + object = kasan_reset_tag(object); #if IS_ENABLED(CONFIG_KUNIT) if (current->kunit_test) @@ -361,7 +361,7 @@ static void __kasan_report(unsigned long addr, size_t size, bool is_write, disable_trace_on_warning(); tagged_addr = (void *)addr; - untagged_addr = reset_tag(tagged_addr); + untagged_addr = kasan_reset_tag(tagged_addr); info.access_addr = tagged_addr; if (addr_has_metadata(untagged_addr)) diff --git a/mm/kasan/report_hw_tags.c b/mm/kasan/report_hw_tags.c index da543eb832cd..57114f0e14d1 100644 --- a/mm/kasan/report_hw_tags.c +++ b/mm/kasan/report_hw_tags.c @@ -22,7 +22,7 @@ const char *get_bug_type(struct kasan_access_info *info) void *find_first_bad_addr(void *addr, size_t size) { - return reset_tag(addr); + return kasan_reset_tag(addr); } void metadata_fetch_row(char *buffer, void *row) diff --git a/mm/kasan/report_sw_tags.c b/mm/kasan/report_sw_tags.c index 317100fd95b9..7604b46239d4 100644 --- a/mm/kasan/report_sw_tags.c +++ b/mm/kasan/report_sw_tags.c @@ -41,7 +41,7 @@ const char *get_bug_type(struct kasan_access_info *info) int i; tag = get_tag(info->access_addr); - addr = reset_tag(info->access_addr); + addr = kasan_reset_tag(info->access_addr); page = kasan_addr_to_page(addr); if (page && PageSlab(page)) { cache = page->slab_cache; @@ -72,7 +72,7 @@ const char *get_bug_type(struct kasan_access_info *info) void *find_first_bad_addr(void *addr, size_t size) { u8 tag = get_tag(addr); - void *p = reset_tag(addr); + void *p = kasan_reset_tag(addr); void *end = p + size; while (p < end && tag == *(u8 *)kasan_mem_to_shadow(p)) diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index d8a122f887a0..37153bd1c126 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -82,7 +82,7 @@ void poison_range(const void *address, size_t size, u8 value) * some of the callers (e.g. kasan_poison_object_data) pass tagged * addresses to this function. */ - address = reset_tag(address); + address = kasan_reset_tag(address); /* Skip KFENCE memory if called explicitly outside of sl*b. */ if (is_kfence_address(address)) @@ -103,7 +103,7 @@ void unpoison_range(const void *address, size_t size) * some of the callers (e.g. kasan_unpoison_object_data) pass tagged * addresses to this function. */ - address = reset_tag(address); + address = kasan_reset_tag(address); /* * Skip KFENCE memory if called explicitly outside of sl*b. Also note diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c index 6d7648cc3b98..e17de2619bbf 100644 --- a/mm/kasan/sw_tags.c +++ b/mm/kasan/sw_tags.c @@ -67,11 +67,6 @@ u8 random_tag(void) return (u8)(state % (KASAN_TAG_MAX + 1)); } -void *kasan_reset_tag(const void *addr) -{ - return reset_tag(addr); -} - bool check_memory_region(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { @@ -107,7 +102,7 @@ bool check_memory_region(unsigned long addr, size_t size, bool write, if (tag == KASAN_TAG_KERNEL) return true; - untagged_addr = reset_tag((const void *)addr); + untagged_addr = kasan_reset_tag((const void *)addr); if (unlikely(untagged_addr < kasan_shadow_to_mem((void *)KASAN_SHADOW_START))) { return !kasan_report(addr, size, write, ret_ip); @@ -126,7 +121,7 @@ bool check_memory_region(unsigned long addr, size_t size, bool write, bool check_invalid_free(void *addr) { u8 tag = get_tag(addr); - u8 shadow_byte = READ_ONCE(*(u8 *)kasan_mem_to_shadow(reset_tag(addr))); + u8 shadow_byte = READ_ONCE(*(u8 *)kasan_mem_to_shadow(kasan_reset_tag(addr))); return (shadow_byte == KASAN_TAG_INVALID) || (tag != KASAN_TAG_KERNEL && tag != shadow_byte); From patchwork Fri Nov 13 22:19:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904965 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D405A1391 for ; Fri, 13 Nov 2020 22:20:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9A63E206F9 for ; Fri, 13 Nov 2020 22:20:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="avk+IvhM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9A63E206F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4682E6B0081; Fri, 13 Nov 2020 17:20:36 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 37A146B0082; Fri, 13 Nov 2020 17:20:36 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 106AF6B0083; Fri, 13 Nov 2020 17:20:36 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0074.hostedemail.com [216.40.44.74]) by kanga.kvack.org (Postfix) with ESMTP id D57DE6B0081 for ; Fri, 13 Nov 2020 17:20:35 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 7E88C181AEF07 for ; Fri, 13 Nov 2020 22:20:35 +0000 (UTC) X-FDA: 77480815230.22.camp18_47095b827312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin22.hostedemail.com (Postfix) with ESMTP id 5F99018038E67 for ; Fri, 13 Nov 2020 22:20:35 +0000 (UTC) X-Spam-Summary: 1,0,0,c873ba4cc3437d68,d41d8cd98f00b204,3sgavxwokch8dqguh1nqyojrrjoh.frpolqx0-ppnydfn.ruj@flex--andreyknvl.bounces.google.com,,RULES_HIT:41:69:152:355:379:541:800:960:966:967:968:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1542:1593:1594:1711:1730:1747:1777:1792:1981:2194:2196:2199:2200:2393:2525:2559:2563:2682:2685:2859:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3353:3867:3868:3870:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4117:4321:4385:4425:5007:6261:6653:6742:7901:8603:8660:9025:9592:9969:10004:10400:11026:11473:11658:11914:12043:12296:12297:12438:12555:12679:12683:12895:12986:13148:13230:14096:14097:14181:14394:14659:14721:21080:21324:21365:21444:21451:21627:21939:21990:30003:30012:30054,0,RBL:209.85.222.202:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-66.100.201.100 62.18.0.100;04ygmtx8mxqyihywrq87c9cbtcjnpoc43i3gkgfuzjxwb8p818odcmx1bhoryko.mscq4bs8td9xdo7jsahek7jfs fcu1ewqy X-HE-Tag: camp18_47095b827312 X-Filterd-Recvd-Size: 6016 Received: from mail-qk1-f202.google.com (mail-qk1-f202.google.com [209.85.222.202]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:34 +0000 (UTC) Received: by mail-qk1-f202.google.com with SMTP id 198so6777208qkj.7 for ; Fri, 13 Nov 2020 14:20:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=YBdsxMVQVE97UJPn5Dc1HSKikQDmHg1H37Pt2LLE+Kk=; b=avk+IvhMlEZbmk9DmjnRO5FW8NWySM86BGQFkLT/09JJyb3bg/256owXPaIQNTTw+l 13Og2C+14ZkKV7O8e1B0R7/6G+IdEiu8xNf0Y8uh/3DJRzSSNnEbHhwBkpYwQvS4Dvr3 87rjbB+pQNHS3eleUgWZo+hpMz8uLG8TSmLKVfdfrHwhXOD3+xXbYQHF+7sQdg+ofvKN XP/DdI5ItTdvz5zVDwfjd/70KW2JGXsaqs/30TYn9umM3DXz9coQj5/FW/3sKbfqEdyX naOOcCQpj6mgZY8q56dtr2CJTx5SWs2jxYitvM0NA/jGV7eyTVjJ78xvtPRr5MRutn/E /wWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=YBdsxMVQVE97UJPn5Dc1HSKikQDmHg1H37Pt2LLE+Kk=; b=qrTN14BGIvdBIU80rs9S9S0GEyLz4LKJkzrq25zOnvbhcqVWnTPRxFuMf9YdZxw1op wMIaAGDnqxAKRzszkurz/gMz/166Iw/WvRTLs+jeSqu6ZwmUZBvo3liMf4Be6D1xL01P SVQQz/jhNSXfav1yaXDaPkZulc8c3KCo/nQezmLhbOvkrQK/BHUAr96ciDJjOAUxCH5m EMODnMGpeKhG4hHYnPFrhHgZzE1ut2wme3RRqVnFv14yyRfDJzwkrNNKkiOFomyQMTOO L5ZVTyHqXa1qtAucBEeAi8eZEciyxidOydMykF9AIdIyC/0/2lFYbip0wPsZlAgBMpgf Q90Q== X-Gm-Message-State: AOAM53004Z8au31VuKqJY7M0p0e/kfG0B7u6insMUDqnAzGpc7ZaLyof g4qVlDg2Ex40yiaHogLAxpYb8LPy2v15+v4h X-Google-Smtp-Source: ABdhPJwJbNc8LBA5L3yID4KPJo+8gcoSu+K34ENqO85OraM7oBvwXTkr8l2irPNsaon5CTeENcbj5cfem0w7nXTz X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:ad4:43ca:: with SMTP id o10mr4757439qvs.33.1605306034217; Fri, 13 Nov 2020 14:20:34 -0800 (PST) Date: Fri, 13 Nov 2020 23:19:58 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 08/19] kasan: inline random_tag for HW_TAGS From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Using random_tag() currently results in a function call. Move its definition to mm/kasan/kasan.h and turn it into a static inline function for hardware tag-based mode to avoid uneeded function calls. Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Link: https://linux-review.googlesource.com/id/Iac5b2faf9a912900e16cca6834d621f5d4abf427 Reviewed-by: Dmitry Vyukov --- mm/kasan/hw_tags.c | 5 ----- mm/kasan/kasan.h | 31 ++++++++++++++----------------- 2 files changed, 14 insertions(+), 22 deletions(-) diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index a34476764f1d..3cdd87d189f6 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -51,11 +51,6 @@ void unpoison_range(const void *address, size_t size) round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); } -u8 random_tag(void) -{ - return hw_get_random_tag(); -} - bool check_invalid_free(void *addr) { u8 ptr_tag = get_tag(addr); diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 5e8cd2080369..7876a2547b7d 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -190,6 +190,12 @@ static inline bool addr_has_metadata(const void *addr) #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ +#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) +void print_tags(u8 addr_tag, const void *addr); +#else +static inline void print_tags(u8 addr_tag, const void *addr) { } +#endif + bool check_invalid_free(void *addr); void *find_first_bad_addr(void *addr, size_t size); @@ -225,23 +231,6 @@ static inline void quarantine_reduce(void) { } static inline void quarantine_remove_cache(struct kmem_cache *cache) { } #endif -#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) - -void print_tags(u8 addr_tag, const void *addr); - -u8 random_tag(void); - -#else - -static inline void print_tags(u8 addr_tag, const void *addr) { } - -static inline u8 random_tag(void) -{ - return 0; -} - -#endif - #ifndef arch_kasan_set_tag static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) { @@ -281,6 +270,14 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) #endif /* CONFIG_KASAN_HW_TAGS */ +#ifdef CONFIG_KASAN_SW_TAGS +u8 random_tag(void); +#elif defined(CONFIG_KASAN_HW_TAGS) +static inline u8 random_tag(void) { return hw_get_random_tag(); } +#else +static inline u8 random_tag(void) { return 0; } +#endif + /* * Exported functions for interfaces called from assembly or from generated * code. Declarations here to avoid warning about missing declarations. From patchwork Fri Nov 13 22:19:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904967 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 513751391 for ; Fri, 13 Nov 2020 22:20:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1756B2223F for ; Fri, 13 Nov 2020 22:20:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="F7pTXHrB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1756B2223F Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F3AA86B0083; Fri, 13 Nov 2020 17:20:38 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E98CA6B0085; Fri, 13 Nov 2020 17:20:38 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC2C76B0087; Fri, 13 Nov 2020 17:20:38 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0137.hostedemail.com [216.40.44.137]) by kanga.kvack.org (Postfix) with ESMTP id 8AD166B0083 for ; Fri, 13 Nov 2020 17:20:38 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 32EE4181AEF07 for ; Fri, 13 Nov 2020 22:20:38 +0000 (UTC) X-FDA: 77480815356.22.dress68_1806d9c27312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin22.hostedemail.com (Postfix) with ESMTP id 07A8818038E67 for ; Fri, 13 Nov 2020 22:20:38 +0000 (UTC) X-Spam-Summary: 1,0,0,12f1b9973f95edf4,d41d8cd98f00b204,3taavxwokciefsiwj3ps0qlttlqj.htrqnsz2-rrp0fhp.twl@flex--andreyknvl.bounces.google.com,,RULES_HIT:41:152:355:379:541:800:960:966:967:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1541:1593:1594:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2693:2859:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3352:3865:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4425:4605:5007:6261:6653:6742:7901:8660:9025:9969:10004:10400:11026:11658:11914:12043:12296:12297:12438:12555:12679:12895:12986:13069:13148:13161:13229:13230:13311:13357:14096:14097:14181:14394:14659:14721:21080:21365:21444:21451:21627:21939:21990:30054:30070,0,RBL:209.85.219.73:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100;04ygictegnsr668a18dgphcnkao69oczwmpj3pdarxtao84pqjxqyfnds7mrewu.159eimm5xid1bcsimkco4gqf3u73eei71w593cf49z t3umkeyx X-HE-Tag: dress68_1806d9c27312 X-Filterd-Recvd-Size: 5259 Received: from mail-qv1-f73.google.com (mail-qv1-f73.google.com [209.85.219.73]) by imf16.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:37 +0000 (UTC) Received: by mail-qv1-f73.google.com with SMTP id v8so6993234qvq.12 for ; Fri, 13 Nov 2020 14:20:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=DB+H7HqoS0BsRTXGNuAwKRf0LlfChpk6LpCcjRkDvbU=; b=F7pTXHrBRZyanjUreyw5h4KDyLv+NRf731NUD0LblbQHl85fHVhh2SMtDJdjbhJ9l1 V1a34qWAYst3ZvrnchN55jYNpbeKhVPN9nhWrO7QdljcZl/3+SnQrsTqVpQMZAEC/Ypl xY2vfRSeYdsyYSzD6VNprwV9REKHfVTvTd3/wEG3VBwT+e4mXCdawCl0lC/PR96ce9wE IOey4TzXaA0dCARaAJ0JINK44loqjAWJje7Uhc+dz8SCJxp53GVxit3+LRu/rRUZlAlD N9o3aBrKwkxkpj2OUzLEBtTHi8snIk9nPlbNFRaIy/mrBGYqTvTe8u4cUOu0cB7UJgdL 2iPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=DB+H7HqoS0BsRTXGNuAwKRf0LlfChpk6LpCcjRkDvbU=; b=kMABYk6fnD/nNx8PhTAhST8nvunhau7w4RJYrp1jKKNNfH79YkV5my44YDjpKa7D/N t8SKcUcrYHxzM0HxBh2wXq73aiNPcNk3bosa5uX56sjln+MUIwxnteOjqxfkgNk+mJVe 55d5HltnihsIkbYdbVB9CXwkhRSFixNX+8Cxnyq1HCJdELsiz9eHYxaAso+yW4OZmVTY 89J3wT3pD52f1PhcR0hTfg27XlvFCtLzlZWOe5in+ZrVpxIjY2XLtXAN5gJTnzt1vIj6 4DkPK4hLXZT97rj18JVoCJnyA9q2R9tiH6vbj6bkdaP0BdnfzORwRBsELFYNR8t13kCB rFng== X-Gm-Message-State: AOAM532BC5dty7GCBzBP5jcjBubV8EPsPJyfCbVeKX/HlmZUSm6MOYbB NAuO++eEWilPCvhSu+8jmyjKN7qwc5rGKBiY X-Google-Smtp-Source: ABdhPJzcA8hQOxnSs80NVqCkReYj+UcQZlyg4LV+//1yN70ab39PQ0at237VgPxvdUPLib1Hm1Mg2FxPUtFHEpt5 X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a05:6214:12ed:: with SMTP id w13mr4550130qvv.23.1605306036643; Fri, 13 Nov 2020 14:20:36 -0800 (PST) Date: Fri, 13 Nov 2020 23:19:59 +0100 In-Reply-To: Message-Id: <4d64025c647190a8b7101d0b1da3deb922535a0d.1605305978.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 09/19] kasan: open-code kasan_unpoison_slab From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There's the external annotation kasan_unpoison_slab() that is currently defined as static inline and uses kasan_unpoison_range(). Open-code this function in mempool.c. Otherwise with an upcoming change this function will result in an unnecessary function call. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/Ia7c8b659f79209935cbaab3913bf7f082cc43a0e Reviewed-by: Marco Elver --- include/linux/kasan.h | 6 ------ mm/mempool.c | 2 +- 2 files changed, 1 insertion(+), 7 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 1594177f86bb..872bf145ddde 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -106,11 +106,6 @@ struct kasan_cache { int free_meta_offset; }; -size_t __ksize(const void *); -static inline void kasan_unpoison_slab(const void *ptr) -{ - kasan_unpoison_range(ptr, __ksize(ptr)); -} size_t kasan_metadata_size(struct kmem_cache *cache); bool kasan_save_enable_multi_shot(void); @@ -166,7 +161,6 @@ static inline bool kasan_slab_free(struct kmem_cache *s, void *object, return false; } -static inline void kasan_unpoison_slab(const void *ptr) { } static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } #endif /* CONFIG_KASAN */ diff --git a/mm/mempool.c b/mm/mempool.c index f473cdddaff0..583a9865b181 100644 --- a/mm/mempool.c +++ b/mm/mempool.c @@ -112,7 +112,7 @@ static __always_inline void kasan_poison_element(mempool_t *pool, void *element) static void kasan_unpoison_element(mempool_t *pool, void *element) { if (pool->alloc == mempool_alloc_slab || pool->alloc == mempool_kmalloc) - kasan_unpoison_slab(element); + kasan_unpoison_range(element, __ksize(element)); else if (pool->alloc == mempool_alloc_pages) kasan_alloc_pages(element, (unsigned long)pool->pool_data); } From patchwork Fri Nov 13 22:20:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904969 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9CD96697 for ; Fri, 13 Nov 2020 22:20:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 573B0206F9 for ; Fri, 13 Nov 2020 22:20:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="a7iYcqhi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 573B0206F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2B6306B0087; Fri, 13 Nov 2020 17:20:41 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2689A6B0088; Fri, 13 Nov 2020 17:20:41 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0E2836B0089; Fri, 13 Nov 2020 17:20:41 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0194.hostedemail.com [216.40.44.194]) by kanga.kvack.org (Postfix) with ESMTP id C7A426B0087 for ; Fri, 13 Nov 2020 17:20:40 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 75AF633CD for ; Fri, 13 Nov 2020 22:20:40 +0000 (UTC) X-FDA: 77480815440.08.ray65_21065f627312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin08.hostedemail.com (Postfix) with ESMTP id 562101819E772 for ; Fri, 13 Nov 2020 22:20:40 +0000 (UTC) X-Spam-Summary: 1,0,0,261fb9a0958c316e,d41d8cd98f00b204,3twavxwokciqivlzm6sv3towwotm.kwutqv25-uus3iks.wzo@flex--andreyknvl.bounces.google.com,,RULES_HIT:41:69:152:355:379:541:800:960:965:966:967:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1544:1593:1594:1711:1730:1747:1777:1792:1981:2194:2196:2199:2200:2393:2525:2559:2563:2682:2685:2859:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3355:3866:3867:3868:3870:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4118:4250:4321:4385:4390:4395:4425:5007:6261:6653:6742:7901:8603:8957:9025:9592:9969:10004:11026:11473:11658:11914:12043:12291:12296:12297:12438:12555:12679:12683:12895:14096:14097:14181:14394:14659:14721:21080:21324:21365:21444:21451:21627:21939:21990:30003:30012:30054:30075,0,RBL:209.85.160.201:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100;04yg4dicncckw6hodnfszqyk3b9raycnycwuuf1ina53tdbwpdtwqqi9s767tbm.75r689zms77zwwncp66rps 4x6iguz6 X-HE-Tag: ray65_21065f627312 X-Filterd-Recvd-Size: 7901 Received: from mail-qt1-f201.google.com (mail-qt1-f201.google.com [209.85.160.201]) by imf32.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:39 +0000 (UTC) Received: by mail-qt1-f201.google.com with SMTP id i20so6666573qtr.0 for ; Fri, 13 Nov 2020 14:20:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=MeFL8PhZCdNKf3Y6a938/A3o+3e79v6OLkm/7y5InEs=; b=a7iYcqhiOEzmoQJIygPhyPCEebvODyOmH8298XwMFZzhnPBc8slvcHMG5pc3xrTPwr eo3pI3jbeiA0ddVRw38Om6NCXa3ppgNdxUTV5JllWdXhzXa8f1tU3INCEQzxkABBvlBc vtnGSglODeaGY/C7wpVTNORwBeFKYAp1vYDzJa/o+IL6JH1hbPvjh9in06nqJiut5VyS 6JrAOcbT8gr53bGUESlGwZGxkUrsI5Pl7Tmybny5kxg4WPgaJ/JzxHtj/bFIrxZgvugG Yw8nsDOV4uCmmz7n9MasLcCSQoxYvuI5QcL+o6R/KpNGKPwxb5KEoglhvBpsMtdv16WZ d7dQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=MeFL8PhZCdNKf3Y6a938/A3o+3e79v6OLkm/7y5InEs=; b=npRK6JaX8uHQVLBP0T+2l8ThMRdYImPmf5giJVGIouOF1AAdkkBAUkRilxsFerjSbK dgg1RWHwIPD8HF8MNUcpnVa/cwnsqLSp86W6An1hMFSTZ1Jcg3YCTntgODf9cU8mIFII N06SGOacS4lStH5E+0lWPrNkfqf+y0+iv998tQEpvv9EK9shVv5zMHMqEimBiGq2K1tp mGJt3HCE3pUn470X1oTzycvnHyMbV6DuieZYuPDzeWo2fdGg3ODYvemEhscWAiAtO9ab 7WQNwAA9xwEyadqK671KbQDidfOPAHJD8iGnW9xP/KUPJ6cfg9D/mq4m/d5zRERZIeGu 5orA== X-Gm-Message-State: AOAM533kMBUJZnaL+POF3/HBVK/fjTRy2KQcZplxP50O/4eUzh/rPPno xfNU7KiH99gwIYKbjXaXpJBVrn/bVJpZ49g2 X-Google-Smtp-Source: ABdhPJxvwWRD6q+V00iGx8/Z5kxruSALO3/XNWuS6mhOxV+VJqQLBGqOmzGaQxVV5xj90d9/dy0xRas9NgWz59wT X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a0c:b6c4:: with SMTP id h4mr4429589qve.35.1605306039132; Fri, 13 Nov 2020 14:20:39 -0800 (PST) Date: Fri, 13 Nov 2020 23:20:00 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 10/19] kasan: inline (un)poison_range and check_invalid_free From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Using (un)poison_range() or check_invalid_free() currently results in function calls. Move their definitions to mm/kasan/kasan.h and turn them into static inline functions for hardware tag-based mode to avoid unneeded function calls. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/Ia9d8191024a12d1374675b3d27197f10193f50bb Reviewed-by: Marco Elver --- mm/kasan/hw_tags.c | 30 ------------------------------ mm/kasan/kasan.h | 45 ++++++++++++++++++++++++++++++++++++++++----- 2 files changed, 40 insertions(+), 35 deletions(-) diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 3cdd87d189f6..863fed4edd3f 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -10,7 +10,6 @@ #include #include -#include #include #include #include @@ -31,35 +30,6 @@ void __init kasan_init_hw_tags(void) pr_info("KernelAddressSanitizer initialized\n"); } -void poison_range(const void *address, size_t size, u8 value) -{ - /* Skip KFENCE memory if called explicitly outside of sl*b. */ - if (is_kfence_address(address)) - return; - - hw_set_mem_tag_range(kasan_reset_tag(address), - round_up(size, KASAN_GRANULE_SIZE), value); -} - -void unpoison_range(const void *address, size_t size) -{ - /* Skip KFENCE memory if called explicitly outside of sl*b. */ - if (is_kfence_address(address)) - return; - - hw_set_mem_tag_range(kasan_reset_tag(address), - round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); -} - -bool check_invalid_free(void *addr) -{ - u8 ptr_tag = get_tag(addr); - u8 mem_tag = hw_get_mem_tag(addr); - - return (mem_tag == KASAN_TAG_INVALID) || - (ptr_tag != KASAN_TAG_KERNEL && ptr_tag != mem_tag); -} - void kasan_set_free_info(struct kmem_cache *cache, void *object, u8 tag) { diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 7876a2547b7d..8aa83b7ad79e 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -3,6 +3,7 @@ #define __MM_KASAN_KASAN_H #include +#include #include #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) @@ -154,9 +155,6 @@ struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, const void *object); -void poison_range(const void *address, size_t size, u8 value); -void unpoison_range(const void *address, size_t size); - #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) static inline const void *kasan_shadow_to_mem(const void *shadow_addr) @@ -196,8 +194,6 @@ void print_tags(u8 addr_tag, const void *addr); static inline void print_tags(u8 addr_tag, const void *addr) { } #endif -bool check_invalid_free(void *addr); - void *find_first_bad_addr(void *addr, size_t size); const char *get_bug_type(struct kasan_access_info *info); void metadata_fetch_row(char *buffer, void *row); @@ -278,6 +274,45 @@ static inline u8 random_tag(void) { return hw_get_random_tag(); } static inline u8 random_tag(void) { return 0; } #endif +#ifdef CONFIG_KASAN_HW_TAGS + +static inline void poison_range(const void *address, size_t size, u8 value) +{ + /* Skip KFENCE memory if called explicitly outside of sl*b. */ + if (is_kfence_address(address)) + return; + + hw_set_mem_tag_range(kasan_reset_tag(address), + round_up(size, KASAN_GRANULE_SIZE), value); +} + +static inline void unpoison_range(const void *address, size_t size) +{ + /* Skip KFENCE memory if called explicitly outside of sl*b. */ + if (is_kfence_address(address)) + return; + + hw_set_mem_tag_range(kasan_reset_tag(address), + round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); +} + +static inline bool check_invalid_free(void *addr) +{ + u8 ptr_tag = get_tag(addr); + u8 mem_tag = hw_get_mem_tag(addr); + + return (mem_tag == KASAN_TAG_INVALID) || + (ptr_tag != KASAN_TAG_KERNEL && ptr_tag != mem_tag); +} + +#else /* CONFIG_KASAN_HW_TAGS */ + +void poison_range(const void *address, size_t size, u8 value); +void unpoison_range(const void *address, size_t size); +bool check_invalid_free(void *addr); + +#endif /* CONFIG_KASAN_HW_TAGS */ + /* * Exported functions for interfaces called from assembly or from generated * code. Declarations here to avoid warning about missing declarations. From patchwork Fri Nov 13 22:20:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904977 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CCA6D697 for ; Fri, 13 Nov 2020 22:20:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6C1A52223F for ; Fri, 13 Nov 2020 22:20:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="ZIkZiVFL" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6C1A52223F Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 21D0F6B0089; Fri, 13 Nov 2020 17:20:44 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1A19C6B008A; Fri, 13 Nov 2020 17:20:44 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 069756B008C; Fri, 13 Nov 2020 17:20:43 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0163.hostedemail.com [216.40.44.163]) by kanga.kvack.org (Postfix) with ESMTP id BDD1C6B0089 for ; Fri, 13 Nov 2020 17:20:43 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 5BDB333CD for ; Fri, 13 Nov 2020 22:20:43 +0000 (UTC) X-FDA: 77480815566.04.maid84_48158c927312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin04.hostedemail.com (Postfix) with ESMTP id 3EF59800CF08 for ; Fri, 13 Nov 2020 22:20:43 +0000 (UTC) X-Spam-Summary: 1,0,0,3ef022618fe76063,d41d8cd98f00b204,3uqavxwokciykxn1o8ux5vqyyqvo.mywvsx47-wwu5kmu.y1q@flex--andreyknvl.bounces.google.com,,RULES_HIT:1:41:152:355:379:541:800:960:966:967:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1593:1594:1605:1730:1747:1777:1792:1801:1981:2194:2196:2199:2200:2393:2525:2559:2564:2637:2682:2685:2859:2894:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4425:4605:5007:6119:6261:6653:6742:7875:7901:8603:8957:9025:9969:10004:11026:11473:11658:11914:12043:12291:12296:12297:12438:12555:12663:12679:12683:12895:12986:13053:13141:13161:13229:13230:14096:14097:14394:14659:21080:21325:21365:21444:21451:21627:21939:21966:21990:30054:30070,0,RBL:209.85.128.74:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100;04ygyys6d7uttd9eicj1xm3sis9o3op7w7fb7a7xwdwrygau5ki8z copd3k61 X-HE-Tag: maid84_48158c927312 X-Filterd-Recvd-Size: 14348 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) by imf26.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:42 +0000 (UTC) Received: by mail-wm1-f74.google.com with SMTP id 14so4721702wmg.1 for ; Fri, 13 Nov 2020 14:20:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=MxVAcTo0hk94ZX+Zq5q+mwXBwZIWhhZSGlgZNdsI0wo=; b=ZIkZiVFLBy+yabzozxOvR3Zd9N0uMrnSqsg1jqb7ahlGPFDQz+1TbXnthaA4GQhP1x JsJu5MQvJKt4fGk8SABnDNIQMmoc+uA7gIYCk3tNvGhy5S65rYsOSXY3kXr6rNkxWAXM K78sNGsH1TGF5VpVMxee7i3DyEJMZLEK26QyQQVoZeo16rIHhzeRmnZPOKNTHUx+HMRz 5QjVJG/DYqWK59TPHeBLsRRQqYSusASu1S9WEfxVCxD8dgeCfDIqIi/11BE9ClC46CbJ RzTIQx9V0vKqFu66BHoIO8x488w+TtM6ss8DejH+vK5YgdDYiITe2FdC+JXhzpOJpM4P OkIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=MxVAcTo0hk94ZX+Zq5q+mwXBwZIWhhZSGlgZNdsI0wo=; b=RNOwFq7wJql9kkpGwo8/bH9GYoFHW0mBgVBU17vQrd29vOhnteIU2i9sGZCf3BZwUH yG6DyFNc0aAmQNVYinRUx2rVyAPmR3QgY68KIfEqLl6oonrwKEp/3MCeikSz1zdMdoZw q/tMaJrM8yTJY8FcbQ/t6AmljX0SQ/YM4ukbXAIDs2SUc8TGbdEHDjvWxyKWLFDGe6HQ XrTwHowPJ1bFRf24Avfnfq+EyCWzXSOTZ+Dlmdql328yxF0NhDAmWvjN6fgA8d3Wndyw /9zaeTPajTfuSL6rx8y37tbWAx8SLdXgXwv5tE1bPc3pvGKgh2Ge7XIPKJw+wButAY+1 Nj8A== X-Gm-Message-State: AOAM532mqveHOqbO6HVDH0rx+CcYs3ri9XU8aOwIE1NEBtaEyRhTQ7tI 9dq3bLA3Vv2Xq6rkt0kxdpNl/FIMdk2/LMPo X-Google-Smtp-Source: ABdhPJyKR3meGMOmAU7hmbjaXFufu+qBx5atnm9eeN5FLHLvSQS99Iqt2PYCkuuJvvCa31Cp9bFz1kNn0+MItfaq X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:adf:f24b:: with SMTP id b11mr6350563wrp.342.1605306041610; Fri, 13 Nov 2020 14:20:41 -0800 (PST) Date: Fri, 13 Nov 2020 23:20:01 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 11/19] kasan: add and integrate kasan boot parameters From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hardware tag-based KASAN mode is intended to eventually be used in production as a security mitigation. Therefore there's a need for finer control over KASAN features and for an existence of a kill switch. This change adds a few boot parameters for hardware tag-based KASAN that allow to disable or otherwise control particular KASAN features. The features that can be controlled are: 1. Whether KASAN is enabled at all. 2. Whether KASAN collects and saves alloc/free stacks. 3. Whether KASAN panics on a detected bug or not. With this change a new boot parameter kasan.mode allows to choose one of three main modes: - kasan.mode=off - KASAN is disabled, no tag checks are performed - kasan.mode=prod - only essential production features are enabled - kasan.mode=full - all KASAN features are enabled The chosen mode provides default control values for the features mentioned above. However it's also possible to override the default values by providing: - kasan.stacktrace=off/on - enable alloc/free stack collection (default: on for mode=full, otherwise off) - kasan.fault=report/panic - only report tag fault or also panic (default: report) If kasan.mode parameter is not provided, it defaults to full when CONFIG_DEBUG_KERNEL is enabled, and to prod otherwise. It is essential that switching between these modes doesn't require rebuilding the kernel with different configs, as this is required by the Android GKI (Generic Kernel Image) initiative [1]. [1] https://source.android.com/devices/architecture/kernel/generic-kernel-image Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/If7d37003875b2ed3e0935702c8015c223d6416a4 Reviewed-by: Marco Elver Reviewed-by: Dmitry Vyukov --- mm/kasan/common.c | 22 +++++-- mm/kasan/hw_tags.c | 151 +++++++++++++++++++++++++++++++++++++++++++++ mm/kasan/kasan.h | 16 +++++ mm/kasan/report.c | 14 ++++- 4 files changed, 196 insertions(+), 7 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 1ac4f435c679..a11e3e75eb08 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -135,6 +135,11 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, unsigned int redzone_size; int redzone_adjust; + if (!kasan_stack_collection_enabled()) { + *flags |= SLAB_KASAN; + return; + } + /* Add alloc meta. */ cache->kasan_info.alloc_meta_offset = *size; *size += sizeof(struct kasan_alloc_meta); @@ -171,6 +176,8 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, size_t kasan_metadata_size(struct kmem_cache *cache) { + if (!kasan_stack_collection_enabled()) + return 0; return (cache->kasan_info.alloc_meta_offset ? sizeof(struct kasan_alloc_meta) : 0) + (cache->kasan_info.free_meta_offset ? @@ -263,11 +270,13 @@ void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, { struct kasan_alloc_meta *alloc_meta; - if (!(cache->flags & SLAB_KASAN)) - return (void *)object; + if (kasan_stack_collection_enabled()) { + if (!(cache->flags & SLAB_KASAN)) + return (void *)object; - alloc_meta = kasan_get_alloc_meta(cache, object); - __memset(alloc_meta, 0, sizeof(*alloc_meta)); + alloc_meta = kasan_get_alloc_meta(cache, object); + __memset(alloc_meta, 0, sizeof(*alloc_meta)); + } if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) object = set_tag(object, assign_tag(cache, object, true, false)); @@ -307,6 +316,9 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, rounded_up_size = round_up(cache->object_size, KASAN_GRANULE_SIZE); poison_range(object, rounded_up_size, KASAN_KMALLOC_FREE); + if (!kasan_stack_collection_enabled()) + return false; + if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || unlikely(!(cache->flags & SLAB_KASAN))) return false; @@ -357,7 +369,7 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, poison_range((void *)redzone_start, redzone_end - redzone_start, KASAN_KMALLOC_REDZONE); - if (cache->flags & SLAB_KASAN) + if (kasan_stack_collection_enabled() && (cache->flags & SLAB_KASAN)) set_alloc_info(cache, (void *)object, flags); return set_tag(object, tag); diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 863fed4edd3f..30ce88935e9d 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -8,18 +8,115 @@ #define pr_fmt(fmt) "kasan: " fmt +#include #include #include #include #include +#include #include #include #include "kasan.h" +enum kasan_arg_mode { + KASAN_ARG_MODE_DEFAULT, + KASAN_ARG_MODE_OFF, + KASAN_ARG_MODE_PROD, + KASAN_ARG_MODE_FULL, +}; + +enum kasan_arg_stacktrace { + KASAN_ARG_STACKTRACE_DEFAULT, + KASAN_ARG_STACKTRACE_OFF, + KASAN_ARG_STACKTRACE_ON, +}; + +enum kasan_arg_fault { + KASAN_ARG_FAULT_DEFAULT, + KASAN_ARG_FAULT_REPORT, + KASAN_ARG_FAULT_PANIC, +}; + +static enum kasan_arg_mode kasan_arg_mode __ro_after_init; +static enum kasan_arg_stacktrace kasan_arg_stacktrace __ro_after_init; +static enum kasan_arg_fault kasan_arg_fault __ro_after_init; + +/* Whether KASAN is enabled at all. */ +DEFINE_STATIC_KEY_FALSE_RO(kasan_flag_enabled); +EXPORT_SYMBOL(kasan_flag_enabled); + +/* Whether to collect alloc/free stack traces. */ +DEFINE_STATIC_KEY_FALSE_RO(kasan_flag_stacktrace); + +/* Whether panic or disable tag checking on fault. */ +bool kasan_flag_panic __ro_after_init; + +/* kasan.mode=off/prod/full */ +static int __init early_kasan_mode(char *arg) +{ + if (!arg) + return -EINVAL; + + if (!strcmp(arg, "off")) + kasan_arg_mode = KASAN_ARG_MODE_OFF; + else if (!strcmp(arg, "prod")) + kasan_arg_mode = KASAN_ARG_MODE_PROD; + else if (!strcmp(arg, "full")) + kasan_arg_mode = KASAN_ARG_MODE_FULL; + else + return -EINVAL; + + return 0; +} +early_param("kasan.mode", early_kasan_mode); + +/* kasan.stack=off/on */ +static int __init early_kasan_flag_stacktrace(char *arg) +{ + if (!arg) + return -EINVAL; + + if (!strcmp(arg, "off")) + kasan_arg_stacktrace = KASAN_ARG_STACKTRACE_OFF; + else if (!strcmp(arg, "on")) + kasan_arg_stacktrace = KASAN_ARG_STACKTRACE_ON; + else + return -EINVAL; + + return 0; +} +early_param("kasan.stacktrace", early_kasan_flag_stacktrace); + +/* kasan.fault=report/panic */ +static int __init early_kasan_fault(char *arg) +{ + if (!arg) + return -EINVAL; + + if (!strcmp(arg, "report")) + kasan_arg_fault = KASAN_ARG_FAULT_REPORT; + else if (!strcmp(arg, "panic")) + kasan_arg_fault = KASAN_ARG_FAULT_PANIC; + else + return -EINVAL; + + return 0; +} +early_param("kasan.fault", early_kasan_fault); + /* kasan_init_hw_tags_cpu() is called for each CPU. */ void kasan_init_hw_tags_cpu(void) { + /* + * There's no need to check that the hardware is MTE-capable here, + * as this function is only called for MTE-capable hardware. + */ + + /* If KASAN is disabled, do nothing. */ + if (kasan_arg_mode == KASAN_ARG_MODE_OFF) + return; + hw_init_tags(KASAN_TAG_MAX); hw_enable_tagging(); } @@ -27,6 +124,60 @@ void kasan_init_hw_tags_cpu(void) /* kasan_init_hw_tags() is called once on boot CPU. */ void __init kasan_init_hw_tags(void) { + /* If hardware doesn't support MTE, do nothing. */ + if (!system_supports_mte()) + return; + + /* Choose KASAN mode if kasan boot parameter is not provided. */ + if (kasan_arg_mode == KASAN_ARG_MODE_DEFAULT) { + if (IS_ENABLED(CONFIG_DEBUG_KERNEL)) + kasan_arg_mode = KASAN_ARG_MODE_FULL; + else + kasan_arg_mode = KASAN_ARG_MODE_PROD; + } + + /* Preset parameter values based on the mode. */ + switch (kasan_arg_mode) { + case KASAN_ARG_MODE_DEFAULT: + /* Shouldn't happen as per the check above. */ + WARN_ON(1); + return; + case KASAN_ARG_MODE_OFF: + /* If KASAN is disabled, do nothing. */ + return; + case KASAN_ARG_MODE_PROD: + static_branch_enable(&kasan_flag_enabled); + break; + case KASAN_ARG_MODE_FULL: + static_branch_enable(&kasan_flag_enabled); + static_branch_enable(&kasan_flag_stacktrace); + break; + } + + /* Now, optionally override the presets. */ + + switch (kasan_arg_stacktrace) { + case KASAN_ARG_STACKTRACE_DEFAULT: + break; + case KASAN_ARG_STACKTRACE_OFF: + static_branch_disable(&kasan_flag_stacktrace); + break; + case KASAN_ARG_STACKTRACE_ON: + static_branch_enable(&kasan_flag_stacktrace); + break; + } + + switch (kasan_arg_fault) { + case KASAN_ARG_FAULT_DEFAULT: + break; + case KASAN_ARG_FAULT_REPORT: + kasan_flag_panic = false; + break; + case KASAN_ARG_FAULT_PANIC: + kasan_flag_panic = true; + break; + } + pr_info("KernelAddressSanitizer initialized\n"); } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 8aa83b7ad79e..d01a5ac34f70 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -6,6 +6,22 @@ #include #include +#ifdef CONFIG_KASAN_HW_TAGS +#include +DECLARE_STATIC_KEY_FALSE(kasan_flag_stacktrace); +static inline bool kasan_stack_collection_enabled(void) +{ + return static_branch_unlikely(&kasan_flag_stacktrace); +} +#else +static inline bool kasan_stack_collection_enabled(void) +{ + return true; +} +#endif + +extern bool kasan_flag_panic __ro_after_init; + #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) #define KASAN_GRANULE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) #else diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 76a0e3ae2049..ffa6076b1710 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -99,6 +99,10 @@ static void end_report(unsigned long *flags) panic_on_warn = 0; panic("panic_on_warn set ...\n"); } +#ifdef CONFIG_KASAN_HW_TAGS + if (kasan_flag_panic) + panic("kasan.fault=panic set ...\n"); +#endif kasan_enable_current(); } @@ -161,8 +165,8 @@ static void describe_object_addr(struct kmem_cache *cache, void *object, (void *)(object_addr + cache->object_size)); } -static void describe_object(struct kmem_cache *cache, void *object, - const void *addr, u8 tag) +static void describe_object_stacks(struct kmem_cache *cache, void *object, + const void *addr, u8 tag) { struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object); @@ -190,7 +194,13 @@ static void describe_object(struct kmem_cache *cache, void *object, } #endif } +} +static void describe_object(struct kmem_cache *cache, void *object, + const void *addr, u8 tag) +{ + if (kasan_stack_collection_enabled()) + describe_object_stacks(cache, object, addr, tag); describe_object_addr(cache, object, addr); } From patchwork Fri Nov 13 22:20:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904979 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 63B7A1391 for ; Fri, 13 Nov 2020 22:20:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0C730206F9 for ; Fri, 13 Nov 2020 22:20:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="pmVSO2bc" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0C730206F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9B9186B008C; Fri, 13 Nov 2020 17:20:46 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 943476B0092; Fri, 13 Nov 2020 17:20:46 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 76DCA6B0093; Fri, 13 Nov 2020 17:20:46 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0150.hostedemail.com [216.40.44.150]) by kanga.kvack.org (Postfix) with ESMTP id 34CC36B008C for ; Fri, 13 Nov 2020 17:20:46 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id D73678249980 for ; Fri, 13 Nov 2020 22:20:45 +0000 (UTC) X-FDA: 77480815650.17.juice56_550b2fc27312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin17.hostedemail.com (Postfix) with ESMTP id BD48D180D0181 for ; Fri, 13 Nov 2020 22:20:45 +0000 (UTC) X-Spam-Summary: 1,0,0,52461edaf60ed464,d41d8cd98f00b204,3uwavxwokcigmzp3qawz7xs00sxq.o0yxuz69-yyw7mow.03s@flex--andreyknvl.bounces.google.com,,RULES_HIT:41:69:152:327:355:379:541:800:960:966:967:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1593:1594:1605:1730:1747:1777:1792:1981:2194:2196:2198:2199:2200:2201:2393:2525:2559:2563:2682:2685:2731:2859:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3740:3865:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4425:4605:5007:6119:6261:6653:6742:7901:7903:8603:8957:9025:9036:9592:9969:10004:11026:11473:11658:11914:12043:12291:12295:12296:12297:12438:12555:12679:12683:12895:12986:14096:14097:14394:14659:21080:21365:21444:21451:21627:21939:21966:21990:30003:30054:30070,0,RBL:209.85.208.73:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-66.100.201.100 62.18.84.100;04yryq97hhpnmqy7j33c86esuc5r1ypd9p1i8pfih5sipo3wmko7gh4todkjd7o.jxsm5rgtf3iq4 7en5ixzm X-HE-Tag: juice56_550b2fc27312 X-Filterd-Recvd-Size: 21353 Received: from mail-ed1-f73.google.com (mail-ed1-f73.google.com [209.85.208.73]) by imf21.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:45 +0000 (UTC) Received: by mail-ed1-f73.google.com with SMTP id bm10so3412308edb.1 for ; Fri, 13 Nov 2020 14:20:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=BSF3Q9byE1BU2/WI2HEHM9hwbewwBwz05B55UYa5pyQ=; b=pmVSO2bc7BAnXEL806cOp0lXGKb0ExI5J9r1GKDimUSP+2+CJBc+mMPgrFJEDSa9r0 vGGXrN/2kvD0868Vh9kfkIDUQPPkJOaTjp24IxtWaKTGf+vORF1lFXqLYntTx2ZACU11 RoQrckOXFKSDAwECPaWeTc5m2ey5a4h7dlVTgGWX3Jd6qfoBsErOuBqbvrBNjWQlyJzB hAFcZyDzyD62Y2tGrz/Gh0eREox+aqX6O5jVYibM+oE15ntLYhYPXKYJicgw84UNRIac 1jY805fCWPbMmY6DpTKl9XfpRdeQJFpa8mLluK3uoI5hPYLxYcg1qfzLWFd0dQNR4joM JOrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=BSF3Q9byE1BU2/WI2HEHM9hwbewwBwz05B55UYa5pyQ=; b=HYZed2Wf7THM68okxeTjOLWqeVA4yJw1tKwGmdqDgmUHZC28VvYIX9nkXrPmiWz1ny tsVL1z0f+O0oCDBwsH+AiBJ15EBnVVYhubGnDhd8RhBQs9eBCVIoHXi1noT75kkwjQtK XhsfLe09MSawDWOSJRbJUQVBG7v0vqUIDzbGFsI9WIZkEvljYHZy4y96lyXjZ3il2tam y3/VmORM6ff48H7+DQ0Kzp21Crm2kpipdZRmRCV39p/7NWqmD0nYfSrYnbLYiUYTgNPj OudIRz5HeY08moG4CntnNWQnXI9w2QFeof/hKgAWVy0CxpO1QGPVTeCYJBdGF89Sg9ob c5HQ== X-Gm-Message-State: AOAM531cySE6TQOZtkIZgXHEO88a+VpMquxIuymnJ91jelkrcUGdNBo+ TW+W3szz1GDAXR7QAKoT11hDFGMZTiutKs8d X-Google-Smtp-Source: ABdhPJzyQn65feDAmDQXGjO6dz1WwqkQZkbmUIZuaPOpHyRX87n/e5TKqCfcdy/zSFJPzQU1P4+lF/xzrBH+ZgxM X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a17:906:1a0b:: with SMTP id i11mr4339374ejf.404.1605306043965; Fri, 13 Nov 2020 14:20:43 -0800 (PST) Date: Fri, 13 Nov 2020 23:20:02 +0100 In-Reply-To: Message-Id: <68b9e818c971a28c4b8082d6dbac52967553bd73.1605305978.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 12/19] kasan, mm: check kasan_enabled in annotations From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov , Vincenzo Frascino X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Declare the kasan_enabled static key in include/linux/kasan.h and in include/linux/mm.h and check it in all kasan annotations. This allows to avoid any slowdown caused by function calls when kasan_enabled is disabled. Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/I2589451d3c96c97abbcbf714baabe6161c6f153e Reviewed-by: Marco Elver Reviewed-by: Dmitry Vyukov --- include/linux/kasan.h | 213 ++++++++++++++++++++++++++++++++---------- include/linux/mm.h | 22 +++-- mm/kasan/common.c | 56 +++++------ 3 files changed, 210 insertions(+), 81 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 872bf145ddde..6bd95243a583 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -2,6 +2,7 @@ #ifndef _LINUX_KASAN_H #define _LINUX_KASAN_H +#include #include struct kmem_cache; @@ -74,54 +75,176 @@ static inline void kasan_disable_current(void) {} #ifdef CONFIG_KASAN -void kasan_unpoison_range(const void *address, size_t size); +struct kasan_cache { + int alloc_meta_offset; + int free_meta_offset; +}; -void kasan_alloc_pages(struct page *page, unsigned int order); -void kasan_free_pages(struct page *page, unsigned int order); +#ifdef CONFIG_KASAN_HW_TAGS +DECLARE_STATIC_KEY_FALSE(kasan_flag_enabled); +static __always_inline bool kasan_enabled(void) +{ + return static_branch_likely(&kasan_flag_enabled); +} +#else +static inline bool kasan_enabled(void) +{ + return true; +} +#endif -void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, - slab_flags_t *flags); +void __kasan_unpoison_range(const void *addr, size_t size); +static __always_inline void kasan_unpoison_range(const void *addr, size_t size) +{ + if (kasan_enabled()) + __kasan_unpoison_range(addr, size); +} -void kasan_poison_slab(struct page *page); -void kasan_unpoison_object_data(struct kmem_cache *cache, void *object); -void kasan_poison_object_data(struct kmem_cache *cache, void *object); -void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, - const void *object); +void __kasan_alloc_pages(struct page *page, unsigned int order); +static __always_inline void kasan_alloc_pages(struct page *page, + unsigned int order) +{ + if (kasan_enabled()) + __kasan_alloc_pages(page, order); +} -void * __must_check kasan_kmalloc_large(const void *ptr, size_t size, - gfp_t flags); -void kasan_kfree_large(void *ptr, unsigned long ip); -void kasan_poison_kfree(void *ptr, unsigned long ip); -void * __must_check kasan_kmalloc(struct kmem_cache *s, const void *object, - size_t size, gfp_t flags); -void * __must_check kasan_krealloc(const void *object, size_t new_size, - gfp_t flags); +void __kasan_free_pages(struct page *page, unsigned int order); +static __always_inline void kasan_free_pages(struct page *page, + unsigned int order) +{ + if (kasan_enabled()) + __kasan_free_pages(page, order); +} -void * __must_check kasan_slab_alloc(struct kmem_cache *s, void *object, - gfp_t flags); -bool kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); +void __kasan_cache_create(struct kmem_cache *cache, unsigned int *size, + slab_flags_t *flags); +static __always_inline void kasan_cache_create(struct kmem_cache *cache, + unsigned int *size, slab_flags_t *flags) +{ + if (kasan_enabled()) + __kasan_cache_create(cache, size, flags); +} -struct kasan_cache { - int alloc_meta_offset; - int free_meta_offset; -}; +size_t __kasan_metadata_size(struct kmem_cache *cache); +static __always_inline size_t kasan_metadata_size(struct kmem_cache *cache) +{ + if (kasan_enabled()) + return __kasan_metadata_size(cache); + return 0; +} + +void __kasan_poison_slab(struct page *page); +static __always_inline void kasan_poison_slab(struct page *page) +{ + if (kasan_enabled()) + return __kasan_poison_slab(page); +} + +void __kasan_unpoison_object_data(struct kmem_cache *cache, void *object); +static __always_inline void kasan_unpoison_object_data(struct kmem_cache *cache, + void *object) +{ + if (kasan_enabled()) + return __kasan_unpoison_object_data(cache, object); +} + +void __kasan_poison_object_data(struct kmem_cache *cache, void *object); +static __always_inline void kasan_poison_object_data(struct kmem_cache *cache, + void *object) +{ + if (kasan_enabled()) + __kasan_poison_object_data(cache, object); +} + +void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache, + const void *object); +static __always_inline void * __must_check kasan_init_slab_obj( + struct kmem_cache *cache, const void *object) +{ + if (kasan_enabled()) + return __kasan_init_slab_obj(cache, object); + return (void *)object; +} + +bool __kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); +static __always_inline bool kasan_slab_free(struct kmem_cache *s, void *object, + unsigned long ip) +{ + if (kasan_enabled()) + return __kasan_slab_free(s, object, ip); + return false; +} + +void * __must_check __kasan_slab_alloc(struct kmem_cache *s, + void *object, gfp_t flags); +static __always_inline void * __must_check kasan_slab_alloc( + struct kmem_cache *s, void *object, gfp_t flags) +{ + if (kasan_enabled()) + return __kasan_slab_alloc(s, object, flags); + return object; +} + +void * __must_check __kasan_kmalloc(struct kmem_cache *s, const void *object, + size_t size, gfp_t flags); +static __always_inline void * __must_check kasan_kmalloc(struct kmem_cache *s, + const void *object, size_t size, gfp_t flags) +{ + if (kasan_enabled()) + return __kasan_kmalloc(s, object, size, flags); + return (void *)object; +} -size_t kasan_metadata_size(struct kmem_cache *cache); +void * __must_check __kasan_kmalloc_large(const void *ptr, + size_t size, gfp_t flags); +static __always_inline void * __must_check kasan_kmalloc_large(const void *ptr, + size_t size, gfp_t flags) +{ + if (kasan_enabled()) + return __kasan_kmalloc_large(ptr, size, flags); + return (void *)ptr; +} + +void * __must_check __kasan_krealloc(const void *object, + size_t new_size, gfp_t flags); +static __always_inline void * __must_check kasan_krealloc(const void *object, + size_t new_size, gfp_t flags) +{ + if (kasan_enabled()) + return __kasan_krealloc(object, new_size, flags); + return (void *)object; +} + +void __kasan_poison_kfree(void *ptr, unsigned long ip); +static __always_inline void kasan_poison_kfree(void *ptr, unsigned long ip) +{ + if (kasan_enabled()) + __kasan_poison_kfree(ptr, ip); +} + +void __kasan_kfree_large(void *ptr, unsigned long ip); +static __always_inline void kasan_kfree_large(void *ptr, unsigned long ip) +{ + if (kasan_enabled()) + __kasan_kfree_large(ptr, ip); +} bool kasan_save_enable_multi_shot(void); void kasan_restore_multi_shot(bool enabled); #else /* CONFIG_KASAN */ +static inline bool kasan_enabled(void) +{ + return false; +} static inline void kasan_unpoison_range(const void *address, size_t size) {} - static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} - static inline void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) {} - +static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } static inline void kasan_poison_slab(struct page *page) {} static inline void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) {} @@ -132,36 +255,32 @@ static inline void *kasan_init_slab_obj(struct kmem_cache *cache, { return (void *)object; } - -static inline void *kasan_kmalloc_large(void *ptr, size_t size, gfp_t flags) +static inline bool kasan_slab_free(struct kmem_cache *s, void *object, + unsigned long ip) { - return ptr; + return false; +} +static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, + gfp_t flags) +{ + return object; } -static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} -static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {} static inline void *kasan_kmalloc(struct kmem_cache *s, const void *object, size_t size, gfp_t flags) { return (void *)object; } +static inline void *kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) +{ + return (void *)ptr; +} static inline void *kasan_krealloc(const void *object, size_t new_size, gfp_t flags) { return (void *)object; } - -static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, - gfp_t flags) -{ - return object; -} -static inline bool kasan_slab_free(struct kmem_cache *s, void *object, - unsigned long ip) -{ - return false; -} - -static inline size_t kasan_metadata_size(struct kmem_cache *cache) { return 0; } +static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {} +static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} #endif /* CONFIG_KASAN */ diff --git a/include/linux/mm.h b/include/linux/mm.h index 947f4f1a6536..24f47e140a4c 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -31,6 +31,7 @@ #include #include #include +#include struct mempolicy; struct anon_vma; @@ -1415,22 +1416,30 @@ static inline bool cpupid_match_pid(struct task_struct *task, int cpupid) #endif /* CONFIG_NUMA_BALANCING */ #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) + static inline u8 page_kasan_tag(const struct page *page) { - return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; + if (kasan_enabled()) + return (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; + return 0xff; } static inline void page_kasan_tag_set(struct page *page, u8 tag) { - page->flags &= ~(KASAN_TAG_MASK << KASAN_TAG_PGSHIFT); - page->flags |= (tag & KASAN_TAG_MASK) << KASAN_TAG_PGSHIFT; + if (kasan_enabled()) { + page->flags &= ~(KASAN_TAG_MASK << KASAN_TAG_PGSHIFT); + page->flags |= (tag & KASAN_TAG_MASK) << KASAN_TAG_PGSHIFT; + } } static inline void page_kasan_tag_reset(struct page *page) { - page_kasan_tag_set(page, 0xff); + if (kasan_enabled()) + page_kasan_tag_set(page, 0xff); } -#else + +#else /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */ + static inline u8 page_kasan_tag(const struct page *page) { return 0xff; @@ -1438,7 +1447,8 @@ static inline u8 page_kasan_tag(const struct page *page) static inline void page_kasan_tag_set(struct page *page, u8 tag) { } static inline void page_kasan_tag_reset(struct page *page) { } -#endif + +#endif /* CONFIG_KASAN_SW_TAGS || CONFIG_KASAN_HW_TAGS */ static inline struct zone *page_zone(const struct page *page) { diff --git a/mm/kasan/common.c b/mm/kasan/common.c index a11e3e75eb08..17918bd20ed9 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -59,7 +59,7 @@ void kasan_disable_current(void) } #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ -void kasan_unpoison_range(const void *address, size_t size) +void __kasan_unpoison_range(const void *address, size_t size) { unpoison_range(address, size); } @@ -87,7 +87,7 @@ asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) } #endif /* CONFIG_KASAN_STACK */ -void kasan_alloc_pages(struct page *page, unsigned int order) +void __kasan_alloc_pages(struct page *page, unsigned int order) { u8 tag; unsigned long i; @@ -101,7 +101,7 @@ void kasan_alloc_pages(struct page *page, unsigned int order) unpoison_range(page_address(page), PAGE_SIZE << order); } -void kasan_free_pages(struct page *page, unsigned int order) +void __kasan_free_pages(struct page *page, unsigned int order) { if (likely(!PageHighMem(page))) poison_range(page_address(page), @@ -128,8 +128,8 @@ static inline unsigned int optimal_redzone(unsigned int object_size) object_size <= (1 << 16) - 1024 ? 1024 : 2048; } -void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, - slab_flags_t *flags) +void __kasan_cache_create(struct kmem_cache *cache, unsigned int *size, + slab_flags_t *flags) { unsigned int orig_size = *size; unsigned int redzone_size; @@ -174,7 +174,7 @@ void kasan_cache_create(struct kmem_cache *cache, unsigned int *size, *flags |= SLAB_KASAN; } -size_t kasan_metadata_size(struct kmem_cache *cache) +size_t __kasan_metadata_size(struct kmem_cache *cache) { if (!kasan_stack_collection_enabled()) return 0; @@ -197,7 +197,7 @@ struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, return kasan_reset_tag(object) + cache->kasan_info.free_meta_offset; } -void kasan_poison_slab(struct page *page) +void __kasan_poison_slab(struct page *page) { unsigned long i; @@ -207,12 +207,12 @@ void kasan_poison_slab(struct page *page) KASAN_KMALLOC_REDZONE); } -void kasan_unpoison_object_data(struct kmem_cache *cache, void *object) +void __kasan_unpoison_object_data(struct kmem_cache *cache, void *object) { unpoison_range(object, cache->object_size); } -void kasan_poison_object_data(struct kmem_cache *cache, void *object) +void __kasan_poison_object_data(struct kmem_cache *cache, void *object) { poison_range(object, round_up(cache->object_size, KASAN_GRANULE_SIZE), @@ -265,7 +265,7 @@ static u8 assign_tag(struct kmem_cache *cache, const void *object, #endif } -void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, +void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache, const void *object) { struct kasan_alloc_meta *alloc_meta; @@ -284,7 +284,7 @@ void * __must_check kasan_init_slab_obj(struct kmem_cache *cache, return (void *)object; } -static bool __kasan_slab_free(struct kmem_cache *cache, void *object, +static bool ____kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip, bool quarantine) { u8 tag; @@ -330,9 +330,9 @@ static bool __kasan_slab_free(struct kmem_cache *cache, void *object, return IS_ENABLED(CONFIG_KASAN_GENERIC); } -bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) +bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) { - return __kasan_slab_free(cache, object, ip, true); + return ____kasan_slab_free(cache, object, ip, true); } static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) @@ -340,7 +340,7 @@ static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); } -static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, +static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object, size_t size, gfp_t flags, bool keep_tag) { unsigned long redzone_start; @@ -375,20 +375,20 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object, return set_tag(object, tag); } -void * __must_check kasan_slab_alloc(struct kmem_cache *cache, void *object, - gfp_t flags) +void * __must_check __kasan_slab_alloc(struct kmem_cache *cache, + void *object, gfp_t flags) { - return __kasan_kmalloc(cache, object, cache->object_size, flags, false); + return ____kasan_kmalloc(cache, object, cache->object_size, flags, false); } -void * __must_check kasan_kmalloc(struct kmem_cache *cache, const void *object, - size_t size, gfp_t flags) +void * __must_check __kasan_kmalloc(struct kmem_cache *cache, const void *object, + size_t size, gfp_t flags) { - return __kasan_kmalloc(cache, object, size, flags, true); + return ____kasan_kmalloc(cache, object, size, flags, true); } -EXPORT_SYMBOL(kasan_kmalloc); +EXPORT_SYMBOL(__kasan_kmalloc); -void * __must_check kasan_kmalloc_large(const void *ptr, size_t size, +void * __must_check __kasan_kmalloc_large(const void *ptr, size_t size, gfp_t flags) { struct page *page; @@ -413,7 +413,7 @@ void * __must_check kasan_kmalloc_large(const void *ptr, size_t size, return (void *)ptr; } -void * __must_check kasan_krealloc(const void *object, size_t size, gfp_t flags) +void * __must_check __kasan_krealloc(const void *object, size_t size, gfp_t flags) { struct page *page; @@ -423,13 +423,13 @@ void * __must_check kasan_krealloc(const void *object, size_t size, gfp_t flags) page = virt_to_head_page(object); if (unlikely(!PageSlab(page))) - return kasan_kmalloc_large(object, size, flags); + return __kasan_kmalloc_large(object, size, flags); else - return __kasan_kmalloc(page->slab_cache, object, size, + return ____kasan_kmalloc(page->slab_cache, object, size, flags, true); } -void kasan_poison_kfree(void *ptr, unsigned long ip) +void __kasan_poison_kfree(void *ptr, unsigned long ip) { struct page *page; @@ -442,11 +442,11 @@ void kasan_poison_kfree(void *ptr, unsigned long ip) } poison_range(ptr, page_size(page), KASAN_FREE_PAGE); } else { - __kasan_slab_free(page->slab_cache, ptr, ip, false); + ____kasan_slab_free(page->slab_cache, ptr, ip, false); } } -void kasan_kfree_large(void *ptr, unsigned long ip) +void __kasan_kfree_large(void *ptr, unsigned long ip) { if (ptr != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); From patchwork Fri Nov 13 22:20:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904981 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EC20E1391 for ; Fri, 13 Nov 2020 22:20:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A7444206F9 for ; Fri, 13 Nov 2020 22:20:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="nHk4m8fI" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A7444206F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 572526B0095; Fri, 13 Nov 2020 17:20:49 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4FB1F6B0096; Fri, 13 Nov 2020 17:20:49 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 37A296B0098; Fri, 13 Nov 2020 17:20:49 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0128.hostedemail.com [216.40.44.128]) by kanga.kvack.org (Postfix) with ESMTP id EA6D26B0095 for ; Fri, 13 Nov 2020 17:20:48 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 9B89F180AD802 for ; Fri, 13 Nov 2020 22:20:48 +0000 (UTC) X-FDA: 77480815776.19.hot11_310e84327312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id 1D2D11ACEA2 for ; Fri, 13 Nov 2020 22:20:48 +0000 (UTC) X-Spam-Summary: 1,0,0,9526ed0420e7edbb,d41d8cd98f00b204,3vgavxwokcisp2s6tdz2a0v33v0t.r310x29c-11zaprz.36v@flex--andreyknvl.bounces.google.com,,RULES_HIT:41:69:152:355:379:541:800:960:965:966:967:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1544:1593:1594:1605:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2859:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3865:3867:3868:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4119:4321:4385:4390:4395:4425:4605:5007:6119:6261:6653:6742:7901:8603:8957:9025:9592:9969:10004:11026:11473:11658:11914:12043:12291:12296:12297:12438:12555:12679:12683:12895:14096:14097:14181:14394:14659:14721:21080:21365:21444:21451:21627:21939:21990:30054:30070,0,RBL:209.85.128.73:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-66.100.201.100 62.18.0.100;04yrpa8y1hb9jg7ynm87pkzejzrjnycogxz69frscj7wkarjxixsbaxba6jky6b.56afm4jhntjo3nr3dnc8ixw9rswdqbc9449htqguy mnaq5qp9 X-HE-Tag: hot11_310e84327312 X-Filterd-Recvd-Size: 8043 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:47 +0000 (UTC) Received: by mail-wm1-f73.google.com with SMTP id e15so4012587wme.4 for ; Fri, 13 Nov 2020 14:20:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=2KLolIFhijPlCk9+fyMngQmrLJRsQkhazwFij5lYkks=; b=nHk4m8fIajcoKGXz71w/hOkG9Y8KPvxJ7kjUbdB8Qx75uQSuFt3tsDdSskBaqLDy1a oravc38rEEZcIkj06gvuom8/BTOPLhVY7Yh79j3zqDW1HQjH0BOFyOQUXvbccD9KVYcH E9+lSheC8kZevYjyBKHr1mP1dhMe3hw7Vy3ZOZ0unCthrJ1LSBHMy6Gu9mILjaI18/NW +lqQsVM64auomxIjP4GHr2oOFFke0Ts0MapRV41avkeB4U556K8w6Sui1SoEScQQoX2C lrWwtc+y3P6hHtcbfuhnuDSF2VSzK4LoUQ0Bi7YTPLklh+QsWM8vrzrHXkPNSesgs8zf pXjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=2KLolIFhijPlCk9+fyMngQmrLJRsQkhazwFij5lYkks=; b=K/sZCXjP5NP0p+jqVmmyRwuiRL+rmCyaIoASTfd7RcTehaNkCLXbzk6ikWSbV/012n zlUR6lRelhmZNqBDacaFQzd5L5rPHXzRR6qdZecpGf0h485xztVq0CZGiUeD0NUNzgeE Qyu7ij2GpwvwS2TgZ1ylCGkZjoLBt2nFt8liZKmxYe53nR7ChUBUi4GuMy4uuOsfRvcR vK9yYs+2qARcY79bx+LuZM+BDZT/wahF6eNl/8o8nwtqcPMOVR+iB3u7ng4sPBoAKdtp wdeOEz4/NcMG6ZzTOQd5dYZMjmPihTIIfDvod1EWEJWexI5Po4/Pghfz66z+CUvex0Sh 1S1g== X-Gm-Message-State: AOAM530qIdOP0fAWZIEh0mseI6bBqRo25EdP3+y85BlwGeAdcjPDozea 6I28h5HECo4WRpnzjgy5sRGjSW8L3y8osiC3 X-Google-Smtp-Source: ABdhPJz+I+zljHSUOpMnhH3ZRq1yNmGQYurOlqmTjFRBj2bTeKOonLpwQijlOqwdQRHlDe2A37Q45dEdqT2C+Ulk X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a1c:3d6:: with SMTP id 205mr4665784wmd.85.1605306046610; Fri, 13 Nov 2020 14:20:46 -0800 (PST) Date: Fri, 13 Nov 2020 23:20:03 +0100 In-Reply-To: Message-Id: <798e1753fafb37151213a0ad0b1b2f08f66c3877.1605305978.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 13/19] kasan, mm: rename kasan_poison_kfree From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Rename kasan_poison_kfree() to kasan_slab_free_mempool() as it better reflects what this annotation does. Also add a comment that explains the PageSlab() check. No functional changes. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/I5026f87364e556b506ef1baee725144bb04b8810 Reviewed-by: Marco Elver --- include/linux/kasan.h | 16 ++++++++-------- mm/kasan/common.c | 40 +++++++++++++++++++++++----------------- mm/mempool.c | 2 +- 3 files changed, 32 insertions(+), 26 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 6bd95243a583..16cf53eac29b 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -175,6 +175,13 @@ static __always_inline bool kasan_slab_free(struct kmem_cache *s, void *object, return false; } +void __kasan_slab_free_mempool(void *ptr, unsigned long ip); +static __always_inline void kasan_slab_free_mempool(void *ptr, unsigned long ip) +{ + if (kasan_enabled()) + __kasan_slab_free_mempool(ptr, ip); +} + void * __must_check __kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags); static __always_inline void * __must_check kasan_slab_alloc( @@ -215,13 +222,6 @@ static __always_inline void * __must_check kasan_krealloc(const void *object, return (void *)object; } -void __kasan_poison_kfree(void *ptr, unsigned long ip); -static __always_inline void kasan_poison_kfree(void *ptr, unsigned long ip) -{ - if (kasan_enabled()) - __kasan_poison_kfree(ptr, ip); -} - void __kasan_kfree_large(void *ptr, unsigned long ip); static __always_inline void kasan_kfree_large(void *ptr, unsigned long ip) { @@ -260,6 +260,7 @@ static inline bool kasan_slab_free(struct kmem_cache *s, void *object, { return false; } +static inline void kasan_slab_free_mempool(void *ptr, unsigned long ip) {} static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags) { @@ -279,7 +280,6 @@ static inline void *kasan_krealloc(const void *object, size_t new_size, { return (void *)object; } -static inline void kasan_poison_kfree(void *ptr, unsigned long ip) {} static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} #endif /* CONFIG_KASAN */ diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 17918bd20ed9..1205faac90bd 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -335,6 +335,29 @@ bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) return ____kasan_slab_free(cache, object, ip, true); } +void __kasan_slab_free_mempool(void *ptr, unsigned long ip) +{ + struct page *page; + + page = virt_to_head_page(ptr); + + /* + * Even though this function is only called for kmem_cache_alloc and + * kmalloc backed mempool allocations, those allocations can still be + * !PageSlab() when the size provided to kmalloc is larger than + * KMALLOC_MAX_SIZE, and kmalloc falls back onto page_alloc. + */ + if (unlikely(!PageSlab(page))) { + if (ptr != page_address(page)) { + kasan_report_invalid_free(ptr, ip); + return; + } + poison_range(ptr, page_size(page), KASAN_FREE_PAGE); + } else { + ____kasan_slab_free(page->slab_cache, ptr, ip, false); + } +} + static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) { kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); @@ -429,23 +452,6 @@ void * __must_check __kasan_krealloc(const void *object, size_t size, gfp_t flag flags, true); } -void __kasan_poison_kfree(void *ptr, unsigned long ip) -{ - struct page *page; - - page = virt_to_head_page(ptr); - - if (unlikely(!PageSlab(page))) { - if (ptr != page_address(page)) { - kasan_report_invalid_free(ptr, ip); - return; - } - poison_range(ptr, page_size(page), KASAN_FREE_PAGE); - } else { - ____kasan_slab_free(page->slab_cache, ptr, ip, false); - } -} - void __kasan_kfree_large(void *ptr, unsigned long ip) { if (ptr != page_address(virt_to_head_page(ptr))) diff --git a/mm/mempool.c b/mm/mempool.c index 583a9865b181..624ed51b060f 100644 --- a/mm/mempool.c +++ b/mm/mempool.c @@ -104,7 +104,7 @@ static inline void poison_element(mempool_t *pool, void *element) static __always_inline void kasan_poison_element(mempool_t *pool, void *element) { if (pool->alloc == mempool_alloc_slab || pool->alloc == mempool_kmalloc) - kasan_poison_kfree(element, _RET_IP_); + kasan_slab_free_mempool(element, _RET_IP_); else if (pool->alloc == mempool_alloc_pages) kasan_free_pages(element, (unsigned long)pool->pool_data); } From patchwork Fri Nov 13 22:20:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904983 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 057F6697 for ; Fri, 13 Nov 2020 22:20:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BEC9C206F9 for ; Fri, 13 Nov 2020 22:20:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="feWIg6qU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BEC9C206F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0BF986B0098; Fri, 13 Nov 2020 17:20:51 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E765E6B009A; Fri, 13 Nov 2020 17:20:50 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C2A286B009B; Fri, 13 Nov 2020 17:20:50 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0193.hostedemail.com [216.40.44.193]) by kanga.kvack.org (Postfix) with ESMTP id 84FC56B0098 for ; Fri, 13 Nov 2020 17:20:50 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 2E8B0181AEF07 for ; Fri, 13 Nov 2020 22:20:50 +0000 (UTC) X-FDA: 77480815860.19.smell99_2010bef27312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id 0266F1AD1B2 for ; Fri, 13 Nov 2020 22:20:49 +0000 (UTC) X-Spam-Summary: 1,0,0,bae82d3b64883301,d41d8cd98f00b204,3waavxwokci0r4u8vf14c2x55x2v.t532z4be-331crt1.58x@flex--andreyknvl.bounces.google.com,,RULES_HIT:41:152:355:379:541:800:960:966:967:968:973:988:989:1042:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1542:1593:1594:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2859:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3352:3865:3866:3867:3868:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4425:5007:6261:6653:6742:7901:8603:9025:9969:10004:10400:10429:10430:10431:11026:11232:11473:11658:11914:12043:12296:12297:12438:12555:12679:12895:12986:14096:14097:14181:14394:14659:14721:19903:19997:21080:21365:21444:21451:21627:21772:21939:21990:30003:30012:30054,0,RBL:209.85.219.74:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-66.100.201.100 62.18.0.100;04yrc3n65qsrpr8kjpknkcs4krh5zypu5youp1fep1zgq93e7dn74xjicxxd5pe.yn9ueqehy8eydpr6istd97me16pzyx1 akisinub X-HE-Tag: smell99_2010bef27312 X-Filterd-Recvd-Size: 5444 Received: from mail-qv1-f74.google.com (mail-qv1-f74.google.com [209.85.219.74]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:49 +0000 (UTC) Received: by mail-qv1-f74.google.com with SMTP id dp12so1224873qvb.2 for ; Fri, 13 Nov 2020 14:20:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=WUG0j444ZBT4A3BTzNH/LM6507YxULafwrr/Sriwg0c=; b=feWIg6qUi7heyfOTSzsy2DpmrO1Qia8xbqIt8JmH5daMnreAv2krFvZcHiqDVKaKv2 1bTiAzutYZP/XiwiZ9rIvAKvtPkQ7Mc6tpkK9c/r6TsmzWPVn8WIrC7+uJ4GTVzK1zmW TARqSwftqGvkqjdh+M4+ZJqYCDPFDqsDB9jR8vkc90/AV5vWJuhIxDotwDlEkh3jH5qe ZmaI6Rzzwp10/w7Ry/ZUbxCMitP2uLv4cyOpLyoScAlq9lSOFBT/gO1S6XhYs7/32ijH 6yjUGd/6BuFXadoaZ8NaAhKvRf9nKcM6diCCz9JjlSN3ZDQG/jWv6oy7qyJYGVlmyAkl JszA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=WUG0j444ZBT4A3BTzNH/LM6507YxULafwrr/Sriwg0c=; b=HrsF3OZBP79yGqfV/69n5ql0+d54thEHH3UHx8HAmQhZx87fccTIo9wN7Hd5LfVgne Q0MfAezi/62jGEjkJEJnztChUNhh2WG13L26H3s4y7NtQAFWdYXyxhkmr4x/qytmGLWy EJmLJq+DT+3+2ov2TRhx3XNWdr01qWxpLY2QQVJ3G/wybs0ThzWI29Jg0L2vWorLx2pz 3PQBqYX82X0mZ0PDI9jhweq82SaN3oIbJOM564/dnG/Z4B6ZDI2r/JFlNH9G01LaLtCO GiU93G6n+tHZZr6PbXeLDlY82TKtKbvzhc+URHUKcC0cS/OCnVvrCzSQ48AhKt1P9dwe e0Hw== X-Gm-Message-State: AOAM5326c1OOdz+nHib7+tOij4KRaCAm+LwlL/jbSE7P8lra7QEO4Bvu aApJWa8lPORx4yx2GdDcKHfjPBtB6sC3nmtK X-Google-Smtp-Source: ABdhPJyDbEDKFY3EfcHLc7j3mQfHgxDyZ09bJBO/2bzJnR89c+WsxkzTNbHHqzArwf4wHZ9mwaSRm4uyurfObZ/0 X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a05:6214:d43:: with SMTP id 3mr4767045qvr.38.1605306048874; Fri, 13 Nov 2020 14:20:48 -0800 (PST) Date: Fri, 13 Nov 2020 23:20:04 +0100 In-Reply-To: Message-Id: <38ed98141f58eb53eb23100caac212b1c1f3bc9f.1605305978.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 14/19] kasan: don't round_up too much From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: For hardware tag-based mode kasan_poison_memory() already rounds up the size. Do the same for software modes and remove round_up() from the common code. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Reviewed-by: Marco Elver Link: https://linux-review.googlesource.com/id/Ib397128fac6eba874008662b4964d65352db4aa4 --- mm/kasan/common.c | 8 ++------ mm/kasan/shadow.c | 1 + 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 1205faac90bd..1a88e4005181 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -214,9 +214,7 @@ void __kasan_unpoison_object_data(struct kmem_cache *cache, void *object) void __kasan_poison_object_data(struct kmem_cache *cache, void *object) { - poison_range(object, - round_up(cache->object_size, KASAN_GRANULE_SIZE), - KASAN_KMALLOC_REDZONE); + poison_range(object, cache->object_size, KASAN_KMALLOC_REDZONE); } /* @@ -289,7 +287,6 @@ static bool ____kasan_slab_free(struct kmem_cache *cache, void *object, { u8 tag; void *tagged_object; - unsigned long rounded_up_size; tag = get_tag(object); tagged_object = object; @@ -313,8 +310,7 @@ static bool ____kasan_slab_free(struct kmem_cache *cache, void *object, return true; } - rounded_up_size = round_up(cache->object_size, KASAN_GRANULE_SIZE); - poison_range(object, rounded_up_size, KASAN_KMALLOC_FREE); + poison_range(object, cache->object_size, KASAN_KMALLOC_FREE); if (!kasan_stack_collection_enabled()) return false; diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index 37153bd1c126..e9efe88f7679 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -83,6 +83,7 @@ void poison_range(const void *address, size_t size, u8 value) * addresses to this function. */ address = kasan_reset_tag(address); + size = round_up(size, KASAN_GRANULE_SIZE); /* Skip KFENCE memory if called explicitly outside of sl*b. */ if (is_kfence_address(address)) From patchwork Fri Nov 13 22:20:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904985 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B00FB697 for ; Fri, 13 Nov 2020 22:20:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 75442206F9 for ; Fri, 13 Nov 2020 22:20:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="gVybcLoF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 75442206F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 304756B009C; Fri, 13 Nov 2020 17:20:54 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2B08B6B009D; Fri, 13 Nov 2020 17:20:54 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1A34E6B009E; Fri, 13 Nov 2020 17:20:54 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0009.hostedemail.com [216.40.44.9]) by kanga.kvack.org (Postfix) with ESMTP id D96CD6B009C for ; Fri, 13 Nov 2020 17:20:53 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 8024933CD for ; Fri, 13 Nov 2020 22:20:53 +0000 (UTC) X-FDA: 77480815986.28.coal31_4507b7d27312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin28.hostedemail.com (Postfix) with ESMTP id 586116C1A for ; Fri, 13 Nov 2020 22:20:53 +0000 (UTC) X-Spam-Summary: 1,0,0,33f2f11fa9ab6655,d41d8cd98f00b204,3wwavxwokcjau7xbyi47f508805y.w86527eh-664fuw4.8b0@flex--andreyknvl.bounces.google.com,,RULES_HIT:41:152:355:379:541:800:960:967:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1542:1593:1594:1711:1730:1747:1777:1792:1981:2194:2199:2393:2525:2559:2563:2682:2685:2859:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3353:3865:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4425:5007:6261:6653:6742:7901:8603:8660:9025:9592:9969:10004:10400:11026:11473:11658:11914:12043:12296:12297:12438:12555:12679:12895:12986:13148:13230:14096:14097:14181:14394:14659:14721:21080:21212:21365:21444:21451:21627:21939:21990:30054:30070,0,RBL:209.85.128.73:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100;04y8s67d4qxsxxzgwg4h16mdrjqm5ycqbpzx93jkr4p8zqpgps383gh7hu8dxsz.qi6jaygancmcqehpywztcw9zsmqeha9kydfbmhxn9n5u6ogoo6usog155y7 c91c.4-l X-HE-Tag: coal31_4507b7d27312 X-Filterd-Recvd-Size: 5679 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) by imf24.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:52 +0000 (UTC) Received: by mail-wm1-f73.google.com with SMTP id j62so3140578wma.4 for ; Fri, 13 Nov 2020 14:20:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=FjDCd3uZs5tZgyebUG073A1B5l2RXwE8lidjbcRsCvs=; b=gVybcLoFQUNnqi8BvDXIuLxWzyBvmJ9r8YgiuYhT5Vu2C/zCCrI7te3RDmVOXAIc8d jWdwWo5LeuXbMT+aa3nFbTr7pi1D7mqVCTn1+Kr06wbwlyMlEdafbkwCQf+Np9kgAEwq G18BCLYbfNs//MAyXfJCvA4BZPSNOgVNjaxII/tGOSP3J7KzzOpraccZJslROAokEP/w gbpLxCXcb8AwsSYVXkVGO1YkPuVk07OlX1Ri3z5eDOhzP2nxBArhzqAoDZvUib0T7cA6 sxJ2VEfODnX22F93ftmfkG7ItpS54IPEpfsGlbzriQ4c07kVaMvIWMlMtIo0OfVbEEEc TDHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=FjDCd3uZs5tZgyebUG073A1B5l2RXwE8lidjbcRsCvs=; b=cq7t8FmYD5jcrcjyuKSNqUPwxKw7XnHjkGW7i82MMs9nd1ysD98+l6gyGWEuTiEWBa n/4UWVawyn44s0ZTNMQwErDhpg/hmLHhxlpthNTFP2IrzSIlx9DHN2Cv0cOozuNbmUd6 pI21dAq1LRuQFmBhaAEbnr9C2F94Mc/WWbhtwBLfAOfiE6DeHWA8kRNJfsF5d7hvVFBL F7l4bVN7Ofp3loBOERXS7V+8rgnij3lJE3yqj/AM+JyeetIKZZRC4CbtQs/sfNwclMtW po69cbd3Cw62oPkITciSNpGCFa4G0binmVJxUgnhqwHsBNdisdXDNt+kie7Z/gJ9MVmp wwlQ== X-Gm-Message-State: AOAM532uftf4PcVIn+ceiOUvCGR/PZAdSLdWNmBAEjlI4BPqwB3DFcvr B+vrIvBJY6lwu47cqYUPAb9Z8sbelehbzlJZ X-Google-Smtp-Source: ABdhPJyhSXjfAMNZDyab6PZ2QpxILyu3jBNhWgi25Hcu7VUSLE8KCaS2ijL7XZdOMCTm1EGjb1TAJwfcjEDFzQlL X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:adf:e74d:: with SMTP id c13mr6274338wrn.277.1605306051483; Fri, 13 Nov 2020 14:20:51 -0800 (PST) Date: Fri, 13 Nov 2020 23:20:05 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 15/19] kasan: simplify assign_tag and set_tag calls From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: set_tag() already ignores the tag for the generic mode, so just call it as is. Add a check for the generic mode to assign_tag(), and simplify its call in ____kasan_kmalloc(). Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Reviewed-by: Marco Elver Link: https://linux-review.googlesource.com/id/I18905ca78fb4a3d60e1a34a4ca00247272480438 --- mm/kasan/common.c | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 1a88e4005181..821678a58ac6 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -234,6 +234,9 @@ void __kasan_poison_object_data(struct kmem_cache *cache, void *object) static u8 assign_tag(struct kmem_cache *cache, const void *object, bool init, bool keep_tag) { + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) + return 0xff; + /* * 1. When an object is kmalloc()'ed, two hooks are called: * kasan_slab_alloc() and kasan_kmalloc(). We assign the @@ -276,8 +279,8 @@ void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache, __memset(alloc_meta, 0, sizeof(*alloc_meta)); } - if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) - object = set_tag(object, assign_tag(cache, object, true, false)); + /* Tag is ignored in set_tag() without CONFIG_KASAN_SW/HW_TAGS */ + object = set_tag(object, assign_tag(cache, object, true, false)); return (void *)object; } @@ -364,7 +367,7 @@ static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object, { unsigned long redzone_start; unsigned long redzone_end; - u8 tag = 0xff; + u8 tag; if (gfpflags_allow_blocking(flags)) quarantine_reduce(); @@ -379,9 +382,7 @@ static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object, KASAN_GRANULE_SIZE); redzone_end = round_up((unsigned long)object + cache->object_size, KASAN_GRANULE_SIZE); - - if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) || IS_ENABLED(CONFIG_KASAN_HW_TAGS)) - tag = assign_tag(cache, object, false, keep_tag); + tag = assign_tag(cache, object, false, keep_tag); /* Tag is ignored in set_tag without CONFIG_KASAN_SW/HW_TAGS */ unpoison_range(set_tag(object, tag), size); From patchwork Fri Nov 13 22:20:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904987 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0AD7F1391 for ; Fri, 13 Nov 2020 22:20:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A638A206F9 for ; Fri, 13 Nov 2020 22:20:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="JaBlZxu7" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A638A206F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A7D676B009E; Fri, 13 Nov 2020 17:20:55 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9DCB66B00A0; Fri, 13 Nov 2020 17:20:55 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 85A846B00A1; Fri, 13 Nov 2020 17:20:55 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0189.hostedemail.com [216.40.44.189]) by kanga.kvack.org (Postfix) with ESMTP id 4A2956B009E for ; Fri, 13 Nov 2020 17:20:55 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 019CF180AD807 for ; Fri, 13 Nov 2020 22:20:55 +0000 (UTC) X-FDA: 77480816070.09.cave95_1704d6f27312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin09.hostedemail.com (Postfix) with ESMTP id D95CF180AD802 for ; Fri, 13 Nov 2020 22:20:54 +0000 (UTC) X-Spam-Summary: 1,0,0,61a4cc3d70e92e95,d41d8cd98f00b204,3xqavxwokcjiw9zd0k69h72aa270.ya8749gj-886hwy6.ad2@flex--andreyknvl.bounces.google.com,,RULES_HIT:41:152:355:379:541:800:960:965:966:967:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1534:1540:1593:1594:1711:1714:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2693:2859:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3351:3865:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4390:4395:4425:5007:6119:6261:6653:6742:7901:7903:9025:9969:10004:10400:11026:11658:11914:12043:12296:12297:12438:12555:12679:12895:13069:13311:13357:14096:14097:14181:14394:14659:14721:21080:21365:21444:21451:21627:21939:30012:30054,0,RBL:209.85.222.201:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-66.100.201.100 62.18.0.100;04y8dun47wcbbfycfefzm44mcf6ckypz41fqqjrjix7owbxckz9ijw1hn7s8ppc.rfjzbgtqu9mrpdumycsqjq4mukbjb9frzch4nnpr4w5sdpc8t8wne4wssz6d9xk.y-lb l8.mails X-HE-Tag: cave95_1704d6f27312 X-Filterd-Recvd-Size: 4284 Received: from mail-qk1-f201.google.com (mail-qk1-f201.google.com [209.85.222.201]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:54 +0000 (UTC) Received: by mail-qk1-f201.google.com with SMTP id b191so7563534qkc.10 for ; Fri, 13 Nov 2020 14:20:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=Yi0H/DT0iolgmo7NG/qMiO64up+Qc265jwkA+xu3tVM=; b=JaBlZxu7lA5jybjquVXOefu6xeeFogoBoXU1MliVu8hO4i5YwE0yAHxz10jpwLhOUG QZIoujH6ck0jbRoIZBq4jaoY1p7aSmkbWN63j+ihIJcWjrgLqYFrcHgb8SaxZ9IpWSsG VOoGkItovexEkREiFcSvUf6iTwO1D2zvHAHwp3Tn21E/jemeZR9ein17hD8X3olghKqF hV9KNqoNkDz0dNidKD/eBsaEhqtf0c7RYIgON5+4XvohOM1GeltX35tQ5aOnIAzpDebT Ly75REWb81h6l041aTpotOhJ5bbwNpW7TLTJdhFtMH+9I3OoDEcW4pgjVVfxTFyFlDin Raxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Yi0H/DT0iolgmo7NG/qMiO64up+Qc265jwkA+xu3tVM=; b=tKyH8SoOqgfQxQJCx1JM1qwQJj6Sax1wPTrGgw0xC5ptkVAq/uPdOxsI7VGmPQPco1 r2Z31blhoBBsthFRDLaxIl/GZZTHt777Efmdo2hQhlERGlexqXhzCigbRK19/j8gcmfW sL1pmwSdAJIovnNwbEUDrmfDE9KDR7LgtNU91mK4M4dhzfJPb/vDQerboSf6x/DWFECN MCZqQwwNqfhAJVNdEiChYZ5hFGYT1Pw8HasFauYsqCGg4/ShLiRQeU9JZ2Tg+AFOWHD8 DwOFhvjGlDSMtF+NxXO1/p9zXiktnmfz2dVoYT5/lVaizcqRQvU4ZiwELQHtnK9JFL+H SpEw== X-Gm-Message-State: AOAM533S1e1YgWjW6wFqabSSmPrpGRI7giRgtExcrLd0ZbHnIN37iNZn C1/uNaeCIrQFhdWhb6a9LfS1VIr3ayZxGPuu X-Google-Smtp-Source: ABdhPJy0Q043vk8cHOStNvQydNlkW0VoeWxy1kyCkq36QgHv0e9VO0VOgQiCfIHAjZb++EoHTDq0EEdvZhMlArtL X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a0c:fdcb:: with SMTP id g11mr4476840qvs.58.1605306053723; Fri, 13 Nov 2020 14:20:53 -0800 (PST) Date: Fri, 13 Nov 2020 23:20:06 +0100 In-Reply-To: Message-Id: Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 16/19] kasan: clarify comment in __kasan_kfree_large From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently it says that the memory gets poisoned by page_alloc code. Clarify this by mentioning the specific callback that poisons the memory. Signed-off-by: Andrey Konovalov Reviewed-by: Dmitry Vyukov Reviewed-by: Marco Elver Link: https://linux-review.googlesource.com/id/I1334dffb69b87d7986fab88a1a039cc3ea764725 --- mm/kasan/common.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 821678a58ac6..42ba64fce8a3 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -453,5 +453,5 @@ void __kasan_kfree_large(void *ptr, unsigned long ip) { if (ptr != page_address(virt_to_head_page(ptr))) kasan_report_invalid_free(ptr, ip); - /* The object will be poisoned by page_alloc. */ + /* The object will be poisoned by kasan_free_pages(). */ } From patchwork Fri Nov 13 22:20:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904989 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 83DF4697 for ; Fri, 13 Nov 2020 22:21:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 26972206F9 for ; Fri, 13 Nov 2020 22:21:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="Bjx95fLY" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 26972206F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7C0376B00A2; Fri, 13 Nov 2020 17:20:58 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 721526B00A3; Fri, 13 Nov 2020 17:20:58 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5C6D16B00A4; Fri, 13 Nov 2020 17:20:58 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0175.hostedemail.com [216.40.44.175]) by kanga.kvack.org (Postfix) with ESMTP id 194AB6B00A2 for ; Fri, 13 Nov 2020 17:20:58 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id B715E180AD802 for ; Fri, 13 Nov 2020 22:20:57 +0000 (UTC) X-FDA: 77480816154.11.root77_300593627312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin11.hostedemail.com (Postfix) with ESMTP id 9F099180F8B81 for ; Fri, 13 Nov 2020 22:20:57 +0000 (UTC) X-Spam-Summary: 1,0,0,6bfa0170aa668cf1,d41d8cd98f00b204,3yaavxwokcjuzc2g3n9cka5dd5a3.1dba7cjm-bb9kz19.dg5@flex--andreyknvl.bounces.google.com,,RULES_HIT:4:41:69:152:355:379:541:800:960:966:967:968:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1593:1594:1605:1730:1747:1777:1792:1981:2194:2196:2198:2199:2200:2201:2393:2525:2559:2563:2682:2685:2731:2859:2898:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4425:4605:5007:6261:6653:6742:7875:7901:8603:8660:9025:9592:9969:10004:11026:11473:11658:11914:12043:12291:12296:12297:12438:12555:12679:12683:12895:12986:13138:13148:13161:13229:13230:13231:14096:14097:14394:14659:21063:21080:21324:21365:21444:21451:21627:21740:21939:21966:21990:30012:30054:30070:30075,0,RBL:209.85.221.74:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100;04ygoo88u9qcsjtob f9pgx46a X-HE-Tag: root77_300593627312 X-Filterd-Recvd-Size: 19176 Received: from mail-wr1-f74.google.com (mail-wr1-f74.google.com [209.85.221.74]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:57 +0000 (UTC) Received: by mail-wr1-f74.google.com with SMTP id w5so2415420wrm.22 for ; Fri, 13 Nov 2020 14:20:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=UzKiiLikh/yr7SuKd1LAVi3VD7zxpPGUKeQZkocVPmU=; b=Bjx95fLYPShRtv8g/ufbvIu9y//mZxxB+eubkSGsPC08JdIDeEUCheBPu3hL+KPpTG pv1uVOufhElpdQp8EIbun+S6MLB/YYShPW4iK0uFB0XVnctt21hRzuzQGc/wgAYjYVK0 yy9JzsIzNljJeqjXjzfCXOygYIftGxSyreDSypZY5foV+7D5bJaeEZJnoVxp9dbG+xPV CcDkEN+13pZ+zloXnMldyy7kPHAPdLuLS2EqL14zMrIudCMsQ/QamfC0w6nX4cFyovwu wMmm6riErkXmTELkDVoIt0kxKChZ8oTi9pJHRVAIGbWPnpzGMbm4u+Q8ZKrRbFOcVzBB Mvhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=UzKiiLikh/yr7SuKd1LAVi3VD7zxpPGUKeQZkocVPmU=; b=l/xa321JQBQWFhwQ95iNCZJmyduFJEarTCgiGpM0jxh8OCtbCh0Cw5bRh2Bd/enwg+ qFIZcRHXcvi7hc8wh1DPg7PBqu5aKJKcoAew+WjAWR2V2+H/lrW29/RIc80WcdXg62fx HrIe6lUUYBzE4FtrGIHNRWY86AbMrg4bNqCzrLZ0RTboxqSL8fKsxPbNKhwPeGGIQ6zU 2+GdR42FSKMIXy6yBdKmcWn1JiOl8JYfTjbZtnR7DKwfk00l5KseG1Gy5FZRn38j2MAX oZw4ohiYyfAg/04rdqn0x/nO/bI8r24qUijMLy0PW6ve+JWqSwLpHBOzcyGc6CBAeG6D yxsQ== X-Gm-Message-State: AOAM532c8QFgDgyyZREdY1rPN1e3WIJIKwg0mG9+SHs72+wC6pM74zOd DmMsT8HGBcrPqqRo6CxPpmKsqihkxbXgESK0 X-Google-Smtp-Source: ABdhPJx46nAxIjBsn9TKraOEyyZoXB7EdQv6hzYJKw6QUWysK6zAJi+rlj4Q+xzSfZn7CD0jY3wUwIqBoQIbEwXS X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:adf:eacb:: with SMTP id o11mr6101808wrn.208.1605306056039; Fri, 13 Nov 2020 14:20:56 -0800 (PST) Date: Fri, 13 Nov 2020 23:20:07 +0100 In-Reply-To: Message-Id: <52518837b34d607abbf30855b3ac4cb1a9486946.1605305978.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 17/19] kasan: clean up metadata allocation and usage From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov , Vincenzo Frascino X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: KASAN marks caches that are sanitized with the SLAB_KASAN cache flag. Currently if the metadata that is appended after the object (stores e.g. stack trace ids) doesn't fit into KMALLOC_MAX_SIZE (can only happen with SLAB, see the comment in the patch), KASAN turns off sanitization completely. With this change sanitization of the object data is always enabled. However the metadata is only stored when it fits. Instead of checking for SLAB_KASAN flag accross the code to find out whether the metadata is there, use cache->kasan_info.alloc/free_meta_offset. As 0 can be a valid value for free_meta_offset, introduce KASAN_NO_FREE_META as an indicator that the free metadata is missing. Along the way rework __kasan_cache_create() and add claryfying comments. Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/Icd947e2bea054cb5cfbdc6cf6652227d97032dcb Reviewed-by: Marco Elver --- mm/kasan/common.c | 112 +++++++++++++++++++++++++------------- mm/kasan/generic.c | 15 ++--- mm/kasan/hw_tags.c | 6 +- mm/kasan/kasan.h | 13 ++++- mm/kasan/quarantine.c | 8 +++ mm/kasan/report.c | 43 ++++++++------- mm/kasan/report_sw_tags.c | 9 ++- mm/kasan/sw_tags.c | 4 ++ 8 files changed, 139 insertions(+), 71 deletions(-) diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 42ba64fce8a3..cf874243efab 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -115,9 +115,6 @@ void __kasan_free_pages(struct page *page, unsigned int order) */ static inline unsigned int optimal_redzone(unsigned int object_size) { - if (!IS_ENABLED(CONFIG_KASAN_GENERIC)) - return 0; - return object_size <= 64 - 16 ? 16 : object_size <= 128 - 32 ? 32 : @@ -131,47 +128,77 @@ static inline unsigned int optimal_redzone(unsigned int object_size) void __kasan_cache_create(struct kmem_cache *cache, unsigned int *size, slab_flags_t *flags) { - unsigned int orig_size = *size; - unsigned int redzone_size; - int redzone_adjust; + unsigned int ok_size; + unsigned int optimal_size; + + /* + * SLAB_KASAN is used to mark caches as ones that are sanitized by + * KASAN. Currently this is used in two places: + * 1. In slab_ksize() when calculating the size of the accessible + * memory within the object. + * 2. In slab_common.c to prevent merging of sanitized caches. + */ + *flags |= SLAB_KASAN; - if (!kasan_stack_collection_enabled()) { - *flags |= SLAB_KASAN; + if (!kasan_stack_collection_enabled()) return; - } - /* Add alloc meta. */ + ok_size = *size; + + /* Add alloc meta into redzone. */ cache->kasan_info.alloc_meta_offset = *size; *size += sizeof(struct kasan_alloc_meta); - /* Add free meta. */ - if (IS_ENABLED(CONFIG_KASAN_GENERIC) && - (cache->flags & SLAB_TYPESAFE_BY_RCU || cache->ctor || - cache->object_size < sizeof(struct kasan_free_meta))) { - cache->kasan_info.free_meta_offset = *size; - *size += sizeof(struct kasan_free_meta); + /* + * If alloc meta doesn't fit, don't add it. + * This can only happen with SLAB, as it has KMALLOC_MAX_SIZE equal + * to KMALLOC_MAX_CACHE_SIZE and doesn't fall back to page_alloc for + * larger sizes. + */ + if (*size > KMALLOC_MAX_SIZE) { + cache->kasan_info.alloc_meta_offset = 0; + *size = ok_size; + /* Continue, since free meta might still fit. */ } - redzone_size = optimal_redzone(cache->object_size); - redzone_adjust = redzone_size - (*size - cache->object_size); - if (redzone_adjust > 0) - *size += redzone_adjust; - - *size = min_t(unsigned int, KMALLOC_MAX_SIZE, - max(*size, cache->object_size + redzone_size)); + /* Only the generic mode uses free meta or flexible redzones. */ + if (!IS_ENABLED(CONFIG_KASAN_GENERIC)) { + cache->kasan_info.free_meta_offset = KASAN_NO_FREE_META; + return; + } /* - * If the metadata doesn't fit, don't enable KASAN at all. + * Add free meta into redzone when it's not possible to store + * it in the object. This is the case when: + * 1. Object is SLAB_TYPESAFE_BY_RCU, which means that it can + * be touched after it was freed, or + * 2. Object has a constructor, which means it's expected to + * retain its content until the next allocation, or + * 3. Object is too small. + * Otherwise cache->kasan_info.free_meta_offset = 0 is implied. */ - if (*size <= cache->kasan_info.alloc_meta_offset || - *size <= cache->kasan_info.free_meta_offset) { - cache->kasan_info.alloc_meta_offset = 0; - cache->kasan_info.free_meta_offset = 0; - *size = orig_size; - return; + if ((cache->flags & SLAB_TYPESAFE_BY_RCU) || cache->ctor || + cache->object_size < sizeof(struct kasan_free_meta)) { + ok_size = *size; + + cache->kasan_info.free_meta_offset = *size; + *size += sizeof(struct kasan_free_meta); + + /* If free meta doesn't fit, don't add it. */ + if (*size > KMALLOC_MAX_SIZE) { + cache->kasan_info.free_meta_offset = KASAN_NO_FREE_META; + *size = ok_size; + } } - *flags |= SLAB_KASAN; + /* Calculate size with optimal redzone. */ + optimal_size = cache->object_size + optimal_redzone(cache->object_size); + /* Limit it with KMALLOC_MAX_SIZE (relevant for SLAB only). */ + if (optimal_size > KMALLOC_MAX_SIZE) + optimal_size = KMALLOC_MAX_SIZE; + /* Use optimal size if the size with added metas is not large enough. */ + if (*size < optimal_size) + *size = optimal_size; } size_t __kasan_metadata_size(struct kmem_cache *cache) @@ -187,15 +214,21 @@ size_t __kasan_metadata_size(struct kmem_cache *cache) struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, const void *object) { + if (!cache->kasan_info.alloc_meta_offset) + return NULL; return kasan_reset_tag(object) + cache->kasan_info.alloc_meta_offset; } +#ifdef CONFIG_KASAN_GENERIC struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, const void *object) { BUILD_BUG_ON(sizeof(struct kasan_free_meta) > 32); + if (cache->kasan_info.free_meta_offset == KASAN_NO_FREE_META) + return NULL; return kasan_reset_tag(object) + cache->kasan_info.free_meta_offset; } +#endif void __kasan_poison_slab(struct page *page) { @@ -272,11 +305,9 @@ void * __must_check __kasan_init_slab_obj(struct kmem_cache *cache, struct kasan_alloc_meta *alloc_meta; if (kasan_stack_collection_enabled()) { - if (!(cache->flags & SLAB_KASAN)) - return (void *)object; - alloc_meta = kasan_get_alloc_meta(cache, object); - __memset(alloc_meta, 0, sizeof(*alloc_meta)); + if (alloc_meta) + __memset(alloc_meta, 0, sizeof(*alloc_meta)); } /* Tag is ignored in set_tag() without CONFIG_KASAN_SW/HW_TAGS */ @@ -318,8 +349,7 @@ static bool ____kasan_slab_free(struct kmem_cache *cache, void *object, if (!kasan_stack_collection_enabled()) return false; - if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine) || - unlikely(!(cache->flags & SLAB_KASAN))) + if ((IS_ENABLED(CONFIG_KASAN_GENERIC) && !quarantine)) return false; kasan_set_free_info(cache, object, tag); @@ -359,7 +389,11 @@ void __kasan_slab_free_mempool(void *ptr, unsigned long ip) static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) { - kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags); + struct kasan_alloc_meta *alloc_meta; + + alloc_meta = kasan_get_alloc_meta(cache, object); + if (alloc_meta) + kasan_set_track(&alloc_meta->alloc_track, flags); } static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object, @@ -389,7 +423,7 @@ static void *____kasan_kmalloc(struct kmem_cache *cache, const void *object, poison_range((void *)redzone_start, redzone_end - redzone_start, KASAN_KMALLOC_REDZONE); - if (kasan_stack_collection_enabled() && (cache->flags & SLAB_KASAN)) + if (kasan_stack_collection_enabled()) set_alloc_info(cache, (void *)object, flags); return set_tag(object, tag); diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index 9c6b77f8c4a4..157df6c762a4 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -338,10 +338,10 @@ void kasan_record_aux_stack(void *addr) cache = page->slab_cache; object = nearest_obj(cache, page, addr); alloc_meta = kasan_get_alloc_meta(cache, object); + if (!alloc_meta) + return; - /* - * record the last two call_rcu() call stacks. - */ + /* Record the last two call_rcu() call stacks. */ alloc_meta->aux_stack[1] = alloc_meta->aux_stack[0]; alloc_meta->aux_stack[0] = kasan_save_stack(GFP_NOWAIT); } @@ -352,11 +352,11 @@ void kasan_set_free_info(struct kmem_cache *cache, struct kasan_free_meta *free_meta; free_meta = kasan_get_free_meta(cache, object); - kasan_set_track(&free_meta->free_track, GFP_NOWAIT); + if (!free_meta) + return; - /* - * the object was freed and has free track set - */ + kasan_set_track(&free_meta->free_track, GFP_NOWAIT); + /* The object was freed and has free track set. */ *(u8 *)kasan_mem_to_shadow(object) = KASAN_KMALLOC_FREETRACK; } @@ -365,5 +365,6 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, { if (*(u8 *)kasan_mem_to_shadow(object) != KASAN_KMALLOC_FREETRACK) return NULL; + /* Free meta must be present with KASAN_KMALLOC_FREETRACK. */ return &kasan_get_free_meta(cache, object)->free_track; } diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 30ce88935e9d..c91f2c06ecb5 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -187,7 +187,8 @@ void kasan_set_free_info(struct kmem_cache *cache, struct kasan_alloc_meta *alloc_meta; alloc_meta = kasan_get_alloc_meta(cache, object); - kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT); + if (alloc_meta) + kasan_set_track(&alloc_meta->free_track[0], GFP_NOWAIT); } struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, @@ -196,5 +197,8 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, struct kasan_alloc_meta *alloc_meta; alloc_meta = kasan_get_alloc_meta(cache, object); + if (!alloc_meta) + return NULL; + return &alloc_meta->free_track[0]; } diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index d01a5ac34f70..88a6e5bee156 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -156,20 +156,31 @@ struct kasan_alloc_meta { struct qlist_node { struct qlist_node *next; }; + +/* + * Generic mode either stores free meta in the object itself or in the redzone + * after the object. In the former case free meta offset is 0, in the latter + * case it has some sane value smaller than INT_MAX. Use INT_MAX as free meta + * offset when free meta isn't present. + */ +#define KASAN_NO_FREE_META INT_MAX + struct kasan_free_meta { +#ifdef CONFIG_KASAN_GENERIC /* This field is used while the object is in the quarantine. * Otherwise it might be used for the allocator freelist. */ struct qlist_node quarantine_link; -#ifdef CONFIG_KASAN_GENERIC struct kasan_track free_track; #endif }; struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache, const void *object); +#ifdef CONFIG_KASAN_GENERIC struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache, const void *object); +#endif #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) diff --git a/mm/kasan/quarantine.c b/mm/kasan/quarantine.c index 0da3d37e1589..23f6bfb1e73f 100644 --- a/mm/kasan/quarantine.c +++ b/mm/kasan/quarantine.c @@ -135,7 +135,12 @@ static void qlink_free(struct qlist_node *qlink, struct kmem_cache *cache) if (IS_ENABLED(CONFIG_SLAB)) local_irq_save(flags); + /* + * As the object now gets freed from the quaratine, assume that its + * free track is now longer valid. + */ *(u8 *)kasan_mem_to_shadow(object) = KASAN_KMALLOC_FREE; + ___cache_free(cache, object, _THIS_IP_); if (IS_ENABLED(CONFIG_SLAB)) @@ -168,6 +173,9 @@ void quarantine_put(struct kmem_cache *cache, void *object) struct qlist_head temp = QLIST_INIT; struct kasan_free_meta *meta = kasan_get_free_meta(cache, object); + if (!meta) + return; + /* * Note: irq must be disabled until after we move the batch to the * global quarantine. Otherwise quarantine_remove_cache() can miss diff --git a/mm/kasan/report.c b/mm/kasan/report.c index ffa6076b1710..8b6656d47983 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -168,32 +168,35 @@ static void describe_object_addr(struct kmem_cache *cache, void *object, static void describe_object_stacks(struct kmem_cache *cache, void *object, const void *addr, u8 tag) { - struct kasan_alloc_meta *alloc_meta = kasan_get_alloc_meta(cache, object); - - if (cache->flags & SLAB_KASAN) { - struct kasan_track *free_track; + struct kasan_alloc_meta *alloc_meta; + struct kasan_track *free_track; + alloc_meta = kasan_get_alloc_meta(cache, object); + if (alloc_meta) { print_track(&alloc_meta->alloc_track, "Allocated"); pr_err("\n"); - free_track = kasan_get_free_track(cache, object, tag); - if (free_track) { - print_track(free_track, "Freed"); - pr_err("\n"); - } + } + + free_track = kasan_get_free_track(cache, object, tag); + if (free_track) { + print_track(free_track, "Freed"); + pr_err("\n"); + } #ifdef CONFIG_KASAN_GENERIC - if (alloc_meta->aux_stack[0]) { - pr_err("Last call_rcu():\n"); - print_stack(alloc_meta->aux_stack[0]); - pr_err("\n"); - } - if (alloc_meta->aux_stack[1]) { - pr_err("Second to last call_rcu():\n"); - print_stack(alloc_meta->aux_stack[1]); - pr_err("\n"); - } -#endif + if (!alloc_meta) + return; + if (alloc_meta->aux_stack[0]) { + pr_err("Last call_rcu():\n"); + print_stack(alloc_meta->aux_stack[0]); + pr_err("\n"); } + if (alloc_meta->aux_stack[1]) { + pr_err("Second to last call_rcu():\n"); + print_stack(alloc_meta->aux_stack[1]); + pr_err("\n"); + } +#endif } static void describe_object(struct kmem_cache *cache, void *object, diff --git a/mm/kasan/report_sw_tags.c b/mm/kasan/report_sw_tags.c index 7604b46239d4..1b026793ad57 100644 --- a/mm/kasan/report_sw_tags.c +++ b/mm/kasan/report_sw_tags.c @@ -48,9 +48,12 @@ const char *get_bug_type(struct kasan_access_info *info) object = nearest_obj(cache, page, (void *)addr); alloc_meta = kasan_get_alloc_meta(cache, object); - for (i = 0; i < KASAN_NR_FREE_STACKS; i++) - if (alloc_meta->free_pointer_tag[i] == tag) - return "use-after-free"; + if (alloc_meta) { + for (i = 0; i < KASAN_NR_FREE_STACKS; i++) { + if (alloc_meta->free_pointer_tag[i] == tag) + return "use-after-free"; + } + } return "out-of-bounds"; } diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c index e17de2619bbf..5dcd830805b2 100644 --- a/mm/kasan/sw_tags.c +++ b/mm/kasan/sw_tags.c @@ -170,6 +170,8 @@ void kasan_set_free_info(struct kmem_cache *cache, u8 idx = 0; alloc_meta = kasan_get_alloc_meta(cache, object); + if (!alloc_meta) + return; #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY idx = alloc_meta->free_track_idx; @@ -187,6 +189,8 @@ struct kasan_track *kasan_get_free_track(struct kmem_cache *cache, int i = 0; alloc_meta = kasan_get_alloc_meta(cache, object); + if (!alloc_meta) + return NULL; #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY for (i = 0; i < KASAN_NR_FREE_STACKS; i++) { From patchwork Fri Nov 13 22:20:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904991 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B80AD697 for ; Fri, 13 Nov 2020 22:21:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 80E34206F9 for ; Fri, 13 Nov 2020 22:21:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="Wogiwfd7" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 80E34206F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 639826B00A4; Fri, 13 Nov 2020 17:21:00 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5E6C26B00A6; Fri, 13 Nov 2020 17:21:00 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 487716B00A7; Fri, 13 Nov 2020 17:21:00 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0234.hostedemail.com [216.40.44.234]) by kanga.kvack.org (Postfix) with ESMTP id 083F86B00A4 for ; Fri, 13 Nov 2020 17:20:59 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id A34172497 for ; Fri, 13 Nov 2020 22:20:59 +0000 (UTC) X-FDA: 77480816238.10.food92_4b16ab127312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id 7E85B16A069 for ; Fri, 13 Nov 2020 22:20:59 +0000 (UTC) X-Spam-Summary: 1,0,0,2d3e53b98550f31f,d41d8cd98f00b204,3ygavxwokcjc1e4i5pbemc7ff7c5.3fdc9elo-ddbm13b.fi7@flex--andreyknvl.bounces.google.com,,RULES_HIT:41:152:355:379:541:800:960:966:967:973:988:989:1260:1277:1313:1314:1345:1359:1431:1437:1516:1518:1535:1543:1593:1594:1711:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2393:2525:2559:2563:2682:2685:2731:2859:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3152:3354:3865:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4118:4250:4321:4385:4425:4605:5007:6261:6653:6742:7901:8603:9025:9969:10004:10400:10450:10455:11026:11232:11473:11658:11914:12043:12295:12296:12297:12438:12555:12679:12701:12737:12895:12986:13161:13229:14181:14394:14659:14721:19904:19999:21063:21080:21365:21444:21450:21451:21627:21740:21939:21966:21990:30003:30012:30054:30070,0,RBL:209.85.219.73:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100;04ygg4c6qxkjmxj94ufhty7 93ajzdyp X-HE-Tag: food92_4b16ab127312 X-Filterd-Recvd-Size: 7067 Received: from mail-qv1-f73.google.com (mail-qv1-f73.google.com [209.85.219.73]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:20:58 +0000 (UTC) Received: by mail-qv1-f73.google.com with SMTP id 12so7003482qvr.8 for ; Fri, 13 Nov 2020 14:20:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=VblPcwxF/Hme6lihpSqzEG9568rY0eTc9RDR6WgTVWw=; b=Wogiwfd7B8TITgy/dJX2SuNVBCGj1sMmA/wWfqUVUW625r77eKdpvQaDaSu3aBij18 zEhjdHmrCW/gZ7A+s95KqIfE8DNCf76WA+Nww0tJUkPRyTX4kRL6suTfbHi2qlyVafam rV+hV43GvnYUfMMgdDZbuxRdthx5R+R5/Y2EHztggMs7SRvCIai8cg3H8bvmqhAi17Nt 36le6MqSx5RpWtq05dKx2kbxkwESQg8uMTwlplCYYGRgGySvTwMQEijRKAHcWAwock2d 3rNzJuPXD5wPMUIWUOa1SBHN9tz+5KvOK3ae7A6N6hVkTe9bRJuMUVjKXK9evatpgUtX DvbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=VblPcwxF/Hme6lihpSqzEG9568rY0eTc9RDR6WgTVWw=; b=BDogbmgo+fAx6Y35NMOY3sCQd0sVJBTISIs+ekDuoF46msT6woagn8K9W/E/qhGFJk MAHSJIbrrmX+7MRGEFI71exTpDpTJx2dyL74eFDk/GJpVldEjPN+WWfgim6Mj8FUXJOo bQS+WUt7MKH7GE7giE5LB04FHS4gqYqZUzqGwgFquyL6i0UF8SVBQkilRZ1txXPAUhko 5TGjOftEZ8BazVDTcDc7bwjwxkxLUBHe9VcAOG13Rr3BtQY6CncW3zEUKQNXBa8aZ+m/ Rf2aVgDCwoNV+Q33lM42696cx+1Ugjhl9of4saNu4hBKFsHH5SDyRAHhxLzA1JI4PwCD 79Gg== X-Gm-Message-State: AOAM532PBMLSqyyITLhP57TeMcaFRf7hCcPpwkcl064QhYI/27BEPY4v ojtHRfgcwdezGxJy3ikIf7JQhNgpUrOCzYYs X-Google-Smtp-Source: ABdhPJxPL1Ew5OnNzRmKgE563d6wfxOAtulSj3z/jp9Yh8hehdEomONqkappFCtKfgmpNirQUmmu+Nt5HUJLowo+ X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a0c:9e53:: with SMTP id z19mr4596536qve.23.1605306058182; Fri, 13 Nov 2020 14:20:58 -0800 (PST) Date: Fri, 13 Nov 2020 23:20:08 +0100 In-Reply-To: Message-Id: <6f0a1e72783ddac000ac08e7315b1d7c0ca4ec51.1605305978.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 18/19] kasan, mm: allow cache merging with no metadata From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov , Vincenzo Frascino X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The reason cache merging is disabled with KASAN is because KASAN puts its metadata right after the allocated object. When the merged caches have slightly different sizes, the metadata ends up in different places, which KASAN doesn't support. It might be possible to adjust the metadata allocation algorithm and make it friendly to the cache merging code. Instead this change takes a simpler approach and allows merging caches when no metadata is present. Which is the case for hardware tag-based KASAN with kasan.mode=prod. Co-developed-by: Vincenzo Frascino Signed-off-by: Vincenzo Frascino Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/Ia114847dfb2244f297d2cb82d592bf6a07455dba Reviewed-by: Marco Elver Reviewed-by: Dmitry Vyukov --- include/linux/kasan.h | 21 +++++++++++++++++++-- mm/kasan/common.c | 11 +++++++++++ mm/slab_common.c | 3 ++- 3 files changed, 32 insertions(+), 3 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 16cf53eac29b..173a8e81d001 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -81,17 +81,30 @@ struct kasan_cache { }; #ifdef CONFIG_KASAN_HW_TAGS + DECLARE_STATIC_KEY_FALSE(kasan_flag_enabled); + static __always_inline bool kasan_enabled(void) { return static_branch_likely(&kasan_flag_enabled); } -#else + +#else /* CONFIG_KASAN_HW_TAGS */ + static inline bool kasan_enabled(void) { return true; } -#endif + +#endif /* CONFIG_KASAN_HW_TAGS */ + +slab_flags_t __kasan_never_merge(void); +static __always_inline slab_flags_t kasan_never_merge(void) +{ + if (kasan_enabled()) + return __kasan_never_merge(); + return 0; +} void __kasan_unpoison_range(const void *addr, size_t size); static __always_inline void kasan_unpoison_range(const void *addr, size_t size) @@ -238,6 +251,10 @@ static inline bool kasan_enabled(void) { return false; } +static inline slab_flags_t kasan_never_merge(void) +{ + return 0; +} static inline void kasan_unpoison_range(const void *address, size_t size) {} static inline void kasan_alloc_pages(struct page *page, unsigned int order) {} static inline void kasan_free_pages(struct page *page, unsigned int order) {} diff --git a/mm/kasan/common.c b/mm/kasan/common.c index cf874243efab..a5a4dcb1254d 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -87,6 +87,17 @@ asmlinkage void kasan_unpoison_task_stack_below(const void *watermark) } #endif /* CONFIG_KASAN_STACK */ +/* + * Only allow cache merging when stack collection is disabled and no metadata + * is present. + */ +slab_flags_t __kasan_never_merge(void) +{ + if (kasan_stack_collection_enabled()) + return SLAB_KASAN; + return 0; +} + void __kasan_alloc_pages(struct page *page, unsigned int order) { u8 tag; diff --git a/mm/slab_common.c b/mm/slab_common.c index 0b5ae1819a8b..075b23ce94ec 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -54,7 +55,7 @@ static DECLARE_WORK(slab_caches_to_rcu_destroy_work, */ #define SLAB_NEVER_MERGE (SLAB_RED_ZONE | SLAB_POISON | SLAB_STORE_USER | \ SLAB_TRACE | SLAB_TYPESAFE_BY_RCU | SLAB_NOLEAKTRACE | \ - SLAB_FAILSLAB | SLAB_KASAN) + SLAB_FAILSLAB | kasan_never_merge()) #define SLAB_MERGE_SAME (SLAB_RECLAIM_ACCOUNT | SLAB_CACHE_DMA | \ SLAB_CACHE_DMA32 | SLAB_ACCOUNT) From patchwork Fri Nov 13 22:20:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Konovalov X-Patchwork-Id: 11904993 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6DD09697 for ; Fri, 13 Nov 2020 22:21:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1A9C6206F9 for ; Fri, 13 Nov 2020 22:21:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="O0Kwm8Q+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1A9C6206F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 882286B00A8; Fri, 13 Nov 2020 17:21:03 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7E5A46B00A9; Fri, 13 Nov 2020 17:21:03 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 612B66B00AA; Fri, 13 Nov 2020 17:21:03 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0159.hostedemail.com [216.40.44.159]) by kanga.kvack.org (Postfix) with ESMTP id 269EC6B00A8 for ; Fri, 13 Nov 2020 17:21:03 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id C0E52181AEF09 for ; Fri, 13 Nov 2020 22:21:02 +0000 (UTC) X-FDA: 77480816364.01.roll14_11103fa27312 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id A2D0310049A5E for ; Fri, 13 Nov 2020 22:21:02 +0000 (UTC) X-Spam-Summary: 1,0,0,ea1cc085b9ba2cc3,d41d8cd98f00b204,3zaavxwokcjk3g6k7rdgoe9hh9e7.5hfebgnq-ffdo35d.hk9@flex--andreyknvl.bounces.google.com,,RULES_HIT:4:41:69:152:355:379:541:800:960:966:967:973:982:988:989:1042:1260:1277:1313:1314:1345:1431:1437:1516:1518:1593:1594:1605:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2393:2525:2559:2566:2570:2639:2682:2685:2693:2703:2859:2892:2894:2901:2902:2903:2911:2918:2924:2925:2926:2933:2937:2939:2942:2945:2947:2951:2954:3022:3152:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4425:4605:5007:6117:6119:6261:7875:7901:7903:7904:8603:8660:8957:9025:9121:9969:10004:11658:12219:13148:13230:30056,0,RBL:209.85.221.73:@flex--andreyknvl.bounces.google.com:.lbl8.mailshell.net-62.18.0.100 66.100.201.100;04yrnwzym35915bdxhdiu9up7e4kxocnixf5dzdpepm38s9txbg95ofjew9bf8d.ad6ycbd97usy1rs4bet5sod3q4ensys4nwmojh9rrruqadfrh9xni3zi1nfghux.g-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian :0.5,0.5 X-HE-Tag: roll14_11103fa27312 X-Filterd-Recvd-Size: 16827 Received: from mail-wr1-f73.google.com (mail-wr1-f73.google.com [209.85.221.73]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Fri, 13 Nov 2020 22:21:02 +0000 (UTC) Received: by mail-wr1-f73.google.com with SMTP id r15so4663599wrn.15 for ; Fri, 13 Nov 2020 14:21:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=eyOrtcKen7fgrfwARq8tGqpWknv3CPxMtmzrBg32L54=; b=O0Kwm8Q+s44JcGB/AtbKCNpR/VG8oK5sOrln2Ln/mpjlt/J5f19GFlvre+tKbwhytY eJk8paW3Ggm34w8sSwTinp4wMEdtG9WBBLm+el5gAyOVTq5rXVOu9Wmldbgi+6doOooA /YBSmmqDzJadJlZb12kBHqkNWf1ae+E+4T9HYhVUiPxlUvU2bc4UFXMEeQYl4Fv6Z5sI j82jVra71cMq42coFmwruTBcl2+KpUDo32GQcg/wDr5xZ6wE1W/uieh0/qiMZRZRvQLV sPYpMtxafq9Re7uK5aG85xPt8RNC3crN9GedGBUhAdVy7yA/X9WL82N7mqqEh1kv+YZ4 ABvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=eyOrtcKen7fgrfwARq8tGqpWknv3CPxMtmzrBg32L54=; b=e8O1gQ/OEzK8feeimdWR9Bjk6LItm4Dl1/KigcV+aYOszqcfIJj/6fSUXkViANIO44 Yu+b23fkEQ66klaqoDKLkJ0fexV2jRKL0jdRbFVqjGb90x4+dmnOQrZZ+bsiXj/ucsDL gtVe2HZAbXyBikR/OtLKPwPHQG1D3/EyGyRmnumOr49gUL74ARI1SRF/PQUFB/tTVsQh KLR7TYUFlebAXNx9g3n5un4ye0bYdnCN2oNwvBpaVF82/zVBNJhy94TxvjMrNYUM+LVX 4DlkniEoCt1brOkkxxSnQVf9pItwtuCC/3+DTGLwn/fFFe+CizTnJe/qMWuP6CuyVNuu leQg== X-Gm-Message-State: AOAM531l8oOTFMGJidOvoUesVA2G7HzKNGs9oNH3jE3TZss/IubbAImf 1TXqwGOy8V1kLQ418wCt6e133WQf+vUKzRCK X-Google-Smtp-Source: ABdhPJzcSLgrfEHEsnxNSacjzJpW5iTQm/d7cUrxxvowHRiIiVk33HEA3Z6rKplR2mcWiPF/pgEiyAiPhNzDdAei X-Received: from andreyknvl3.muc.corp.google.com ([2a00:79e0:15:13:7220:84ff:fe09:7e9d]) (user=andreyknvl job=sendgmr) by 2002:a1c:dc0a:: with SMTP id t10mr4712946wmg.5.1605306060823; Fri, 13 Nov 2020 14:21:00 -0800 (PST) Date: Fri, 13 Nov 2020 23:20:09 +0100 In-Reply-To: Message-Id: <1668fc1d020f9c1f2b8bf57467f0433bfd74d0a3.1605305978.git.andreyknvl@google.com> Mime-Version: 1.0 References: X-Mailer: git-send-email 2.29.2.299.gdc1121823c-goog Subject: [PATCH mm v3 19/19] kasan: update documentation From: Andrey Konovalov To: Andrew Morton Cc: Catalin Marinas , Will Deacon , Vincenzo Frascino , Dmitry Vyukov , Andrey Ryabinin , Alexander Potapenko , Marco Elver , Evgenii Stepanov , Branislav Rankov , Kevin Brodsky , kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This change updates KASAN documentation to reflect the addition of boot parameters and also reworks and clarifies some of the existing sections, in particular: defines what a memory granule is, mentions quarantine, makes Kunit section more readable. Signed-off-by: Andrey Konovalov Link: https://linux-review.googlesource.com/id/Ib1f83e91be273264b25f42b04448ac96b858849f Reviewed-by: Marco Elver Reviewed-by: Dmitry Vyukov --- Documentation/dev-tools/kasan.rst | 186 +++++++++++++++++++----------- 1 file changed, 116 insertions(+), 70 deletions(-) diff --git a/Documentation/dev-tools/kasan.rst b/Documentation/dev-tools/kasan.rst index ffbae8ce5748..0d5d77919b1a 100644 --- a/Documentation/dev-tools/kasan.rst +++ b/Documentation/dev-tools/kasan.rst @@ -4,8 +4,9 @@ The Kernel Address Sanitizer (KASAN) Overview -------- -KernelAddressSANitizer (KASAN) is a dynamic memory error detector designed to -find out-of-bound and use-after-free bugs. KASAN has three modes: +KernelAddressSANitizer (KASAN) is a dynamic memory safety error detector +designed to find out-of-bound and use-after-free bugs. KASAN has three modes: + 1. generic KASAN (similar to userspace ASan), 2. software tag-based KASAN (similar to userspace HWASan), 3. hardware tag-based KASAN (based on hardware memory tagging). @@ -39,23 +40,13 @@ CONFIG_KASAN_INLINE. Outline and inline are compiler instrumentation types. The former produces smaller binary while the latter is 1.1 - 2 times faster. Both software KASAN modes work with both SLUB and SLAB memory allocators, -hardware tag-based KASAN currently only support SLUB. -For better bug detection and nicer reporting, enable CONFIG_STACKTRACE. +while the hardware tag-based KASAN currently only support SLUB. + +For better error reports that include stack traces, enable CONFIG_STACKTRACE. To augment reports with last allocation and freeing stack of the physical page, it is recommended to enable also CONFIG_PAGE_OWNER and boot with page_owner=on. -To disable instrumentation for specific files or directories, add a line -similar to the following to the respective kernel Makefile: - -- For a single file (e.g. main.o):: - - KASAN_SANITIZE_main.o := n - -- For all files in one directory:: - - KASAN_SANITIZE := n - Error reports ~~~~~~~~~~~~~ @@ -140,22 +131,75 @@ freed (in case of a use-after-free bug report). Next comes a description of the accessed slab object and information about the accessed memory page. In the last section the report shows memory state around the accessed address. -Reading this part requires some understanding of how KASAN works. - -The state of each 8 aligned bytes of memory is encoded in one shadow byte. -Those 8 bytes can be accessible, partially accessible, freed or be a redzone. -We use the following encoding for each shadow byte: 0 means that all 8 bytes -of the corresponding memory region are accessible; number N (1 <= N <= 7) means -that the first N bytes are accessible, and other (8 - N) bytes are not; -any negative value indicates that the entire 8-byte word is inaccessible. -We use different negative values to distinguish between different kinds of -inaccessible memory like redzones or freed memory (see mm/kasan/kasan.h). +Internally KASAN tracks memory state separately for each memory granule, which +is either 8 or 16 aligned bytes depending on KASAN mode. Each number in the +memory state section of the report shows the state of one of the memory +granules that surround the accessed address. + +For generic KASAN the size of each memory granule is 8. The state of each +granule is encoded in one shadow byte. Those 8 bytes can be accessible, +partially accessible, freed or be a part of a redzone. KASAN uses the following +encoding for each shadow byte: 0 means that all 8 bytes of the corresponding +memory region are accessible; number N (1 <= N <= 7) means that the first N +bytes are accessible, and other (8 - N) bytes are not; any negative value +indicates that the entire 8-byte word is inaccessible. KASAN uses different +negative values to distinguish between different kinds of inaccessible memory +like redzones or freed memory (see mm/kasan/kasan.h). In the report above the arrows point to the shadow byte 03, which means that the accessed address is partially accessible. For tag-based KASAN this last report section shows the memory tags around the -accessed address (see Implementation details section). +accessed address (see `Implementation details`_ section). + +Boot parameters +~~~~~~~~~~~~~~~ + +Hardware tag-based KASAN mode (see the section about different mode below) is +intended for use in production as a security mitigation. Therefore it supports +boot parameters that allow to disable KASAN competely or otherwise control +particular KASAN features. + +The things that can be controlled are: + +1. Whether KASAN is enabled at all. +2. Whether KASAN collects and saves alloc/free stacks. +3. Whether KASAN panics on a detected bug or not. + +The ``kasan.mode`` boot parameter allows to choose one of three main modes: + +- ``kasan.mode=off`` - KASAN is disabled, no tag checks are performed +- ``kasan.mode=prod`` - only essential production features are enabled +- ``kasan.mode=full`` - all KASAN features are enabled + +The chosen mode provides default control values for the features mentioned +above. However it's also possible to override the default values by providing: + +- ``kasan.stacktrace=off`` or ``=on`` - enable alloc/free stack collection + (default: ``on`` for ``mode=full``, + otherwise ``off``) +- ``kasan.fault=report`` or ``=panic`` - only print KASAN report or also panic + (default: ``report``) + +If ``kasan.mode`` parameter is not provided, it defaults to ``full`` when +``CONFIG_DEBUG_KERNEL`` is enabled, and to ``prod`` otherwise. + +For developers +~~~~~~~~~~~~~~ + +Software KASAN modes use compiler instrumentation to insert validity checks. +Such instrumentation might be incompatible with some part of the kernel, and +therefore needs to be disabled. To disable instrumentation for specific files +or directories, add a line similar to the following to the respective kernel +Makefile: + +- For a single file (e.g. main.o):: + + KASAN_SANITIZE_main.o := n + +- For all files in one directory:: + + KASAN_SANITIZE := n Implementation details @@ -164,10 +208,10 @@ Implementation details Generic KASAN ~~~~~~~~~~~~~ -From a high level, our approach to memory error detection is similar to that -of kmemcheck: use shadow memory to record whether each byte of memory is safe -to access, and use compile-time instrumentation to insert checks of shadow -memory on each memory access. +From a high level perspective, KASAN's approach to memory error detection is +similar to that of kmemcheck: use shadow memory to record whether each byte of +memory is safe to access, and use compile-time instrumentation to insert checks +of shadow memory on each memory access. Generic KASAN dedicates 1/8th of kernel memory to its shadow memory (e.g. 16TB to cover 128TB on x86_64) and uses direct mapping with a scale and offset to @@ -194,7 +238,10 @@ function calls GCC directly inserts the code to check the shadow memory. This option significantly enlarges kernel but it gives x1.1-x2 performance boost over outline instrumented kernel. -Generic KASAN prints up to 2 call_rcu() call stacks in reports, the last one +Generic KASAN is the only mode that delays the reuse of freed object via +quarantine (see mm/kasan/quarantine.c for implementation). + +Generic KASAN prints up to two call_rcu() call stacks in reports, the last one and the second to last. Software tag-based KASAN @@ -304,15 +351,15 @@ therefore be wasteful. Furthermore, to ensure that different mappings use different shadow pages, mappings would have to be aligned to ``KASAN_GRANULE_SIZE * PAGE_SIZE``. -Instead, we share backing space across multiple mappings. We allocate +Instead, KASAN shares backing space across multiple mappings. It allocates a backing page when a mapping in vmalloc space uses a particular page of the shadow region. This page can be shared by other vmalloc mappings later on. -We hook in to the vmap infrastructure to lazily clean up unused shadow +KASAN hooks into the vmap infrastructure to lazily clean up unused shadow memory. -To avoid the difficulties around swapping mappings around, we expect +To avoid the difficulties around swapping mappings around, KASAN expects that the part of the shadow region that covers the vmalloc space will not be covered by the early shadow page, but will be left unmapped. This will require changes in arch-specific code. @@ -323,24 +370,31 @@ architectures that do not have a fixed module region. CONFIG_KASAN_KUNIT_TEST & CONFIG_TEST_KASAN_MODULE -------------------------------------------------- -``CONFIG_KASAN_KUNIT_TEST`` utilizes the KUnit Test Framework for testing. -This means each test focuses on a small unit of functionality and -there are a few ways these tests can be run. +KASAN tests consist on two parts: + +1. Tests that are integrated with the KUnit Test Framework. Enabled with +``CONFIG_KASAN_KUNIT_TEST``. These tests can be run and partially verified +automatically in a few different ways, see the instructions below. -Each test will print the KASAN report if an error is detected and then -print the number of the test and the status of the test: +2. Tests that are currently incompatible with KUnit. Enabled with +``CONFIG_TEST_KASAN_MODULE`` and can only be run as a module. These tests can +only be verified manually, by loading the kernel module and inspecting the +kernel log for KASAN reports. -pass:: +Each KUnit-compatible KASAN test prints a KASAN report if an error is detected. +Then the test prints its number and status. + +When a test passes:: ok 28 - kmalloc_double_kzfree -or, if kmalloc failed:: +When a test fails due to a failed ``kmalloc``:: # kmalloc_large_oob_right: ASSERTION FAILED at lib/test_kasan.c:163 Expected ptr is not null, but is not ok 4 - kmalloc_large_oob_right -or, if a KASAN report was expected, but not found:: +When a test fails due to a missing KASAN report:: # kmalloc_double_kzfree: EXPECTATION FAILED at lib/test_kasan.c:629 Expected kasan_data->report_expected == kasan_data->report_found, but @@ -348,46 +402,38 @@ or, if a KASAN report was expected, but not found:: kasan_data->report_found == 0 not ok 28 - kmalloc_double_kzfree -All test statuses are tracked as they run and an overall status will -be printed at the end:: +At the end the cumulative status of all KASAN tests is printed. On success:: ok 1 - kasan -or:: +Or, if one of the tests failed:: not ok 1 - kasan -(1) Loadable Module -~~~~~~~~~~~~~~~~~~~~ + +There are a few ways to run KUnit-compatible KASAN tests. + +1. Loadable module +~~~~~~~~~~~~~~~~~~ With ``CONFIG_KUNIT`` enabled, ``CONFIG_KASAN_KUNIT_TEST`` can be built as -a loadable module and run on any architecture that supports KASAN -using something like insmod or modprobe. The module is called ``test_kasan``. +a loadable module and run on any architecture that supports KASAN by loading +the module with insmod or modprobe. The module is called ``test_kasan``. -(2) Built-In -~~~~~~~~~~~~~ +2. Built-In +~~~~~~~~~~~ With ``CONFIG_KUNIT`` built-in, ``CONFIG_KASAN_KUNIT_TEST`` can be built-in -on any architecure that supports KASAN. These and any other KUnit -tests enabled will run and print the results at boot as a late-init -call. +on any architecure that supports KASAN. These and any other KUnit tests enabled +will run and print the results at boot as a late-init call. -(3) Using kunit_tool -~~~~~~~~~~~~~~~~~~~~~ +3. Using kunit_tool +~~~~~~~~~~~~~~~~~~~ -With ``CONFIG_KUNIT`` and ``CONFIG_KASAN_KUNIT_TEST`` built-in, we can also -use kunit_tool to see the results of these along with other KUnit -tests in a more readable way. This will not print the KASAN reports -of tests that passed. Use `KUnit documentation `_ for more up-to-date -information on kunit_tool. +With ``CONFIG_KUNIT`` and ``CONFIG_KASAN_KUNIT_TEST`` built-in, it's also +possible use ``kunit_tool`` to see the results of these and other KUnit tests +in a more readable way. This will not print the KASAN reports of the tests that +passed. Use `KUnit documentation `_ +for more up-to-date information on ``kunit_tool``. .. _KUnit: https://www.kernel.org/doc/html/latest/dev-tools/kunit/index.html - -``CONFIG_TEST_KASAN_MODULE`` is a set of KASAN tests that could not be -converted to KUnit. These tests can be run only as a module with -``CONFIG_TEST_KASAN_MODULE`` built as a loadable module and -``CONFIG_KASAN`` built-in. The type of error expected and the -function being run is printed before the expression expected to give -an error. Then the error is printed, if found, and that test -should be interpretted to pass only if the error was the one expected -by the test.