From patchwork Tue Aug 29 17:11:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: andrey.konovalov@linux.dev X-Patchwork-Id: 13369229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 017E9C83F12 for ; Tue, 29 Aug 2023 17:11:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 75F0F8E002D; Tue, 29 Aug 2023 13:11:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 70DAF8E0029; Tue, 29 Aug 2023 13:11:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5D73A8E002D; Tue, 29 Aug 2023 13:11:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 4E9938E0029 for ; Tue, 29 Aug 2023 13:11:39 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 1EA1AC023C for ; Tue, 29 Aug 2023 17:11:39 +0000 (UTC) X-FDA: 81177783918.14.05F1AAE Received: from out-247.mta1.migadu.com (out-247.mta1.migadu.com [95.215.58.247]) by imf30.hostedemail.com (Postfix) with ESMTP id 47C7B8000E for ; Tue, 29 Aug 2023 17:11:37 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=dnT146Mf; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf30.hostedemail.com: domain of andrey.konovalov@linux.dev designates 95.215.58.247 as permitted sender) smtp.mailfrom=andrey.konovalov@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693329097; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=4p8eo+KqdaDj4APbJngGg5i5hNM+Az1Zd3d9hROUHxc=; b=bhPSTcSJmBJR478h92MJLIGucU7JpundrSQBL1tKVCJ2oOCTHpz3bJ0MhDzlz9DiCMTeL2 XiYTRCZaxwtLwQpO9kVfd3psQ4sXyfn0X7qYGxxyf7+SnHWl89F+ooF3XO0hKRhCcxIftu 0wl6Gyk61drnpxrncjPB7ZSxu9GGLK8= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=dnT146Mf; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf30.hostedemail.com: domain of andrey.konovalov@linux.dev designates 95.215.58.247 as permitted sender) smtp.mailfrom=andrey.konovalov@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1693329097; a=rsa-sha256; cv=none; b=F1aVfLNfv5T4PHzXczNbwIHnSTxcQfd5i5o557DX7/mqB6Dxzeomr1EJ33ZFPdnag1OA1L /oE1BY8N7RWSSPFZQumKlbgl8H9PP1GFLETB5TAp3a/YxaIGN+URQ5w0SRLW1vc3+Lvtq+ zByDodOKQ6UVeEx+MCAmjnd/j9xq8HU= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1693329095; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=4p8eo+KqdaDj4APbJngGg5i5hNM+Az1Zd3d9hROUHxc=; b=dnT146MfGXO/FX/5OU0nnhGox5bH9Q9iL3wR8QzreWKi8HM6ftxenQbNh0giCupLWbBvZ3 yrtJFrQqJMbwnzOPm1V30EMbaXRjHIUhQydgFATDwvyKhMPATQoGC3i906ED2ejiv5kKAp 3LsU6hfXBzBqwgYR0HkngXLEuukvWWo= From: andrey.konovalov@linux.dev To: Marco Elver , Alexander Potapenko Cc: Andrey Konovalov , Dmitry Vyukov , Vlastimil Babka , kasan-dev@googlegroups.com, Evgenii Stepanov , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov Subject: [PATCH 00/15] stackdepot: allow evicting stack traces Date: Tue, 29 Aug 2023 19:11:10 +0200 Message-Id: MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Stat-Signature: 5aotsej57d3x8uxrgdc7mkpfctauo1i8 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 47C7B8000E X-HE-Tag: 1693329097-335305 X-HE-Meta: U2FsdGVkX1/zHXZe6wbcmzH/VPC/P6Xq+xp0yNyzPQ7PB71B+XeJ1B2xqKWnq/1+5o4hqhfuIxn0+NtuDXrnc37ZWDA1HoTKilYBl/mgUHr3rUqcZqLNEPf9AXSkUeYwQvCC/NlRMRhYt8s8I5uSeAiFhCyVfd3Dror6a6LCSy1NJ9RYjQxAZ5XBfxBTeKSe+mGe/aLuXXcJRE5uIeQ+WPv/1o5gduqdN+2lIT+RThmMc7ULXfaR0HcoKCsrJhDH6wDLBnqDUpX2SSKoveAFbhL1xYAfjwQYez2FAnyFH8m1N9I5n4kaLItRPGTEz8EAIhKuCTj+dbfPyCqei+dPxBc70Y6ZndiY0BtJnN7h7OK/LINSyek4wt16cOaKshio9nrMcGWDrN8JJn78S6ZgcHXQDL3zB50IPDVrP4xhxt6njiBMRxurH0HZZBlm2Y94GO+wkZIpKB4DtDX601+SVj7maY74giPyP0mczmfa4F9I3Oo/lHU601ut9NUWeZmOZNSalkD9A+uKqud7i82b/oIUT3HeiiLybIsfgIOYUVmdXdn80WsS2KvmxfQA3MbsEwa5hrT3yXY3aqtnT5Bpj7e0IIOJLaElPI2789tPUEtmECQEr+v7L5hqT8UFFBtHzIwK5Wr22bk5EoPzYs6lPfUBWghkE3VgYDjZ6YI9kFkFfGs5UovW10bfyhuRwk9yg263gEH/H9WcmihFSp1V8qM6YzwjZuytCmiHNPk+CPK+7j+zIeVvHGpId88E0pOdLHQL9LwdWSRbMwhBHMe4V3soYdPD8wsOwxSMlS9AkYRGM9UfuI8GOKHt99J7zWhk7IPMCFiU3s7tqliyCkxBRwMm/MGJZ8/kpc45OX45seBP4uEcXDEesVSyHgneEzstJBbSjrdBB0ZMQahwjD2D+xeV/HQu/37D7dv5uAGIlTv692KvWhxfdeLmBCTjbSiwIjd7tS9FtV7MjttZM4u 6dCId3XN Q1ncAPF/HsEFameZeBUuBAQyvP0SbLSx/mM2dTlR1MkIMi/HkYH75qqFos2dRYz+ZiDqYYS3NmrTs+6veOcTET6XHN2mAZUbVdMQY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Currently, the stack depot grows indefinitely until it reaches its capacity. Once that happens, the stack depot stops saving new stack traces. This creates a problem for using the stack depot for in-field testing and in production. For such uses, an ideal stack trace storage should: 1. Allow saving fresh stack traces on systems with a large uptime while limiting the amount of memory used to store the traces; 2. Have a low performance impact. Implementing #1 in the stack depot is impossible with the current keep-forever approach. This series targets to address that. Issue #2 is left to be addressed in a future series. This series changes the stack depot implementation to allow evicting unneeded stack traces from the stack depot. The users of the stack depot can do that via a new stack_depot_evict API. Internal changes to the stack depot code include: 1. Storing stack traces in 32-frame-sized slots (vs precisely-sized slots in the current implementation); 2. Keeping available slots in a freelist (vs keeping an offset to the next free slot); 3. Using a read/write lock for synchronization (vs a lock-free approach combined with a spinlock). This series also integrates the eviction functionality in the tag-based KASAN modes. (I will investigate integrating it into the Generic mode as well in the following iterations of this series.) Despite wasting some space on rounding up the size of each stack record to 32 frames, with this change, the tag-based KASAN modes end up consuming ~5% less memory in stack depot during boot (with the default stack ring size of 32k entries). The reason for this is the eviction of irrelevant stack traces from the stack depot, which frees up space for other stack traces. For other tools that heavily rely on the stack depot, like Generic KASAN and KMSAN, this change leads to the stack depot capacity being reached sooner than before. However, as these tools are mainly used in fuzzing scenarios where the kernel is frequently rebooted, this outcome should be acceptable. There is no measurable boot time performace impact of these changes for KASAN on x86-64. I haven't done any tests for arm64 modes (the stack depot without performance optimizations is not suitable for intended use of those anyway), but I expect a similar result. Obtaining and copying stack trace frames when saving them into stack depot is what takes the most time. This series does not yet provide a way to configure the maximum size of the stack depot externally (e.g. via a command-line parameter). This will either be added in the following iterations of this series (if the used approach gets approval) or will be added together with the performance improvement changes. Andrey Konovalov (15): stackdepot: check disabled flag when fetching stackdepot: simplify __stack_depot_save stackdepot: drop valid bit from handles stackdepot: add depot_fetch_stack helper stackdepot: use fixed-sized slots for stack records stackdepot: fix and clean-up atomic annotations stackdepot: rework helpers for depot_alloc_stack stackdepot: rename next_pool_required to new_pool_required stackdepot: store next pool pointer in new_pool stackdepot: store free stack records in a freelist stackdepot: use read/write lock stackdepot: add refcount for records stackdepot: add backwards links to hash table buckets stackdepot: allow users to evict stack traces kasan: use stack_depot_evict for tag-based modes include/linux/stackdepot.h | 11 ++ lib/stackdepot.c | 361 ++++++++++++++++++++++++------------- mm/kasan/tags.c | 7 +- 3 files changed, 249 insertions(+), 130 deletions(-)