From patchwork Tue Jul 11 14:33:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexander Potapenko X-Patchwork-Id: 13308885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CB5C3EB64DC for ; Tue, 11 Jul 2023 14:34:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: Mime-Version:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Owner; bh=zJkpx4I7BX/0NcvILSwCzFoAb9l5s+YQMc4vJ3MaXrw=; b=NsN /aB86eAGI/5fiW7atW3DP7wF5X8HkVx9KYnFWGlcW7WQsROIQjIw47bEzvP9c/IQ/fRcCijYpOqLC GyeshLJ3RYlVNyIGP3L+PSn9R+pNG7PhiFHCTQMdrF2UTbqO6+awPzuJLVnoFGH2/pLToKoFxantO VWXpRS4c+mhiX2bssJUzjC2QlLAinO8R1fc7D14//ie54bilPRqu8GKv2b33JskLTjnaAeWn7WYHU WIJy56mviMkFlaZh5Q8+Kthjq10bDQnVgOtsPlj4V6gJDTpStuxicyRqcGgrNp71JHaVJB8PzNa/i 9JHck7KW4NGht1VZssMDixPAjd4+C5A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qJER6-00F8hm-21; Tue, 11 Jul 2023 14:33:52 +0000 Received: from mail-ed1-x54a.google.com ([2a00:1450:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qJER4-00F8gR-0M for linux-arm-kernel@lists.infradead.org; Tue, 11 Jul 2023 14:33:51 +0000 Received: by mail-ed1-x54a.google.com with SMTP id 4fb4d7f45d1cf-51866148986so3631476a12.3 for ; Tue, 11 Jul 2023 07:33:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1689086026; x=1691678026; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=VGRLS9PZfnsNdW8E7+rt3QYOMitkf0dfb3D8ONqanVU=; b=ACM+cmGPaftl+r3a03wCYTyz2PTNsij/STVLE0V1vKEw3eDekfjR2QqscyUvAcrHV3 QGkpbdnQNoqC975Ln+UEnAT8MAFdiF7vk8wpKVcOjwCMJPgCiUT3Z4Loe0Qrcu0RgJkt cMhJU1XgJyJznWvlLYniCeM+vUrEe1caZbLfkbgRFTu2S0OvU1IS+QOIfUAhO9o7et0A i5oKV90GR6Av8ruuPsx32GRhKe4GRqhxmdmkntNNXPEIqb7g/fjMNp+Wd5jatboPFKjA fus/lzZ24m9bSMyr98fSmlv1TxZOLLqGkAmuMg815NfxzMDvgWaEgBcrghFvygJFuET1 OZZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689086026; x=1691678026; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=VGRLS9PZfnsNdW8E7+rt3QYOMitkf0dfb3D8ONqanVU=; b=BUd3I4TQXoGV+KfPRCk+akseJnKUTmq47eTz/IkFymphv9BQQ/mRgSPlo4yjucQ5l0 84tib00pw+vuPH1a6I3R0SYPU0uHavIwVTCGfaUttf7EiuZlLq/pAZF3iJV3LyhbX6rv 6blIz5qBKKg4V2hzyPKWCjBkN9QdFhFepIMU9EJBe3rXb9PqS7i3FCT0P4m7DIZAvWpT 0kpRK1fzgoSSvH6kGYbzUXq2X5HvRoDHIRaAkD9+0tekFRLAryi03PhoAttTesAMn+tv IeEfe0bCwgktZsWIyW37UJWnAX9lNl1MFx+/tVy/GhXlU54oU8vSK7YHqHveJW/7KUqI ZWzw== X-Gm-Message-State: ABy/qLaMVXfBCFEe4Evwc1DvQGhMfycVNgDWqywIsfRmKYTMvZFvtuKp fxeg0Od25cIVR3ijhhrLaWJFyYod2CQ= X-Google-Smtp-Source: APBJJlEc2CXHpERZgeEynf388KWaCCGl5zbXJJ2YYem5Q9YuIP0R6kODxnUyy5vGIV8WQAS14Zxnx8YEWQg= X-Received: from glider.muc.corp.google.com ([2a00:79e0:9c:201:564d:3aaa:6b5f:4419]) (user=glider job=sendgmr) by 2002:a50:bb67:0:b0:51e:164c:5444 with SMTP id y94-20020a50bb67000000b0051e164c5444mr48540ede.8.1689086026234; Tue, 11 Jul 2023 07:33:46 -0700 (PDT) Date: Tue, 11 Jul 2023 16:33:27 +0200 Mime-Version: 1.0 X-Mailer: git-send-email 2.41.0.255.g8b1d071c50-goog Message-ID: <20230711143337.3086664-1-glider@google.com> Subject: [PATCH 0/5] Implement MTE tag compression for swapped pages From: Alexander Potapenko To: glider@google.com, catalin.marinas@arm.com, will@kernel.org, pcc@google.com, andreyknvl@gmail.com Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, eugenis@google.com, yury.norov@gmail.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230711_073350_153361_F4585AA3 X-CRM114-Status: GOOD ( 14.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, when MTE pages are swapped out, the tags are kept in the memory, occupying 128 bytes per page. This is especially problematic for devices that use zram-backed in-memory swap, because tags stored uncompressed in the heap effectively reduce the available amount of swap memory. The RLE-based EA0 algorithm suggested by Evgenii Stepanov and implemented in this patch series is able to efficiently compress 128-byte tag buffers, resulting in practical compression ratio between 2.5x and 20x. In most cases it is possible to store the compressed data in 63-bit Xarray values, resulting in no extra memory allocations. Our measurements show that EA0 provides better compression than existing kernel compression algorithms (LZ4, LZO, LZ4HC, ZSTD) can offer, because EA0 specifically targets 128-byte buffers. To implement compression/decompression, we introduce , which provides a simple bit queue interface. We refactor arch/arm64/mm/mteswap.c to support both the compressed (CONFIG_ARM64_MTE_COMP) and non-compressed case. For the former, in addition to tag compression, we move tag allocation from kmalloc() to separate kmem caches, providing greater locality and relaxing the alignment requirements. Alexander Potapenko (5): linux/bitqueue.h: add a KUnit test for bitqueue.h arm64: mte: implement CONFIG_ARM64_MTE_COMP arm64: mte: add a test for MTE tags compression arm64: mte: add compression support to mteswap.c fixup mteswap arch/arm64/Kconfig | 20 ++ arch/arm64/include/asm/mtecomp.h | 60 +++++ arch/arm64/mm/Makefile | 7 + arch/arm64/mm/mtecomp.c | 398 +++++++++++++++++++++++++++++++ arch/arm64/mm/mteswap.c | 19 +- arch/arm64/mm/mteswap.h | 12 + arch/arm64/mm/mteswap_comp.c | 50 ++++ arch/arm64/mm/mteswap_nocomp.c | 37 +++ arch/arm64/mm/test_mtecomp.c | 175 ++++++++++++++ lib/Kconfig.debug | 8 + lib/Makefile | 1 + lib/test_bitqueue.c | 244 +++++++++++++++++++ 12 files changed, 1020 insertions(+), 11 deletions(-) create mode 100644 arch/arm64/include/asm/mtecomp.h create mode 100644 arch/arm64/mm/mtecomp.c create mode 100644 arch/arm64/mm/mteswap.h create mode 100644 arch/arm64/mm/mteswap_comp.c create mode 100644 arch/arm64/mm/mteswap_nocomp.c create mode 100644 arch/arm64/mm/test_mtecomp.c create mode 100644 lib/test_bitqueue.c