From patchwork Wed Jan 31 15:59:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandre Ghiti X-Patchwork-Id: 13539569 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA998C47258 for ; Wed, 31 Jan 2024 15:59:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2642F6B0072; Wed, 31 Jan 2024 10:59:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 214026B007B; Wed, 31 Jan 2024 10:59:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B53E6B0087; Wed, 31 Jan 2024 10:59:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id EC8306B0072 for ; Wed, 31 Jan 2024 10:59:36 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 6694E120C50 for ; Wed, 31 Jan 2024 15:59:36 +0000 (UTC) X-FDA: 81740066352.23.8C1626C Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) by imf03.hostedemail.com (Postfix) with ESMTP id 69DEE20017 for ; Wed, 31 Jan 2024 15:59:34 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b="k/fGeVq3"; spf=pass (imf03.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.54 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706716774; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=SJEZ1qPaiDldAW8C05rM2rqR0hTzPI1igVobAmalZsY=; b=AYEcNdcqRWDCVV9d69nanCvSv8WnQqs5sny9T7lX34a4JVwKc+IJOu0GrEJZS2KY3FPIj4 xVKRxJNfpxSxTozp1juJIMjQ4bkf0bgQ+Lg1TqgSGZJkaxzMMc1xLmhEi7KnIrxIdd1JJr /FuQ+IoiIEPnbzFD4t48U5o4PUbZCvs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706716774; a=rsa-sha256; cv=none; b=FN+wB/glF4grGDWpMCtktpR6F7JO/RGB3LH/9m4bzBA1ujCufjVSBixS1Nx6Ai03eBhoox Fr2J783HZXwljYIaRMdeVLe+M56bLT1Qp2uzmSITJAg/sohpy4Y2M5q2oF5LCPQvk5+R/r kurHH/6FhCRB/MK88cFSisn5GpVJHZU= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b="k/fGeVq3"; spf=pass (imf03.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.128.54 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-40e80046264so60596395e9.0 for ; Wed, 31 Jan 2024 07:59:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1706716773; x=1707321573; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=SJEZ1qPaiDldAW8C05rM2rqR0hTzPI1igVobAmalZsY=; b=k/fGeVq3FPW4bisnZR8fIehsSd+YhthLVJ4j89+XXCOgTiRS/DB/X5ByD5K8GSUKOm Er8+EiHsfq+fJIWMCNxTdPzNXJfFquzqTfvsFho97LoKo8cG01gGGCuT0BYZ17BcLPo/ K+zw1hlHytKEGpdXK3VdMXGjL5LcISxKm8lgELZBxJwRY3sC7VWa9ofAfjzGY+qXa5AL jRwZ7tFU0lcAoj88As2F1HgdnJmFztNHNOeK4nZDcmSYQSBbrv5x96OFfrHFdcb+96qK o51f9smlCE0VisbXjcPmZpZJyoBG7NkspRPjQN5Bz89hABVj/b1rLGwFtW1ZqZF4CTDx VwLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706716773; x=1707321573; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SJEZ1qPaiDldAW8C05rM2rqR0hTzPI1igVobAmalZsY=; b=inDk5DsmouwqVCxUHItBA6UGUCZ4dwsabHs4kZCUDn7fIjcavy5AB4puohYLAhZ7Ee kraSNAh0heoc5N6tBvLg5gbl+4Ymk/TkpvqxC9wVwDwRDl5/1R5MLuVQIEMZjC3bgqKn AiKQGGRTw+vPfYYgOljJmrGi6IxbXl+Wzbh9VhdaGFKwixIw4PKTXHLAknp5E1kmELrm LZQuz7mXyqcus8UGle4HuHL+Pmy8rG6g/78bor8xPyn0qmr2WUMwXx68VfMNMLGzAEe1 aWIXisC9sVIV09y7Dnz/OFrZl9VRonXoMLqR2pgXpc8NWTGdOQLUTgVP8IrMlScf7NRE uKjA== X-Gm-Message-State: AOJu0YxVO4/HrZqJB2efpoyRD44mBf/58qA6vqQDZO9KzX078r1kyXkC esYjaTeBdTWetWNAqYTXUJUVApxufKoAVxFeAFCu2pgAQ59ULrILYUl9DBM3Slc= X-Google-Smtp-Source: AGHT+IE0y4HMNlJGrQQF1/vBmusVf+vGvj5kGeOQLs8zgDMsoQ9mThbIK2SfMvCJNZS4ESF4P8sD+g== X-Received: by 2002:a05:600c:468b:b0:40e:e834:3d86 with SMTP id p11-20020a05600c468b00b0040ee8343d86mr1590072wmo.37.1706716772821; Wed, 31 Jan 2024 07:59:32 -0800 (PST) X-Forwarded-Encrypted: i=0; AJvYcCU5toVRodwweHTcQIyYJFP2Sf3z6pJe8uOj3kJL8Bf9n+o3G8oH7qCtVvtBB6ZzANPh3d2Znisr/JVcpjb5tj+mnQOtD0bGrsmT4CRsS6KdtKyxWKOCl38Giy/mwdkpg1+QQxv40odbMZlnkY5SYdo2XLk0XdYDTp7WRoTnVpNlAlYCWWsQHxNfcLCDVAWDcXk3GvVdHhPFZ62Gx3ql6nSu6/WKzbRDqzGUIYgkgqpdB1vq50EO2ieKr+XUNsum2CV+pMMxiPj9TkdU06yVd81ZbG9I+ff7kAkhj9IowwAQm+N5BwHjhUoaeye58UCta73W6pRJRoGC3DvGN4oHjbsgUSKWyszDmS4NdxPTBbwp2RdU2Ks6e9clrZ+UECY6zRr/NAd2cDDBxGCLVOmgvb+jVeV5smy1Xly69fTl7+hY8866CxoOjFO1I48MgVF0UdOI5C1yuLuymZyybFPw6nm1nAcMitNsxXW/PlXLReWUH5BFV6Jriyq2AKNFsNYuHKk6srdfiXlJ0B77YIP+V0E/i01m7OxC/pDhPX+9RSpIvCQkc7ZVpcu7PMqJqsCDBi7cS4PdMDvYQwktSWOvXekDuYCwO0eKbSeWmguapyXy15tDZbbtoRH2ScNNIwiU21PKEtXagxXWu+FGng== Received: from alex-rivos.ba.rivosinc.com (amontpellier-656-1-456-62.w92-145.abo.wanadoo.fr. [92.145.124.62]) by smtp.gmail.com with ESMTPSA id t24-20020a1c7718000000b0040f0219c371sm1926678wmi.19.2024.01.31.07.59.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 31 Jan 2024 07:59:32 -0800 (PST) From: Alexandre Ghiti To: Catalin Marinas , Will Deacon , Thomas Bogendoerfer , Michael Ellerman , Nicholas Piggin , Christophe Leroy , Paul Walmsley , Palmer Dabbelt , Albert Ou , Andrew Morton , Ved Shanbhogue , Matt Evans , Dylan Jhong , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-mm@kvack.org Cc: Alexandre Ghiti Subject: [PATCH RFC v2 0/4] Svvptc extension to remove preventive sfence.vma Date: Wed, 31 Jan 2024 16:59:25 +0100 Message-Id: <20240131155929.169961-1-alexghiti@rivosinc.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 X-Rspamd-Queue-Id: 69DEE20017 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: iadr81ja6ogi4bc6yjf7uy6s7uijii4u X-HE-Tag: 1706716774-707931 X-HE-Meta: U2FsdGVkX1+TynyrEXEBcnN37dzBgFtlh1XBtq2Idd2bt1MT1u+/8k/30kQqsmWHpZ7yaLicyaEQYq3y+SzQQjOtTfbzIvWsF7bFbPiS7wehVDp/kZbbxfATkOBAZcs2FO74KPX6K/wPA18Mf4lCDiyib0nBZ8Z8BLoMcFPmG7+Ruj1YAqJngs6CLRFEbHsc5pCKvJOlNxS5B8jprahspFFwXMXrR2OwoGB2mWiXNIBZ+s27yVSxZFS/XY43SCyJzAs1VuxdsPBkX0LaUSQlhYQT2k5ocCF8D0smx+UXS34c+lfPcgTvJNhG1dkeLI45UvXZxL2SnmqxQ4TopJ/+JHrtKR/ci5NmWfpdjoQwObxqD0fHguSciY2m4rLCodrR0wIzevHQyyF78TemYkr3LV5xCEK8WyQFtr6og0ODvhbVkMevSoRuO59yvKyLH8yYKCuoSLTqMeUHsGCdnV+dVDDr3Rm6jihT0OjaGOcYH0kA6Be4wQF61TG8wZbuLNfHbMR26djIhLlVBKi6Ycn+iZRaaFN6/S8zrDReGear/zRP9je9saureLXoFG9EriO9ggI3fimKDRvN4Kz1hZQgIEt3+kdtOyCQlyqP0vMcCCDKNZpI2rshyEEN3OZk2Ti+ezB8rbBMSl0fs7t+QPlP+wG50fIt3RCws57m8RYzvAbSWGhpcQeDgLYtqpqT7ePPZgsCHgzlI7Q9b1SGlZxXuz7NABDUbjeBFmtaPvADzKPYjBApSIzd3AHzsYiO3UW9rccfpALRlFgBk6XFA7pi6pmuxyXxqslup0jRk06ei1YEcFLp4LOp2yzwPlKvW5nExpEcw8zuz5IfwrAEdVYpkS06qO8WXCWkap+/39mwtwZa6N3r5yJaTPs9X4xO7c2oeZpM3o7K6bH4OyEiDZ10QsrUnm47hF8li5aYn7nYKZNPOQnUNd7IR2bXJQ7ef5SP220R4xKssBrBPZw8gei 8mQjC528 t2T7S5gMJWzwmrSwHjYiVAklGP0rTg/tZo+oRLGMUH53dFt+BtW5OcE4gauPTPhOfKN4KlI9/yt2sGZ0HzjBmdrt+eYqdCNzonDP18nympfLV/zRq7MuKG86DflH3xQ/6ZRLDrh3U7Porw6n4A2+6bzEUP4fO4elwlx6sPVVvp9bfI9athbRrzG9j/RNWkE7Z+5caJtFztFrLoduncuwKL+OtqygW7yo1ky+n2FmFOBqy9pr8gJmfIhaSOP+oknSIKfU434enJ4yXJZ2OWWtTImC0Plf2nAXqtcP5wAN3lAa97UgopJjMXNd6V+Ba/XSiTSjCJrAcd014ie2KVacE+GMCag7+OnFCG8umvHh6dYuy65azMrphuSYg8oYiv/IZTJ6+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In RISC-V, after a new mapping is established, a sfence.vma needs to be emitted for different reasons: - if the uarch caches invalid entries, we need to invalidate it otherwise we would trap on this invalid entry, - if the uarch does not cache invalid entries, a reordered access could fail to see the new mapping and then trap (sfence.vma acts as a fence). We can actually avoid emitting those (mostly) useless and costly sfence.vma by handling the traps instead: - for new kernel mappings: only vmalloc mappings need to be taken care of, other new mapping are rare and already emit the required sfence.vma if needed. That must be achieved very early in the exception path as explained in patch 3, and this also fixes our fragile way of dealing with vmalloc faults. - for new user mappings: Svvptc makes update_mmu_cache() a no-op and no traps can happen since xRET instructions now act as fences. Patch 1 and 2 introduce Svvptc extension probing. It's still an RFC because Svvptc is not ratified yet. On our uarch that does not cache invalid entries and a 6.5 kernel, the gains are measurable: * Kernel boot: 6% * ltp - mmapstress01: 8% * lmbench - lat_pagefault: 20% * lmbench - lat_mmap: 5% Thanks to Ved and Matt Evans for triggering the discussion that led to this patchset! Any feedback, test or relevant benchmark are welcome :) Changes in v2: - Rebase on top of 6.8-rc1 - Remove patch with runtime detection of tlb caching and debugfs patch - Add patch that probes Svvptc - Add patch that defines the new Svvptc dt-binding - Leave the behaviour as-is for uarchs that cache invalid TLB entries since I don't have any good perf numbers - Address comments from Christoph on v1 - Fix a race condition in new_vmalloc update: ld a2, 0(a0) <= this could load something which is != -1 not a1, a1 <= here or in the instruction after, flush_cache_vmap() could make the whole bitmap to 1 and a1, a2, a1 sd a1, 0(a0) <= here we would clear bits that should not be cleared! Instead, replace the whole sequence with: amoxor.w a0, a1, (a0) Alexandre Ghiti (4): riscv: Add ISA extension parsing for Svvptc dt-bindings: riscv: Add Svvptc ISA extension description riscv: Stop emitting preventive sfence.vma for new vmalloc mappings riscv: Stop emitting preventive sfence.vma for new userspace mappings with Svvptc .../devicetree/bindings/riscv/extensions.yaml | 7 ++ arch/riscv/include/asm/cacheflush.h | 18 +++- arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/include/asm/pgtable.h | 16 +++- arch/riscv/include/asm/thread_info.h | 5 ++ arch/riscv/kernel/asm-offsets.c | 5 ++ arch/riscv/kernel/cpufeature.c | 1 + arch/riscv/kernel/entry.S | 84 +++++++++++++++++++ arch/riscv/mm/init.c | 2 + arch/riscv/mm/pgtable.c | 13 +++ 10 files changed, 150 insertions(+), 2 deletions(-)