From patchwork Thu Sep 3 09:17:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11752971 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E83AB618 for ; Thu, 3 Sep 2020 09:19:28 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B3AB6206C0 for ; Thu, 3 Sep 2020 09:19:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="KzuzGjd/"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="VDPMh54U" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B3AB6206C0 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=c3goHfxSbAh0mu0mQpCnUNiXBPtty/NnZBFqz/LIT9c=; b=KzuzGjd/AFFjv1bWa5O+2C84g uAqXixYr3y06cSs8wHE6M10WwC4xmC+Nm1buIKIBOjFwUz+rMkxqn4Jn5MdB4kwdEjwiYXNqMswAi DAnCsNv60PNoF0fwItJTPBOpkYtd/d0Ymd1GI7shxl3xVmsgz8Sh5b0SoaVFP5xgF+iFZcWrT++Nc 6YnM1A9K/6CaKMYIOs4ppBDdMKeDBzRnfv/3QH2ZQ6as1TZRSZJtOQBNgN/+w8bAmyLeifPq5M/Vt h+dCdHn3c1ufhIdjFq18JddYqID7AO3ZKjeIfYvk2v5Ij+FRfDbhutUFnG666akFZiOGX4MbvmZvT aIywSJrzQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNG-00041U-DJ; Thu, 03 Sep 2020 09:17:42 +0000 Received: from mail-ed1-x532.google.com ([2a00:1450:4864:20::532]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNC-00040U-Jo for linux-arm-kernel@lists.infradead.org; Thu, 03 Sep 2020 09:17:39 +0000 Received: by mail-ed1-x532.google.com with SMTP id i1so1789033edv.2 for ; Thu, 03 Sep 2020 02:17:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ks8zbpt3K53ylw0T/eWvGEXanQshe4hPX9kTva9RnTk=; b=VDPMh54UU9w25z4QHVBYJjPLF92rQvlDEmZh2WtnxD3Bzdp7sogs2sroU5P8Ur3FAw dv6ULAm2SlhYlvBSIAuPvDAxIDfiPm+Xl5qQW6TODWahjjtzYxh3+twFOVluLHMIgxz6 w2Eh0wttggdlvWHo3aw1Rmp6MBbEUcdHEpVWstWJJH51qV9CchXfrCoAG4KXmvAEDlob 46JIR8MQRfQ8Jfm3o1xNopn3QfNSGesPEGM2pKxNzZMTPmzwXW7dEhiRXFHJcAydDEnK dOqVw2ar8bKWyJY1V9CQN8WlsrUu+0XVUERIfK37nCxqpwsTFFpOaGTWPpcwdI2fGHIi AaQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ks8zbpt3K53ylw0T/eWvGEXanQshe4hPX9kTva9RnTk=; b=PQgjl0253LDvcaB7rAmBT8oHdFQ91xjkgZqonKx/tj1KqQFIv+utfQeGIjwYnEg0Ge F8XWunjYC7a0FD4juHR2QztFz5SwwFr2wr0PRt/01m8HVph2aVrvsS2dOk6/+YCxiJuF oGqH+6tra7jeVFpqi8ijeP9CCyX/dKMKwsPMNkqBXPRttZCod8pJDO/if5QpLEHdJYN7 bs53VdTty5O8vMXei4XOcrOwuH2G3dGKfu+HLDnT5rp5dW4d667Ve5n7GF2wCpSAJlgV bwzyKHXFxBOj0cxhvFLAGojdu9+tNoSjgwwavX0TJr133KJZIwtfldBVabpqT9eoFpQO nByA== X-Gm-Message-State: AOAM530noToehlNN9SgSj7K5VPPZGg+inBlXZ/L/Dr4LEpHFVwTHBM8T dqjR07vr8CPjsLAROQ1jDJjwPg== X-Google-Smtp-Source: ABdhPJxWMzIchcIkSdxyCs4vGPUMhXltKg67YBEhN8oWRx/8t3VGb7k28L3pS6EUEDV05f+huINp0Q== X-Received: by 2002:a50:f69a:: with SMTP id d26mr1970262edn.21.1599124657089; Thu, 03 Sep 2020 02:17:37 -0700 (PDT) Received: from localhost (dynamic-2a00-1028-919a-a06e-64ac-0036-822c-68d3.ipv6.broadband.iol.cz. [2a00:1028:919a:a06e:64ac:36:822c:68d3]) by smtp.gmail.com with ESMTPSA id ot19sm2547529ejb.121.2020.09.03.02.17.35 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 03 Sep 2020 02:17:36 -0700 (PDT) From: David Brazdil To: Marc Zyngier , Catalin Marinas , Will Deacon , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH v2 01/10] Macros to override naming of percpu symbols and sections Date: Thu, 3 Sep 2020 11:17:03 +0200 Message-Id: <20200903091712.46456-2-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200903091712.46456-1-dbrazdil@google.com> References: <20200903091712.46456-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200903_051738_670114_6666EFEB X-CRM114-Status: GOOD ( 16.92 ) X-Spam-Score: -16.2 (----------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-16.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:532 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.5 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, James Morse , Julien Thierry , David Brazdil , kernel-team@android.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Modify generic linker script macros to generate section/symbol names for percpu area using overridable macros. No functional changes. This will allow arm64 linker script to define a second KVM-specific percpu data section using the generic PERCPU_SECTION macro. Signed-off-by: David Brazdil --- include/asm-generic/vmlinux.lds.h | 40 +++++++++++++++++++++---------- 1 file changed, 27 insertions(+), 13 deletions(-) diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index 5430febd34be..8f3f5c45e891 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -920,6 +920,20 @@ #define INIT_RAM_FS #endif +/* + * Macros to override the naming of percpu symbols and sections. + * Used by arm64 linker script to define a separate percpu area for KVM. + */ +#define PERCPU_SECTION_BASE_NAME .data..percpu + +#ifndef PERCPU_SECTION_NAME +#define PERCPU_SECTION_NAME(suffix) PERCPU_SECTION_BASE_NAME ## suffix +#endif + +#ifndef PERCPU_SYMBOL_NAME +#define PERCPU_SYMBOL_NAME(name) name +#endif + /* * Memory encryption operates on a page basis. Since we need to clear * the memory encryption mask for this section, it needs to be aligned @@ -931,7 +945,7 @@ #ifdef CONFIG_AMD_MEM_ENCRYPT #define PERCPU_DECRYPTED_SECTION \ . = ALIGN(PAGE_SIZE); \ - *(.data..percpu..decrypted) \ + *(PERCPU_SECTION_NAME(..decrypted)) \ . = ALIGN(PAGE_SIZE); #else #define PERCPU_DECRYPTED_SECTION @@ -975,17 +989,17 @@ * sharing between subsections for different purposes. */ #define PERCPU_INPUT(cacheline) \ - __per_cpu_start = .; \ - *(.data..percpu..first) \ + PERCPU_SYMBOL_NAME(__per_cpu_start) = .; \ + *(PERCPU_SECTION_NAME(..first)) \ . = ALIGN(PAGE_SIZE); \ - *(.data..percpu..page_aligned) \ + *(PERCPU_SECTION_NAME(..page_aligned)) \ . = ALIGN(cacheline); \ - *(.data..percpu..read_mostly) \ + *(PERCPU_SECTION_NAME(..read_mostly)) \ . = ALIGN(cacheline); \ - *(.data..percpu) \ - *(.data..percpu..shared_aligned) \ + *(PERCPU_SECTION_NAME()) \ + *(PERCPU_SECTION_NAME(..shared_aligned)) \ PERCPU_DECRYPTED_SECTION \ - __per_cpu_end = .; + PERCPU_SYMBOL_NAME(__per_cpu_end) = .; /** * PERCPU_VADDR - define output section for percpu area @@ -1012,11 +1026,11 @@ * address, use PERCPU_SECTION. */ #define PERCPU_VADDR(cacheline, vaddr, phdr) \ - __per_cpu_load = .; \ - .data..percpu vaddr : AT(__per_cpu_load - LOAD_OFFSET) { \ + PERCPU_SYMBOL_NAME(__per_cpu_load) = .; \ + PERCPU_SECTION_NAME() vaddr : AT(PERCPU_SYMBOL_NAME(__per_cpu_load) - LOAD_OFFSET) { \ PERCPU_INPUT(cacheline) \ } phdr \ - . = __per_cpu_load + SIZEOF(.data..percpu); + . = PERCPU_SYMBOL_NAME(__per_cpu_load) + SIZEOF(PERCPU_SECTION_NAME()); /** * PERCPU_SECTION - define output section for percpu area, simple version @@ -1032,8 +1046,8 @@ */ #define PERCPU_SECTION(cacheline) \ . = ALIGN(PAGE_SIZE); \ - .data..percpu : AT(ADDR(.data..percpu) - LOAD_OFFSET) { \ - __per_cpu_load = .; \ + PERCPU_SECTION_NAME() : AT(ADDR(PERCPU_SECTION_NAME()) - LOAD_OFFSET) { \ + PERCPU_SYMBOL_NAME(__per_cpu_load) = .; \ PERCPU_INPUT(cacheline) \ } From patchwork Thu Sep 3 09:17:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11752957 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1564A618 for ; Thu, 3 Sep 2020 09:17:59 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B92CC2071B for ; Thu, 3 Sep 2020 09:17:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="iNFj+sZN"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="bKVLGaca" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B92CC2071B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ze8lf11bYK36W0kCd+RGfoFjBvvW9DNmHSkvkkcQQJ0=; b=iNFj+sZN5FZyl1+Z6dGKkEmuO 8ZLzYnjHUD6P7TQX8Ja3dPUiUtU3aUIkFbdJ4NGn/h4MmQ0+0EIwIbkbaXg+BoXpS3hKKsPKRgiiJ R30SN951f4bX6lFWv9Yjcmx6ddjtjOKfyC4bGlZJtUb8CtE+yUWPkijeMULveW18Jgsb7XLj2jykF Wu/I4KXlK8rjXF/65oBN0L+xa2W+hZnnXKz3JurfOkaS1vFTGIl7C/+Hct89HOgm7/tUfgNv+n5D6 BqJKYLl+ds0E6hooKoCUGISfAn0feHPouho6jVGyMCOyeoWRhQMHysxlKNnWY84OCLuwwg2IJabAd 3vs+ZLJtw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNK-00043X-6p; Thu, 03 Sep 2020 09:17:46 +0000 Received: from mail-ej1-x642.google.com ([2a00:1450:4864:20::642]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNF-000418-CU for linux-arm-kernel@lists.infradead.org; Thu, 03 Sep 2020 09:17:42 +0000 Received: by mail-ej1-x642.google.com with SMTP id nw23so2840647ejb.4 for ; Thu, 03 Sep 2020 02:17:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zNvDbfBdNwTHuqwmJ87zZAgkBHe207JszB0bJn56FTQ=; b=bKVLGacayIn54nHdLPKqW3i8uEBaZAwki24ZWWmC6StEw8/dwhS84EJljmlUfqPDzN os8Cz89rtKYRMOQozzZ6QHFUcakWPYFWt85KojPT2HEAJ5vAzQvf8RVMU4xH/NRXfvG1 3bdX7bTf8sgCIGmRYkuDw3RUrax3VG60ypb2xKs8gIochB3Zkg1ZfP4oG2oGGUEX4ZtS S1DaMW7NxbxZ4GQrSBQL+3bYpDlVEz1Xcf2zT3DD55F6UuPfyc7VljV1HOWLMd6nT1bg bYSJOwJm6ojHGXX5B8rzmyp0EZhWXLVr9dl6F6VT7JHKmVgtrZYrBcaNOk77rMkhzTW/ C3Qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zNvDbfBdNwTHuqwmJ87zZAgkBHe207JszB0bJn56FTQ=; b=NVtXawxJ1CMoJhMGH7uVRPe8MhMLvRR8CrE8+xfJrzOCjyd7JBCKL+t7e9OYRJffsP 2OARoSjT6fIM1wrTgm9sWryKJnQ+xG/I/lAGwv4X9Pu6cDcSFpf3KttG+vZ8hzWlIuWV rmq5kqHTE6XOQ3ivPlDJXlFXZ9XILVXDhoYliTSEE48tU+k66HQQhy7S5QNc9wjQpNQ4 4EyjwUfxfsrrndPEtvQ9Hc4NzabuiymhiuCPfU9wLtDmjoycYfvuEgDPKk4yP8uxQ7qW SOgqa8LJIH+v7N9atmSXc4WPOfWdr2tia12sTZ/lvjc/m/1wIAZB5rQzAQxYJgd2PXrY NJKQ== X-Gm-Message-State: AOAM533WumJ1keC03Uu6KHvcfpzNo2TQpFKJdAatXiUwHCri+z17MPvr L/hU4pEgtx1FE++iDrvK60V3rg== X-Google-Smtp-Source: ABdhPJxjc2flZ0tLE+wu0DM0G8WaozAnEsLB5cW424g4zHnmjkJ4t03cn83T3cbh6pDJJ5h5XXDJhQ== X-Received: by 2002:a17:906:3955:: with SMTP id g21mr1170038eje.69.1599124659856; Thu, 03 Sep 2020 02:17:39 -0700 (PDT) Received: from localhost (dynamic-2a00-1028-919a-a06e-64ac-0036-822c-68d3.ipv6.broadband.iol.cz. [2a00:1028:919a:a06e:64ac:36:822c:68d3]) by smtp.gmail.com with ESMTPSA id g19sm2670317ejz.5.2020.09.03.02.17.38 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 03 Sep 2020 02:17:38 -0700 (PDT) From: David Brazdil To: Marc Zyngier , Catalin Marinas , Will Deacon , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH v2 02/10] kvm: arm64: Partially link nVHE hyp code, simplify HYPCOPY Date: Thu, 3 Sep 2020 11:17:04 +0200 Message-Id: <20200903091712.46456-3-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200903091712.46456-1-dbrazdil@google.com> References: <20200903091712.46456-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200903_051741_463685_FBBA6387 X-CRM114-Status: GOOD ( 22.03 ) X-Spam-Score: -16.2 (----------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-16.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:642 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.5 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, James Morse , Julien Thierry , David Brazdil , kernel-team@android.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Previous series introduced custom build rules for nVHE hyp code, using objcopy to prefix ELF section and symbol names to separate nVHE code into its own "namespace". This approach was limited by the expressiveness of objcopy's command line interface, eg. missing support for wildcards. Improve the build rules by partially linking all '.hyp.o' files and prefixing their ELF section names using a linker script. Continue using objcopy for prefixing ELF symbol names. One immediate advantage of this approach is that all subsections matching a pattern can be merged into a single prefixed section, eg. .text and .text.* can be linked into a single '.hyp.text'. This removes the need for -fno-reorder-functions on GCC and will be useful in the future too: LTO builds use .text subsections, compilers routinely generate .rodata subsections, etc. Partially linking all hyp code into a single object file also makes it easier to analyze. Signed-off-by: David Brazdil --- arch/arm64/kvm/hyp/nvhe/Makefile | 56 ++++++++++++++++--------------- arch/arm64/kvm/hyp/nvhe/hyp.lds.S | 14 ++++++++ 2 files changed, 43 insertions(+), 27 deletions(-) create mode 100644 arch/arm64/kvm/hyp/nvhe/hyp.lds.S diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index aef76487edc2..1b2fbb19f3e8 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -10,40 +10,42 @@ obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o -obj-y := $(patsubst %.o,%.hyp.o,$(obj-y)) -extra-y := $(patsubst %.hyp.o,%.hyp.tmp.o,$(obj-y)) +## +## Build rules for compiling nVHE hyp code +## Output of this folder is `hyp.o`, a partially linked object file containing +## all nVHE hyp code and data. +## -$(obj)/%.hyp.tmp.o: $(src)/%.c FORCE +hyp-obj := $(patsubst %.o,%.hyp.o,$(obj-y)) +obj-y := hyp.o +extra-y := $(hyp-obj) hyp.tmp.o hyp.lds + +# 1) Compile all source files to `.hyp.o` object files. The file extension +# avoids file name clashes for files shared with VHE. +$(obj)/%.hyp.o: $(src)/%.c FORCE $(call if_changed_rule,cc_o_c) -$(obj)/%.hyp.tmp.o: $(src)/%.S FORCE +$(obj)/%.hyp.o: $(src)/%.S FORCE $(call if_changed_rule,as_o_S) -$(obj)/%.hyp.o: $(obj)/%.hyp.tmp.o FORCE - $(call if_changed,hypcopy) -# Disable reordering functions by GCC (enabled at -O2). -# This pass puts functions into '.text.*' sections to aid the linker -# in optimizing ELF layout. See HYPCOPY comment below for more info. -ccflags-y += $(call cc-option,-fno-reorder-functions) +# 2) Compile linker script. +$(obj)/hyp.lds: $(src)/hyp.lds.S FORCE + $(call if_changed_dep,cpp_lds_S) + +# 3) Partially link all '.hyp.o' files and apply the linker script. +# Prefixes names of ELF sections with '.hyp', eg. '.hyp.text'. +LDFLAGS_hyp.tmp.o := -r -T $(obj)/hyp.lds +$(obj)/hyp.tmp.o: $(addprefix $(obj)/,$(hyp-obj)) $(obj)/hyp.lds FORCE + $(call if_changed,ld) + +# 4) Produce the final 'hyp.o', ready to be linked into 'vmlinux'. +# Prefixes names of ELF symbols with '__kvm_nvhe_'. +$(obj)/hyp.o: $(obj)/hyp.tmp.o FORCE + $(call if_changed,hypcopy) # The HYPCOPY command uses `objcopy` to prefix all ELF symbol names -# and relevant ELF section names to avoid clashes with VHE code/data. -# -# Hyp code is assumed to be in the '.text' section of the input object -# files (with the exception of specialized sections such as -# '.hyp.idmap.text'). This assumption may be broken by a compiler that -# divides code into sections like '.text.unlikely' so as to optimize -# ELF layout. HYPCOPY checks that no such sections exist in the input -# using `objdump`, otherwise they would be linked together with other -# kernel code and not memory-mapped correctly at runtime. +# to avoid clashes with VHE code/data. quiet_cmd_hypcopy = HYPCOPY $@ - cmd_hypcopy = \ - if $(OBJDUMP) -h $< | grep -F '.text.'; then \ - echo "$@: function reordering not supported in nVHE hyp code" >&2; \ - /bin/false; \ - fi; \ - $(OBJCOPY) --prefix-symbols=__kvm_nvhe_ \ - --rename-section=.text=.hyp.text \ - $< $@ + cmd_hypcopy = $(OBJCOPY) --prefix-symbols=__kvm_nvhe_ $< $@ # Remove ftrace and Shadow Call Stack CFLAGS. # This is equivalent to the 'notrace' and '__noscs' annotations. diff --git a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S new file mode 100644 index 000000000000..aaa0ce133a32 --- /dev/null +++ b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Linker script used during partial linking of nVHE EL2 object files. + * Written by David Brazdil + */ + +/* + * Defines an ELF hyp section from input section @NAME and its subsections. + */ +#define HYP_SECTION(NAME) .hyp##NAME : { *(NAME NAME##.[0-9a-zA-Z_]*) } + +SECTIONS { + HYP_SECTION(.text) +} From patchwork Thu Sep 3 09:17:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11752959 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2895214E5 for ; Thu, 3 Sep 2020 09:18:11 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D9697206C0 for ; Thu, 3 Sep 2020 09:18:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="bgXRbpql"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="HRlIL51l" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D9697206C0 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=iFZ6XzWmM59T9lA6ywmQlqQ9tMKOaFB2KfCG55nWVuI=; b=bgXRbpqlnHg8wewy97M2Bte2y cRMBMvLY8S2fIU8AcQHVfkX+NOsD0z17rFQmqR+M1jKzn5eSO8GYSal4SCJ22FSuRjdV99VHKfHwB VpwEK7xJ06jjTu+atf4GeD44T0JS2vOXfBPHtJu4ZsA1KaaI4+Ff9RY3Pc4zDerbEQibCXmUIntm+ /zu0C+MsLBW7Z4kAvCkzGp2orgMuFzq3fRHGXhalwFe/JWw2UjVi56U0PSEU/GXv3JVdxwXA0Pfw3 IEi6VHBCGXM/PitPF6qrAZ/FlTlXjKJmFHn7jvp24ivkb2q1FGy4RuObzUaI4eoS5Vd+x3NGYppUu HjhA0SvTQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNT-00048m-Kf; Thu, 03 Sep 2020 09:17:55 +0000 Received: from mail-ej1-x641.google.com ([2a00:1450:4864:20::641]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNH-00041q-Rf for linux-arm-kernel@lists.infradead.org; Thu, 03 Sep 2020 09:17:50 +0000 Received: by mail-ej1-x641.google.com with SMTP id gr14so1707512ejb.1 for ; Thu, 03 Sep 2020 02:17:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SrGd93fxkwD2pyEKRstXopHn1z8dcE4gBzaKU7CKZCA=; b=HRlIL51l59ebWCHj5J3CT4xkM9IlN8rKbGmg8e0Y9n86Lri/I1YlfU6fXP/zgUmqw3 nI8nRSnDI3ygU5E/oBVeCszNwIutKvZ6t1sxteGwQ0KPwzLsDX4JM3hYqiP33Bwr+Gyr bkvINlzb0gkhs/MK+1+UNMhwQsNntGM/ivuFzAxzp1qwt3i+pA3M+i4kc706jN0aad/D eXhAux7wYJSt0YamlmnEbFIguBmpR0upVjXVL8Bvqs/5w/5+ugq1VJ/GICOHT1KU57/p P2GzWRMcQGTSN20ojg2OTkQK1ruNM2qz+o4lFnMGtyVfVJVFe6jTjoKi8EEKKbTBo0LJ g3KA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SrGd93fxkwD2pyEKRstXopHn1z8dcE4gBzaKU7CKZCA=; b=jr0KH9q7H0HhoVHpgX2FA0tArgNhODi1nDGTCHrQfHcqy1YGsOLq7kfqzpbfqQNrOC WiHeuzZAjJIz+ZoQ4u5S5IZmifzx73fDzcchbw/2xvEs3Wy2WOyrjj4AqWkRiGGDOnLu Ia/KN2zyk5ESwPjFc/GQfESPU90irLmGMKh5q1ZhQqbFP6VG5n2nI6VSK+y0HfmD+7Qz YDLuI0wWcla3Yh/oJXKYWgJEgMOoqbrQBO6oBtJoykQIlE1HpMV9e3A5+51kZkAI/VVd FilNW4BoKUr84H9ZppfneH5OOa7b4Omf93ahzWCCwH7brNFzcgluR+uAOePTDteCeBtF zXVw== X-Gm-Message-State: AOAM531KD2C9AZevLdxPZg9zvkKHzXqJfJmD24D4VCXywl4o4vBB57jg ZyBnzu+/gs1BOyJLZNlotaFc9Q== X-Google-Smtp-Source: ABdhPJzHjBZW2yUpqLooMNfefLtRAJ+xiOCyrfqzLjU8r36iKi/k8YqA2eAD2V40RD+Fun2kL9iNRA== X-Received: by 2002:a17:907:2168:: with SMTP id rl8mr1136229ejb.308.1599124662488; Thu, 03 Sep 2020 02:17:42 -0700 (PDT) Received: from localhost (dynamic-2a00-1028-919a-a06e-64ac-0036-822c-68d3.ipv6.broadband.iol.cz. [2a00:1028:919a:a06e:64ac:36:822c:68d3]) by smtp.gmail.com with ESMTPSA id bq24sm2672780ejb.27.2020.09.03.02.17.40 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 03 Sep 2020 02:17:41 -0700 (PDT) From: David Brazdil To: Marc Zyngier , Catalin Marinas , Will Deacon , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH v2 03/10] kvm: arm64: Remove __hyp_this_cpu_read Date: Thu, 3 Sep 2020 11:17:05 +0200 Message-Id: <20200903091712.46456-4-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200903091712.46456-1-dbrazdil@google.com> References: <20200903091712.46456-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200903_051743_927282_8F4A6505 X-CRM114-Status: GOOD ( 21.31 ) X-Spam-Score: -16.2 (----------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-16.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:641 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.5 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, James Morse , Julien Thierry , David Brazdil , kernel-team@android.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org this_cpu_ptr is meant for use in kernel proper because it selects between TPIDR_EL1/2 based on nVHE/VHE. __hyp_this_cpu_ptr was used in hyp to always select TPIDR_EL2. Unify all users behind this_cpu_ptr and friends by selecting _EL2 register under __KVM_NVHE_HYPERVISOR__. Under CONFIG_DEBUG_PREEMPT, the kernel helpers perform a preemption check which is omitted by the hyp helpers. Preserve the behavior for nVHE by overriding the corresponding macros under __KVM_NVHE_HYPERVISOR__. Extend the checks into VHE hyp code. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_asm.h | 20 -------------- arch/arm64/include/asm/percpu.h | 33 +++++++++++++++++++++-- arch/arm64/kvm/hyp/include/hyp/debug-sr.h | 4 +-- arch/arm64/kvm/hyp/include/hyp/switch.h | 8 +++--- arch/arm64/kvm/hyp/nvhe/switch.c | 2 +- arch/arm64/kvm/hyp/vhe/switch.c | 2 +- arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 4 +-- 7 files changed, 41 insertions(+), 32 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 6f98fbd0ac81..9149079f0269 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -149,26 +149,6 @@ extern char __smccc_workaround_1_smc[__SMCCC_WORKAROUND_1_SMC_SZ]; addr; \ }) -/* - * Home-grown __this_cpu_{ptr,read} variants that always work at HYP, - * provided that sym is really a *symbol* and not a pointer obtained from - * a data structure. As for SHIFT_PERCPU_PTR(), the creative casting keeps - * sparse quiet. - */ -#define __hyp_this_cpu_ptr(sym) \ - ({ \ - void *__ptr; \ - __verify_pcpu_ptr(&sym); \ - __ptr = hyp_symbol_addr(sym); \ - __ptr += read_sysreg(tpidr_el2); \ - (typeof(sym) __kernel __force *)__ptr; \ - }) - -#define __hyp_this_cpu_read(sym) \ - ({ \ - *__hyp_this_cpu_ptr(sym); \ - }) - #define __KVM_EXTABLE(from, to) \ " .pushsection __kvm_ex_table, \"a\"\n" \ " .align 3\n" \ diff --git a/arch/arm64/include/asm/percpu.h b/arch/arm64/include/asm/percpu.h index 0b6409b89e5e..b4008331475b 100644 --- a/arch/arm64/include/asm/percpu.h +++ b/arch/arm64/include/asm/percpu.h @@ -19,7 +19,21 @@ static inline void set_my_cpu_offset(unsigned long off) :: "r" (off) : "memory"); } -static inline unsigned long __my_cpu_offset(void) +static inline unsigned long __hyp_my_cpu_offset(void) +{ + unsigned long off; + + /* + * We want to allow caching the value, so avoid using volatile and + * instead use a fake stack read to hazard against barrier(). + */ + asm("mrs %0, tpidr_el2" : "=r" (off) : + "Q" (*(const unsigned long *)current_stack_pointer)); + + return off; +} + +static inline unsigned long __kern_my_cpu_offset(void) { unsigned long off; @@ -35,7 +49,12 @@ static inline unsigned long __my_cpu_offset(void) return off; } -#define __my_cpu_offset __my_cpu_offset() + +#ifdef __KVM_NVHE_HYPERVISOR__ +#define __my_cpu_offset __hyp_my_cpu_offset() +#else +#define __my_cpu_offset __kern_my_cpu_offset() +#endif #define PERCPU_RW_OPS(sz) \ static inline unsigned long __percpu_read_##sz(void *ptr) \ @@ -227,4 +246,14 @@ PERCPU_RET_OP(add, add, ldadd) #include +/* Redefine macros for nVHE hyp under DEBUG_PREEMPT to avoid its dependencies. */ +#if defined(__KVM_NVHE_HYPERVISOR__) && defined(CONFIG_DEBUG_PREEMPT) +#undef this_cpu_ptr +#define this_cpu_ptr raw_cpu_ptr +#undef __this_cpu_read +#define __this_cpu_read raw_cpu_read +#undef __this_cpu_write +#define __this_cpu_write raw_cpu_write +#endif + #endif /* __ASM_PERCPU_H */ diff --git a/arch/arm64/kvm/hyp/include/hyp/debug-sr.h b/arch/arm64/kvm/hyp/include/hyp/debug-sr.h index 5e28ea6aa097..4ebe9f558f3a 100644 --- a/arch/arm64/kvm/hyp/include/hyp/debug-sr.h +++ b/arch/arm64/kvm/hyp/include/hyp/debug-sr.h @@ -135,7 +135,7 @@ static inline void __debug_switch_to_guest_common(struct kvm_vcpu *vcpu) if (!(vcpu->arch.flags & KVM_ARM64_DEBUG_DIRTY)) return; - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; guest_ctxt = &vcpu->arch.ctxt; host_dbg = &vcpu->arch.host_debug_state.regs; guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr); @@ -154,7 +154,7 @@ static inline void __debug_switch_to_host_common(struct kvm_vcpu *vcpu) if (!(vcpu->arch.flags & KVM_ARM64_DEBUG_DIRTY)) return; - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; guest_ctxt = &vcpu->arch.ctxt; host_dbg = &vcpu->arch.host_debug_state.regs; guest_dbg = kern_hyp_va(vcpu->arch.debug_ptr); diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 5b6b8fa00f0a..f150407fa798 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -386,7 +386,7 @@ static inline bool __hyp_handle_ptrauth(struct kvm_vcpu *vcpu) !esr_is_ptrauth_trap(kvm_vcpu_get_esr(vcpu))) return false; - ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; __ptrauth_save_key(ctxt, APIA); __ptrauth_save_key(ctxt, APIB); __ptrauth_save_key(ctxt, APDA); @@ -495,7 +495,7 @@ static inline void __set_guest_arch_workaround_state(struct kvm_vcpu *vcpu) * guest wants it disabled, so be it... */ if (__needs_ssbd_off(vcpu) && - __hyp_this_cpu_read(arm64_ssbd_callback_required)) + __this_cpu_read(arm64_ssbd_callback_required)) arm_smccc_1_1_smc(ARM_SMCCC_ARCH_WORKAROUND_2, 0, NULL); #endif } @@ -507,7 +507,7 @@ static inline void __set_host_arch_workaround_state(struct kvm_vcpu *vcpu) * If the guest has disabled the workaround, bring it back on. */ if (__needs_ssbd_off(vcpu) && - __hyp_this_cpu_read(arm64_ssbd_callback_required)) + __this_cpu_read(arm64_ssbd_callback_required)) arm_smccc_1_1_smc(ARM_SMCCC_ARCH_WORKAROUND_2, 1, NULL); #endif } @@ -521,7 +521,7 @@ static inline void __kvm_unexpected_el2_exception(void) entry = hyp_symbol_addr(__start___kvm_ex_table); end = hyp_symbol_addr(__stop___kvm_ex_table); - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; while (entry < end) { addr = (unsigned long)&entry->insn + entry->insn; diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index 0970442d2dbc..cc4f8e790fb3 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -175,7 +175,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu) vcpu = kern_hyp_va(vcpu); - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; host_ctxt->__hyp_running_vcpu = vcpu; guest_ctxt = &vcpu->arch.ctxt; diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index c1da4f86ccac..575e8054f116 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -108,7 +108,7 @@ static int __kvm_vcpu_run_vhe(struct kvm_vcpu *vcpu) struct kvm_cpu_context *guest_ctxt; u64 exit_code; - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; host_ctxt->__hyp_running_vcpu = vcpu; guest_ctxt = &vcpu->arch.ctxt; diff --git a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c index 996471e4c138..2a0b8c88d74f 100644 --- a/arch/arm64/kvm/hyp/vhe/sysreg-sr.c +++ b/arch/arm64/kvm/hyp/vhe/sysreg-sr.c @@ -66,7 +66,7 @@ void kvm_vcpu_load_sysregs_vhe(struct kvm_vcpu *vcpu) struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt; struct kvm_cpu_context *host_ctxt; - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; __sysreg_save_user_state(host_ctxt); /* @@ -100,7 +100,7 @@ void kvm_vcpu_put_sysregs_vhe(struct kvm_vcpu *vcpu) struct kvm_cpu_context *guest_ctxt = &vcpu->arch.ctxt; struct kvm_cpu_context *host_ctxt; - host_ctxt = &__hyp_this_cpu_ptr(kvm_host_data)->host_ctxt; + host_ctxt = &this_cpu_ptr(&kvm_host_data)->host_ctxt; deactivate_traps_vhe_put(); __sysreg_save_el1_state(guest_ctxt); From patchwork Thu Sep 3 09:17:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11752961 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7ACCF14E5 for ; Thu, 3 Sep 2020 09:18:21 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5038D206C0 for ; Thu, 3 Sep 2020 09:18:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="taLzJb2/"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="Uw5/3Frz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5038D206C0 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=e9gfn0M0r8kqe8AnKZiUH+NDK2/hah3aeSecckMcwRs=; b=taLzJb2/5/b/REkG2TYVIqEdP URSQ8Kbb2OdXUJJ7ZFwk2zXGxkRiWXm0FML/jkrSwm+KsQgb98rm26qSNyqnGDbuIdmoNeHlHwHic WrdqPv0A/BpHJp2XxDab0btjatEPF8BkcQLpwb1yBxkMn2O2qnbWLhDbFsez6Ig6CJUdM1ST9VrgA ztAxqOe00imydRdhMmNh8zTZyDSPzo/Jsx/2EGMwou0djFkQrkIfkZHk7E6VEIRtfOW3O3fjokgFu HuHzpVf/m8GhUGS/xkVp+zMgP8MEn2C1WQuPqGqQmNSTzBbcqDUQ6g+BHrJxPtdf97MZdlkJVh/jW DafL2B1kA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNe-0004Ez-Hu; Thu, 03 Sep 2020 09:18:06 +0000 Received: from mail-ed1-x544.google.com ([2a00:1450:4864:20::544]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNK-00043a-J6 for linux-arm-kernel@lists.infradead.org; Thu, 03 Sep 2020 09:17:54 +0000 Received: by mail-ed1-x544.google.com with SMTP id i1so1789426edv.2 for ; Thu, 03 Sep 2020 02:17:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QisZTdt2TL1FFTIx3f7MU7jGvA7bpKi0v1t83QtatC8=; b=Uw5/3FrzMA18Q3Hr8bbsY/elC1IYtbQMCmA2A1rm0P1TwtZ0HUJ3+Lkb6GHUz/KfyT 5Al3hHDQuNzPO3OeXgaWJfxewKIYqv4MKaFVppm/Z/TcAZfbOgiQ4zYQxjLikS2PukDB eddkTC6h0wqw6WxN7MJlFEOae5fT5x7v3eWNKPT2CfUJ1fHB3Cd5E0hq+uPMBEc83KPG 7U6q8+wSN+z9aq3qXwyi1FejKiLD1Gpj7p/VuJsVIpVoihNVXCgu7KkkYPjGDpN0WPhY IF/SUzyo608sod7cU3tL38lq+aGlpUqVnUKjU3x2oKB72hrYza4FntfToHRguFg1IgDJ +80Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QisZTdt2TL1FFTIx3f7MU7jGvA7bpKi0v1t83QtatC8=; b=Pv1J76kl7TlI3n8hMvNLxipBNPE9HqaE/KIPcpBTRuJAGyFIXDacYmoqJbMw3m/j2X p++uk4l/HlqW5LgjTOWmYykCpzU6zHdCVOj4jVhvwuyypkH45GYyYnj4fFekrUgs0pfq eNSMNuKcgeK81yfBKhRwR1+OjGNw3im2OSzJ4VRQK9lQk9KYwmWJspZ4BDmFtLKFaNNI T4txiF65b2XAKsVzKSW4XNK8pz8an5OhMQZNyo0PlpWG65AT/pjSqT2EqACGOQuv+On+ jyxM3MzMXjHTa7N49nrK89LbS6/kPfsYD1SWXcb9KCBMsGWHEhEHVTOnb2G9ZvpfjOfl +w/Q== X-Gm-Message-State: AOAM531NDf62I0kwFIdSWzbOV35SYLYcHvMCZoruHxiQO4YL4GBdaTYX zoV5NdCm79c20/Of37g+HbljgA== X-Google-Smtp-Source: ABdhPJyZQ+B3hRMDuLLmB1WZ72xdRkglhDJaUzjBZFK8sPqN0Tzw3VvIDM0m15Hpqcb86+xd083gjw== X-Received: by 2002:a05:6402:486:: with SMTP id k6mr2060104edv.83.1599124665051; Thu, 03 Sep 2020 02:17:45 -0700 (PDT) Received: from localhost (dynamic-2a00-1028-919a-a06e-64ac-0036-822c-68d3.ipv6.broadband.iol.cz. [2a00:1028:919a:a06e:64ac:36:822c:68d3]) by smtp.gmail.com with ESMTPSA id m4sm2829132ejn.31.2020.09.03.02.17.43 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 03 Sep 2020 02:17:44 -0700 (PDT) From: David Brazdil To: Marc Zyngier , Catalin Marinas , Will Deacon , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH v2 04/10] kvm: arm64: Remove hyp_adr/ldr_this_cpu Date: Thu, 3 Sep 2020 11:17:06 +0200 Message-Id: <20200903091712.46456-5-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200903091712.46456-1-dbrazdil@google.com> References: <20200903091712.46456-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200903_051746_763515_A711F552 X-CRM114-Status: GOOD ( 15.10 ) X-Spam-Score: -16.2 (----------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-16.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:544 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.5 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, James Morse , Julien Thierry , David Brazdil , kernel-team@android.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org The hyp_adr/ldr_this_cpu helpers were introduced for use in hyp code because they always needed to use TPIDR_EL2 for base, while adr/ldr_this_cpu from kernel proper would select between TPIDR_EL2 and _EL1 based on VHE/nVHE. Simplify this now that the nVHE hyp mode case can be handled using the __KVM_NVHE_HYPERVISOR__ macro. VHE selects _EL2 with alternatives. Signed-off-by: David Brazdil Acked-by: Andrew Scull --- arch/arm64/include/asm/assembler.h | 27 +++++++++++++++++---------- arch/arm64/include/asm/kvm_asm.h | 14 +------------- arch/arm64/kvm/hyp/hyp-entry.S | 2 +- 3 files changed, 19 insertions(+), 24 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 54d181177656..b392a977efb6 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -218,6 +218,21 @@ lr .req x30 // link register str \src, [\tmp, :lo12:\sym] .endm + /* + * @dst: destination register (32 or 64 bit wide) + */ + .macro this_cpu_offset, dst +#ifdef __KVM_NVHE_HYPERVISOR__ + mrs \dst, tpidr_el2 +#else +alternative_if_not ARM64_HAS_VIRT_HOST_EXTN + mrs \dst, tpidr_el1 +alternative_else + mrs \dst, tpidr_el2 +alternative_endif +#endif + .endm + /* * @dst: Result of per_cpu(sym, smp_processor_id()) (can be SP) * @sym: The name of the per-cpu variable @@ -226,11 +241,7 @@ lr .req x30 // link register .macro adr_this_cpu, dst, sym, tmp adrp \tmp, \sym add \dst, \tmp, #:lo12:\sym -alternative_if_not ARM64_HAS_VIRT_HOST_EXTN - mrs \tmp, tpidr_el1 -alternative_else - mrs \tmp, tpidr_el2 -alternative_endif + this_cpu_offset \tmp add \dst, \dst, \tmp .endm @@ -241,11 +252,7 @@ alternative_endif */ .macro ldr_this_cpu dst, sym, tmp adr_l \dst, \sym -alternative_if_not ARM64_HAS_VIRT_HOST_EXTN - mrs \tmp, tpidr_el1 -alternative_else - mrs \tmp, tpidr_el2 -alternative_endif + this_cpu_offset \tmp ldr \dst, [\dst, \tmp] .endm diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 9149079f0269..469c0662f7f3 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -179,20 +179,8 @@ extern char __smccc_workaround_1_smc[__SMCCC_WORKAROUND_1_SMC_SZ]; #else /* __ASSEMBLY__ */ -.macro hyp_adr_this_cpu reg, sym, tmp - adr_l \reg, \sym - mrs \tmp, tpidr_el2 - add \reg, \reg, \tmp -.endm - -.macro hyp_ldr_this_cpu reg, sym, tmp - adr_l \reg, \sym - mrs \tmp, tpidr_el2 - ldr \reg, [\reg, \tmp] -.endm - .macro get_host_ctxt reg, tmp - hyp_adr_this_cpu \reg, kvm_host_data, \tmp + adr_this_cpu \reg, kvm_host_data, \tmp add \reg, \reg, #HOST_DATA_CONTEXT .endm diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S index 46b4dab933d0..fba91c2ab410 100644 --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -132,7 +132,7 @@ alternative_cb_end str x0, [x2, #VCPU_WORKAROUND_FLAGS] /* Check that we actually need to perform the call */ - hyp_ldr_this_cpu x0, arm64_ssbd_callback_required, x2 + ldr_this_cpu x0, arm64_ssbd_callback_required, x2 cbz x0, wa2_end mov w0, #ARM_SMCCC_ARCH_WORKAROUND_2 From patchwork Thu Sep 3 09:17:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11752963 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 60B39618 for ; Thu, 3 Sep 2020 09:18:27 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 346C4206C0 for ; Thu, 3 Sep 2020 09:18:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="gMrtZZhD"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="ibaN6UVu" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 346C4206C0 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=77uDIK6QT9NhxjwrKUL3VV0qvGbmI/atV7sea+PaFhk=; b=gMrtZZhD7lfL90myVWX7EH4ko T5vEGJZp3uredO4BZ7gyxgywrdMUTIWs6Efp/fJh1/d/TiCllWtmuWcwpVhCpk+TiG7SFLYlHBIsR ZLItztKeKVRFjK6Qv/jRK/jCNIsmVMLcALyQMRabzH7zczQQiUSowTTeO2iBrsu3Yr8Cxmn/FNKQM PMlKw2I9b4hx1ql2cf00Hgd3telhpUWTgJb1urH7cn4xh/h1IsoZ2AihEfWoVOiuVsCY6kB6EoNsI Q+TrwUBIfTbqsYttyXWhK9dpjlBnMUAdWexPtepb9mLYuAIPqZmuYl8DLxI7icjPL6kJ8iurZJVdx uoKjykJnA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNm-0004Is-JT; Thu, 03 Sep 2020 09:18:14 +0000 Received: from mail-ej1-x641.google.com ([2a00:1450:4864:20::641]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNN-000459-8s for linux-arm-kernel@lists.infradead.org; Thu, 03 Sep 2020 09:17:56 +0000 Received: by mail-ej1-x641.google.com with SMTP id z22so2815860ejl.7 for ; Thu, 03 Sep 2020 02:17:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lr26ux0OOjKqBElJON72X2bDuUHIIFflkukJO3BWbvU=; b=ibaN6UVuyMWanD/p/syhzRmTyFuupe1wk12IPBNXGichEgTkklyUUzy7oxzAUVwF14 5bnjjmfHLlMy5ji9KSMCLbP7UER0UcEHRPFkukXNMEJrAXyWkKKhNA6xGa5CjZ55qrBn M1JIPvdv5M1d5OYPv0nEEBa9+CWY76zMCqtZxvvUXWq+fTYMqlOYzh0VOa8DBh4sIxIU gPAB3kITbTsjqhMnV9JKcuqDcJEd+mL27vzwiNl8NSdIRVhBbvONoim/RFqzbpZ1zGDX KdUU4dB2O45UnNf3T4r/QYNBddgXEb080EB4ixmCZwI96W4HF9pMuBnfG+4vo77c214P +hmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lr26ux0OOjKqBElJON72X2bDuUHIIFflkukJO3BWbvU=; b=rX3tdIGDd3Ihw4XAmO0CgzFaP1stEbpjLBuayJ/uBM8sswHFGamjT1FAwPtXN/d48f 1dN15zhJ/xIT4gt0GwtT2wr9UAHxO+jA8z7rjl40Y4MVSgLt1/V8U3c1EkhjTy2ikrNm GqusfrUpj+pEBAHLYtCJPmNNsIdmljJWWF+GrvO8sO/5RinCGz7SLkJrNOs0D/IZASBd sSUykGG2vqLNW7AvZczEYcD+UbqxLwXlTVjFskET/FI8E7358S42QeFgqIy3zcoQdyaK zed0R2JAe+nDVlifw0ikJd3GgT5rLN44UlcqNVG9cekYo4G/uik6G0f1jk/Wf1bMerZH O/qA== X-Gm-Message-State: AOAM5312qJri06P3SrPIk/PidIqe5RSQhqZ6kFEjxhw+G3bBYoGbzAoY v/mfsUiszIr0iuuGtAtr+BPzLQ== X-Google-Smtp-Source: ABdhPJzEdTeSQp0oSfef6z3rNf21sKeagThs669GopkjjafQDEK3m3QDA/V0e2Zh7SmEpjSfWGhgLw== X-Received: by 2002:a17:906:5856:: with SMTP id h22mr1120110ejs.480.1599124667692; Thu, 03 Sep 2020 02:17:47 -0700 (PDT) Received: from localhost (dynamic-2a00-1028-919a-a06e-64ac-0036-822c-68d3.ipv6.broadband.iol.cz. [2a00:1028:919a:a06e:64ac:36:822c:68d3]) by smtp.gmail.com with ESMTPSA id gw6sm2638439ejb.47.2020.09.03.02.17.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 03 Sep 2020 02:17:46 -0700 (PDT) From: David Brazdil To: Marc Zyngier , Catalin Marinas , Will Deacon , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH v2 05/10] kvm: arm64: Add helpers for accessing nVHE hyp per-cpu vars Date: Thu, 3 Sep 2020 11:17:07 +0200 Message-Id: <20200903091712.46456-6-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200903091712.46456-1-dbrazdil@google.com> References: <20200903091712.46456-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200903_051749_409569_1ED79A7B X-CRM114-Status: GOOD ( 15.72 ) X-Spam-Score: -16.2 (----------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-16.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:641 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.5 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, James Morse , Julien Thierry , David Brazdil , kernel-team@android.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Defining a per-CPU variable in hyp/nvhe will result in its name being prefixed with __kvm_nvhe_. Add helpers for declaring these variables in kernel proper and accessing them with this_cpu_ptr and per_cpu_ptr. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_asm.h | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 469c0662f7f3..2b89817cdb01 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -60,9 +60,21 @@ DECLARE_KVM_VHE_SYM(sym); \ DECLARE_KVM_NVHE_SYM(sym) +#define DECLARE_KVM_VHE_PER_CPU(type, sym) \ + DECLARE_PER_CPU(type, sym) +#define DECLARE_KVM_NVHE_PER_CPU(type, sym) \ + DECLARE_PER_CPU(type, kvm_nvhe_sym(sym)) + +#define DECLARE_KVM_HYP_PER_CPU(type, sym) \ + DECLARE_KVM_VHE_PER_CPU(type, sym); \ + DECLARE_KVM_NVHE_PER_CPU(type, sym) + #define CHOOSE_VHE_SYM(sym) sym #define CHOOSE_NVHE_SYM(sym) kvm_nvhe_sym(sym) +#define this_cpu_ptr_nvhe(sym) this_cpu_ptr(&kvm_nvhe_sym(sym)) +#define per_cpu_ptr_nvhe(sym, cpu) per_cpu_ptr(&kvm_nvhe_sym(sym), cpu) + #ifndef __KVM_NVHE_HYPERVISOR__ /* * BIG FAT WARNINGS: @@ -75,12 +87,21 @@ * - Don't let the nVHE hypervisor have access to this, as it will * pick the *wrong* symbol (yes, it runs at EL2...). */ -#define CHOOSE_HYP_SYM(sym) (is_kernel_in_hyp_mode() ? CHOOSE_VHE_SYM(sym) \ +#define CHOOSE_HYP_SYM(sym) (is_kernel_in_hyp_mode() \ + ? CHOOSE_VHE_SYM(sym) \ : CHOOSE_NVHE_SYM(sym)) +#define this_cpu_ptr_hyp(sym) (is_kernel_in_hyp_mode() \ + ? this_cpu_ptr(&sym) \ + : this_cpu_ptr_nvhe(sym)) +#define per_cpu_ptr_hyp(sym, cpu) (is_kernel_in_hyp_mode() \ + ? per_cpu_ptr(&sym, cpu) \ + : per_cpu_ptr_nvhe(sym, cpu)) #else /* The nVHE hypervisor shouldn't even try to access anything */ extern void *__nvhe_undefined_symbol; -#define CHOOSE_HYP_SYM(sym) __nvhe_undefined_symbol +#define CHOOSE_HYP_SYM(sym) __nvhe_undefined_symbol +#define this_cpu_ptr_hyp(sym) (&__nvhe_undefined_symbol) +#define per_cpu_ptr_hyp(sym, cpu) (&__nvhe_undefined_symbol) #endif /* Translate a kernel address @ptr into its equivalent linear mapping */ From patchwork Thu Sep 3 09:17:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11752965 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 614C7618 for ; Thu, 3 Sep 2020 09:18:30 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 314B8207EA for ; Thu, 3 Sep 2020 09:18:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="09QApZ6C"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="fSryvZyp" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 314B8207EA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=0zegdWW2oOyIl2lfwjbMITUkXzjvw3vIRXSxYAQHLQ8=; b=09QApZ6CH11HlUd6WmDOjQu2h l9qZMvaj2+eRLTWnx2bQu5LLx+5+GoS5TdvcdcM+m19xVMnchDdvcnQbz802Etve4+OzsHtjHuG5w wXQhZ1YvyVesfV85ptc6CIHlw9hZt9HkJDjV0N41/RV6WM1LPCJxYqeqUpd0wflv9Yc88KsTp87Fd FSfIiZnifmrbfWqJMFovqrAmRrviFP4Vk/bFHInUUqZrjIljsJKpe/KRE8+7JqK3lwMfPe0x4Rck1 zWoDnHuDwQ+j11Kx/i8+pnn6tpQ+/yL4zkTAKzayoYfM8ZzJI04e+IG2oWxjJS9vSsud9uwGRBH+R Scn2g2XpQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNq-0004K9-Gh; Thu, 03 Sep 2020 09:18:18 +0000 Received: from mail-ej1-x643.google.com ([2a00:1450:4864:20::643]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNQ-00047M-0v for linux-arm-kernel@lists.infradead.org; Thu, 03 Sep 2020 09:18:01 +0000 Received: by mail-ej1-x643.google.com with SMTP id e23so2854443eja.3 for ; Thu, 03 Sep 2020 02:17:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=mwdRMAavqdTzOvo7Ir5BJvWRsG9Q05qSRw6OHtvkMgg=; b=fSryvZypFiRayJ6qbXdHAuS+6FDnue70hvCWNO7yOK8m0hMkAwkbG1pA/JeNI0g4bP 6BM7jhcArJY2rH/F2rySk6k1pOyCgrpB3aynRg2CWQT9D+5Gdy/9FaJUCJgmN9O/SF3N 4+pQVl03npehC5dbcL3VFetoYyKoZjOs1/SuBZd8+sE6LrP6fd+64/cDg6RMPEfAHBdj yJWNrZ4eV4GH5S84SAX+WO2Nk68D36mOzVejo4YmLFm+B0tkHmYZu33yKfcuj2dtUon6 DM77MiypYG43OUNr5/rK0Bs6MkABNFfbRrRvyezu9uc+d9VeuHmo81HSmvyhD1Ku4Qg2 NVrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mwdRMAavqdTzOvo7Ir5BJvWRsG9Q05qSRw6OHtvkMgg=; b=RpQE50cEFAI5KinJcAsTFjDcZgaSgvGmxqdvAVNp09BJAKC8ObgGeawrM9KXddZjC6 TPeZKMIgZzTETeRsa7OWxN6EGHwbiCIpbEOUu22W9I9UX8L0xL2AYZQmY8nvycEsJrwh /LZKDPDEnW8YatTcX4lnVNhMWBYP6lYFEDXxk+aZvXVjIcL3hY3/aTVOfQ+97MoGePHN JBqAwqxJOLlj4qhpWnediMHn5GBLjzSlZjRrAygRyXOXpBigZe4CqY4EXZe1MkNP2gNi 2TNUuXcZP+BVC+ZZKmGaynYsaHeqDb390fl9clf9xeRSnyEi7LR7ZMW3rSsHw+gUilwl KGcw== X-Gm-Message-State: AOAM531xirftxbtNctxnjfY4rs/BSU3KsxVgAZjUzdkNjrgvAIHGopKY XKSpBGbhb2Gswrd/iuzElgpQag== X-Google-Smtp-Source: ABdhPJyIo0tfIVl9zQQOPILZ6Iv+dAifCx9Hj7ENqoQaHZteaS0OleBiI3IwVUhr2OYIrgj5J+SKnQ== X-Received: by 2002:a17:906:a1d8:: with SMTP id bx24mr1085448ejb.161.1599124670371; Thu, 03 Sep 2020 02:17:50 -0700 (PDT) Received: from localhost (dynamic-2a00-1028-919a-a06e-64ac-0036-822c-68d3.ipv6.broadband.iol.cz. [2a00:1028:919a:a06e:64ac:36:822c:68d3]) by smtp.gmail.com with ESMTPSA id l19sm2594526ejn.67.2020.09.03.02.17.48 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 03 Sep 2020 02:17:49 -0700 (PDT) From: David Brazdil To: Marc Zyngier , Catalin Marinas , Will Deacon , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH v2 06/10] kvm: arm64: Duplicate arm64_ssbd_callback_required for nVHE hyp Date: Thu, 3 Sep 2020 11:17:08 +0200 Message-Id: <20200903091712.46456-7-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200903091712.46456-1-dbrazdil@google.com> References: <20200903091712.46456-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200903_051752_137654_64FFA583 X-CRM114-Status: GOOD ( 18.78 ) X-Spam-Score: -16.2 (----------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-16.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:643 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.5 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, James Morse , Julien Thierry , David Brazdil , kernel-team@android.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Hyp keeps track of which cores require SSBD callback by accessing a kernel-proper global variable. Create an nVHE symbol of the same name and copy the value from kernel proper to nVHE at KVM init time. Done in preparation for separating percpu memory owned by kernel proper and nVHE. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_mmu.h | 10 +++++++--- arch/arm64/kernel/image-vars.h | 1 - arch/arm64/kvm/arm.c | 2 +- arch/arm64/kvm/hyp/nvhe/switch.c | 3 +++ 4 files changed, 11 insertions(+), 5 deletions(-) diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 189839c3706a..9db93da35606 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -529,23 +529,27 @@ static inline int kvm_map_vectors(void) #ifdef CONFIG_ARM64_SSBD DECLARE_PER_CPU_READ_MOSTLY(u64, arm64_ssbd_callback_required); +DECLARE_KVM_NVHE_PER_CPU(u64, arm64_ssbd_callback_required); -static inline int hyp_map_aux_data(void) +static inline int hyp_init_aux_data(void) { int cpu, err; for_each_possible_cpu(cpu) { u64 *ptr; - ptr = per_cpu_ptr(&arm64_ssbd_callback_required, cpu); + ptr = per_cpu_ptr_nvhe(arm64_ssbd_callback_required, cpu); err = create_hyp_mappings(ptr, ptr + 1, PAGE_HYP); if (err) return err; + + /* Copy value from kernel to hyp. */ + *ptr = per_cpu(arm64_ssbd_callback_required, cpu); } return 0; } #else -static inline int hyp_map_aux_data(void) +static inline int hyp_init_aux_data(void) { return 0; } diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 8982b68289b7..5fc5eadfb3e6 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -69,7 +69,6 @@ KVM_NVHE_ALIAS(kvm_patch_vector_branch); KVM_NVHE_ALIAS(kvm_update_va_mask); /* Global kernel state accessed by nVHE hyp code. */ -KVM_NVHE_ALIAS(arm64_ssbd_callback_required); KVM_NVHE_ALIAS(kvm_host_data); KVM_NVHE_ALIAS(kvm_vgic_global_state); diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 46dc3d75cf13..1bb960812493 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1549,7 +1549,7 @@ static int init_hyp_mode(void) } } - err = hyp_map_aux_data(); + err = hyp_init_aux_data(); if (err) kvm_err("Cannot map host auxiliary data: %d\n", err); diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index cc4f8e790fb3..4662df6330d7 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -27,6 +27,9 @@ #include #include +/* Non-VHE copy of the kernel symbol. */ +DEFINE_PER_CPU_READ_MOSTLY(u64, arm64_ssbd_callback_required); + static void __activate_traps(struct kvm_vcpu *vcpu) { u64 val; From patchwork Thu Sep 3 09:17:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11752967 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0E206618 for ; Thu, 3 Sep 2020 09:18:38 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CA36420E65 for ; Thu, 3 Sep 2020 09:18:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="kkahpTQ1"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="nMfWtLwF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CA36420E65 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ZKKjziuZu4gI3SI3aOfbiGeyeCWb/g4Pja1zfAhXoiA=; b=kkahpTQ1LQRH5xoRjAgEXLrQh +mcAeCfa7I6WVjJG+sxM6SWitaRXuOPCdBVz8soxJeVniErcFTK5qEjzgOBpXp2tosVgW5v4X7P2Q +fNRQVaMgQTiaTDG8xt/g5DmSqRBhi9r1O5bGUR8LZS2Jw1R+2L1PJR6FEJ0GWXqpgY00zuqTGlTS GVTrz+HUtZQWSMyNeotWTNjBp9Kguoqu/eA3UEKhv/wGXIlobL+RorMySjkIuTWmX3f0N7M59AQG8 qjNTqHV3DbXPQbAEHh+tv/ZVO1H34qb87A9w1Fmmz+Ykv01PTNtrRulMvVS3OPveeOuKLgqfT+xo5 qwZXzAuyg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNu-0004MF-Jq; Thu, 03 Sep 2020 09:18:22 +0000 Received: from mail-ej1-x642.google.com ([2a00:1450:4864:20::642]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNS-000490-FN for linux-arm-kernel@lists.infradead.org; Thu, 03 Sep 2020 09:18:05 +0000 Received: by mail-ej1-x642.google.com with SMTP id m22so2791430eje.10 for ; Thu, 03 Sep 2020 02:17:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=22FDsG2iZpK/4rGNv58xs8DHFdUQri5Jr1UNE+6mylQ=; b=nMfWtLwFlKOKjnQRBR8B1Cf3oI5FbF7UZcTPADs/DolIRxiGjjVgPUhp7JOS4z2rbb uLLDsPat/+nF1SZHOeuqv5yof7xurbgnFAnWmj3y1LAP2D/M5cYjGuR2IZeoOizewrMx As1Q/GCznqtOImaDt++9UkE1rlwejNJlIG6W7mmQkxnFAUB/+oGZ+AzNTD8+1M2O2u1l 6YB9EXX5DLq1CKv9EHKUc2KRxhJ5No1Nif9rprHTMGleWBjQkTyfMdntl43ps8v5kOYL AZtRLAWGSgk8M0FjnGtgsRo79ZO1DodIMFTJ6M/ysYHNSKNicuD5JrjRW334QcaIgx2R eYmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=22FDsG2iZpK/4rGNv58xs8DHFdUQri5Jr1UNE+6mylQ=; b=lAjX/JL+OYhDUpXep90ZHxESWCYyEOfXSPsV9z2fKxRlrrcWY7W9WdiZW2pEzb/O7e YU80sfPjHnJLGH2gTxY+5KD7NtCXghjaGuA4RzzZg0nuDwuRmnm/7E7LdBeHV0Gvrpip t+nAH9HZ4sNpLevu2H8RtSTXUlEWxn7tATJTg6tSwhkV3At3Cte7uzqwMKT6L/pF4akq 5NndHNGSBs4BDGJ6zMVBegDbUZlOvh2+Hhqyno4K9rQdEedh8gOkFRgA7Jmu27wfoA6q OlIX7F7Yk2pZjj0yg+qthb171eJbn9WyFKKSVamzIbrmPJD1/ZVx3Pf25XC6lNKWUm2T c+pA== X-Gm-Message-State: AOAM531fQUNqCRKClfHMGtxkpjp5RatJ7m8nEY7HY7MaHT+GtWp+/I44 fYadkvI+SsUQh4ZTlbq35zNjgh3tQpcawGHCRew= X-Google-Smtp-Source: ABdhPJxyfhSRIDHpyvYf5FPIpo/5Aj41XM/FjbhFjdrBvNpmd3UNySmojerVgxx5Cn0qBhV+NQdlxA== X-Received: by 2002:a17:906:f1cf:: with SMTP id gx15mr1090496ejb.241.1599124673252; Thu, 03 Sep 2020 02:17:53 -0700 (PDT) Received: from localhost (dynamic-2a00-1028-919a-a06e-64ac-0036-822c-68d3.ipv6.broadband.iol.cz. [2a00:1028:919a:a06e:64ac:36:822c:68d3]) by smtp.gmail.com with ESMTPSA id y8sm2587216ejd.57.2020.09.03.02.17.51 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 03 Sep 2020 02:17:52 -0700 (PDT) From: David Brazdil To: Marc Zyngier , Catalin Marinas , Will Deacon , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH v2 07/10] kvm: arm64: Create separate instances of kvm_host_data for VHE/nVHE Date: Thu, 3 Sep 2020 11:17:09 +0200 Message-Id: <20200903091712.46456-8-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200903091712.46456-1-dbrazdil@google.com> References: <20200903091712.46456-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200903_051754_617361_7ABA53E7 X-CRM114-Status: GOOD ( 18.01 ) X-Spam-Score: -16.2 (----------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-16.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:642 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.5 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, James Morse , Julien Thierry , David Brazdil , kernel-team@android.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Host CPU context is stored in a global per-cpu variable `kvm_host_data`. In preparation for introducing independent per-CPU region for nVHE hyp, create two separate instances of `kvm_host_data`, one for VHE and one for nVHE. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_host.h | 2 +- arch/arm64/kernel/image-vars.h | 1 - arch/arm64/kvm/arm.c | 5 ++--- arch/arm64/kvm/hyp/nvhe/switch.c | 3 +++ arch/arm64/kvm/hyp/vhe/switch.c | 3 +++ arch/arm64/kvm/pmu.c | 8 ++++---- 6 files changed, 13 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index e52c927aade5..964e05777fe3 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -565,7 +565,7 @@ void kvm_set_sei_esr(struct kvm_vcpu *vcpu, u64 syndrome); struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, unsigned long mpidr); -DECLARE_PER_CPU(kvm_host_data_t, kvm_host_data); +DECLARE_KVM_HYP_PER_CPU(kvm_host_data_t, kvm_host_data); static inline void kvm_init_host_cpu_context(struct kvm_cpu_context *cpu_ctxt) { diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index 5fc5eadfb3e6..21307e2db3fc 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -69,7 +69,6 @@ KVM_NVHE_ALIAS(kvm_patch_vector_branch); KVM_NVHE_ALIAS(kvm_update_va_mask); /* Global kernel state accessed by nVHE hyp code. */ -KVM_NVHE_ALIAS(kvm_host_data); KVM_NVHE_ALIAS(kvm_vgic_global_state); /* Kernel constant needed to compute idmap addresses. */ diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 1bb960812493..d437052c5481 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -46,7 +46,6 @@ __asm__(".arch_extension virt"); #endif -DEFINE_PER_CPU(kvm_host_data_t, kvm_host_data); static DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); /* The VMID used in the VTTBR */ @@ -1305,7 +1304,7 @@ static void cpu_hyp_reset(void) static void cpu_hyp_reinit(void) { - kvm_init_host_cpu_context(&this_cpu_ptr(&kvm_host_data)->host_ctxt); + kvm_init_host_cpu_context(&this_cpu_ptr_hyp(kvm_host_data)->host_ctxt); cpu_hyp_reset(); @@ -1540,7 +1539,7 @@ static int init_hyp_mode(void) for_each_possible_cpu(cpu) { kvm_host_data_t *cpu_data; - cpu_data = per_cpu_ptr(&kvm_host_data, cpu); + cpu_data = per_cpu_ptr_hyp(kvm_host_data, cpu); err = create_hyp_mappings(cpu_data, cpu_data + 1, PAGE_HYP); if (err) { diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/switch.c index 4662df6330d7..a7e9b03bd9d1 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -30,6 +30,9 @@ /* Non-VHE copy of the kernel symbol. */ DEFINE_PER_CPU_READ_MOSTLY(u64, arm64_ssbd_callback_required); +/* Non-VHE instance of kvm_host_data. */ +DEFINE_PER_CPU(kvm_host_data_t, kvm_host_data); + static void __activate_traps(struct kvm_vcpu *vcpu) { u64 val; diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c index 575e8054f116..0949fc97bf03 100644 --- a/arch/arm64/kvm/hyp/vhe/switch.c +++ b/arch/arm64/kvm/hyp/vhe/switch.c @@ -28,6 +28,9 @@ const char __hyp_panic_string[] = "HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n"; +/* VHE instance of kvm_host_data. */ +DEFINE_PER_CPU(kvm_host_data_t, kvm_host_data); + static void __activate_traps(struct kvm_vcpu *vcpu) { u64 val; diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c index 3c224162b3dd..6d80ffe1ebfc 100644 --- a/arch/arm64/kvm/pmu.c +++ b/arch/arm64/kvm/pmu.c @@ -31,7 +31,7 @@ static bool kvm_pmu_switch_needed(struct perf_event_attr *attr) */ void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr) { - struct kvm_host_data *ctx = this_cpu_ptr(&kvm_host_data); + struct kvm_host_data *ctx = this_cpu_ptr_hyp(kvm_host_data); if (!kvm_pmu_switch_needed(attr)) return; @@ -47,7 +47,7 @@ void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr) */ void kvm_clr_pmu_events(u32 clr) { - struct kvm_host_data *ctx = this_cpu_ptr(&kvm_host_data); + struct kvm_host_data *ctx = this_cpu_ptr_hyp(kvm_host_data); ctx->pmu_events.events_host &= ~clr; ctx->pmu_events.events_guest &= ~clr; @@ -173,7 +173,7 @@ void kvm_vcpu_pmu_restore_guest(struct kvm_vcpu *vcpu) return; preempt_disable(); - host = this_cpu_ptr(&kvm_host_data); + host = this_cpu_ptr_hyp(kvm_host_data); events_guest = host->pmu_events.events_guest; events_host = host->pmu_events.events_host; @@ -193,7 +193,7 @@ void kvm_vcpu_pmu_restore_host(struct kvm_vcpu *vcpu) if (!has_vhe()) return; - host = this_cpu_ptr(&kvm_host_data); + host = this_cpu_ptr_hyp(kvm_host_data); events_guest = host->pmu_events.events_guest; events_host = host->pmu_events.events_host; From patchwork Thu Sep 3 09:17:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11752973 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 483DD14E5 for ; Thu, 3 Sep 2020 09:20:04 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 176F0206C0 for ; Thu, 3 Sep 2020 09:20:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="3P7pubhq"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="siEh3Fy4"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="hUF/wWzn" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 176F0206C0 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=fCuoXNFcvAG1RgJOvNjNFmVrVSfd474gEI69YkP3dY4=; b=3P7pubhqYhecCde1Xv1r26i+6 gt1tl1Dup7D02DdTQRYi2DRWVtQmwfg6eQOG8420Hlq2FDzwjZWyyGbea5kdYqty57Y2mL8kF9HhK uohU7zjpcCmp2w5KNOvUzrfY+NYQuwFy174nLrLVR3xyc3Zkw1vVDjpW47Z4Un6WlnTsNeUw3l4jr YAFtP57CgxFT+DJQz9IbSCOFw/Esog1d8CCxPki6N5+ueDUSpelpPeto+UzDC0RKetOmeskU74h/r UA93z2LF2xOojNkXUtbSU1KbwLHp8y64vskrEDhe40FHt4d4jElpyT9t9QiHRTXk7ukjcM2BqWu86 rK0FQS/RA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNy-0004Nx-TM; Thu, 03 Sep 2020 09:18:26 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNe-0004Fp-06 for linux-arm-kernel@merlin.infradead.org; Thu, 03 Sep 2020 09:18:06 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=SnlRtG/CWSkZI4NVeKIKEydnJw+LdVFZjKE2XgTifuQ=; b=siEh3Fy4SSCUlnrTN+tQ7hEGej iIx0MSFvUlLVZoEidQWta4yXG8O6oISlQDQbhRbIRzkecS9viLR9m0bwaRCBuML4DsGzcX7hize38 yPTcnsK72tkTF4Tqiwg2P9uzVKP4xOPyeFFYNSeUP4vzfz96DzobOtX/nt9EO3gigyepb846DZEAu xEgZCfCM7gQPA9EdNALd2nrNMTg7xlOs2EaR/MWxQXLq630ZOsrwTdQMnYpMwgJsEL5cl6Fr0jTuY ty6B+fFhx8AHCd1T9H0jgylOeYECYHZ6ElKFRrdfrs4Ekcf4W/vFGCGz7M8l4sxDNNKA+hVkZD+O4 eFpSRU7Q==; Received: from mail-ed1-x541.google.com ([2a00:1450:4864:20::541]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNX-0004KG-Vo for linux-arm-kernel@lists.infradead.org; Thu, 03 Sep 2020 09:18:04 +0000 Received: by mail-ed1-x541.google.com with SMTP id a12so1880558eds.13 for ; Thu, 03 Sep 2020 02:17:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SnlRtG/CWSkZI4NVeKIKEydnJw+LdVFZjKE2XgTifuQ=; b=hUF/wWznrhIV4kD0nNa5LDuTyvYBcRJqJRT4AjaF0cV0Cm85jlZhKS8Ka8OrV/1IAb 3ShKef+kcOd+ghf91meHp8twmWmMTmr4i5li7S6u7jJW6COg/KwmunepQBEEG8a5QFIu 1S2h+XyMXH330ZazXmjmCzIWyT2Qywoc6aqRJCEVxaYU4ilpOuUylSu269/RPYD7X8bP NjDFh5AJcfRDgrzcsJvPmljNBjZqDVaWRNQ/lIcy0aO4vsDvRiJzYGI3BUma+LFFGV1K 2wy6oRw6OCKcrv6ifEqMw0FCTMubNFMKgTFfLp2vzTIloUIxgJ9boRrWrZ6ZZz5qUzG+ xdAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SnlRtG/CWSkZI4NVeKIKEydnJw+LdVFZjKE2XgTifuQ=; b=cg8FFbO544KZUPqS9OoCbSWjJyJc55QVNUPOlI/fvwPo+BA1aIq990iD9s8tPPEAY6 mGkWzxkk849QV77mC+IUThws9xRsnYHYJDbeUdnZu2UQlszvh0eF9vL3oGm+H21qqzq3 xVArGXkBCb9CmQ4O/Zo5oyVH7fgkNE+tbMCiwq+SEykzj7CkVHACe26oJ/Pjhg5KHp66 MUWWtOF2b1G0QX2gVsEAvlFTYxScDtN8VbbyHApJFOnUehajWAuuB2pl7HUjchjuiCmH fE0uq/bA1pyIAVKDjMnWe5yARLJS/LegeoAwhsvq5gcGBnwfPM4MnLFsN/7M+r2Q2MAq 7I9A== X-Gm-Message-State: AOAM532uscLTEW69cIdRTrVA1FtoMNX9tTRS/T+gwb1uMVYQmPe3ZqWZ FiAiBz8oc/8t55t6lI/qjOKHog== X-Google-Smtp-Source: ABdhPJyT2Eil9AjKw8R/4nwj3PA9MLb8SnTJe5Rd8N1txrd2cT5Dl89hJVcJvGxtWZCvckgjsovWlA== X-Received: by 2002:aa7:dc16:: with SMTP id b22mr2013761edu.252.1599124675844; Thu, 03 Sep 2020 02:17:55 -0700 (PDT) Received: from localhost (dynamic-2a00-1028-919a-a06e-64ac-0036-822c-68d3.ipv6.broadband.iol.cz. [2a00:1028:919a:a06e:64ac:36:822c:68d3]) by smtp.gmail.com with ESMTPSA id v13sm2714856edl.9.2020.09.03.02.17.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 03 Sep 2020 02:17:55 -0700 (PDT) From: David Brazdil To: Marc Zyngier , Catalin Marinas , Will Deacon , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH v2 08/10] kvm: arm64: Mark hyp stack pages reserved Date: Thu, 3 Sep 2020 11:17:10 +0200 Message-Id: <20200903091712.46456-9-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200903091712.46456-1-dbrazdil@google.com> References: <20200903091712.46456-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200903_101800_576113_2059F074 X-CRM114-Status: GOOD ( 18.71 ) X-Spam-Score: -18.1 (------------------) X-Spam-Report: SpamAssassin version 3.4.4 on casper.infradead.org summary: Content analysis details: (-18.1 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:541 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.5 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, James Morse , Julien Thierry , David Brazdil , kernel-team@android.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org In preparation for unmapping hyp pages from host stage-2, allocate/free hyp stack using new helpers which automatically mark the pages reserved. Signed-off-by: David Brazdil --- arch/arm64/kvm/arm.c | 49 ++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 47 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index d437052c5481..8a1fcf4dffca 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1453,13 +1453,58 @@ static int init_subsystems(void) return err; } +/* + * Alloc pages and mark them reserved so the kernel never tries to + * take them away from the hypervisor. + */ +static unsigned long alloc_hyp_pages(gfp_t flags, unsigned int order) +{ + struct page *page; + unsigned long i; + + page = alloc_pages(flags, order); + if (!page) + return 0; + + for (i = 0; i < (1ul << order); ++i) + mark_page_reserved(page + i); + + return (unsigned long)page_address(page); +} + +static unsigned long alloc_hyp_page(gfp_t flags) +{ + return alloc_hyp_pages(flags, 0); +} + +/* + * Free pages which were previously marked reserved for the hypervisor. + */ +static void free_hyp_pages(unsigned long addr, unsigned int order) +{ + unsigned long i; + struct page *page; + + if (!addr) + return; + + page = virt_to_page(addr); + for (i = 0; i < (1ul << order); ++i) + free_reserved_page(page + i); +} + +static void free_hyp_page(unsigned long addr) +{ + return free_hyp_pages(addr, 0); +} + static void teardown_hyp_mode(void) { int cpu; free_hyp_pgds(); for_each_possible_cpu(cpu) - free_page(per_cpu(kvm_arm_hyp_stack_page, cpu)); + free_hyp_page(per_cpu(kvm_arm_hyp_stack_page, cpu)); } /** @@ -1483,7 +1528,7 @@ static int init_hyp_mode(void) for_each_possible_cpu(cpu) { unsigned long stack_page; - stack_page = __get_free_page(GFP_KERNEL); + stack_page = alloc_hyp_page(GFP_KERNEL); if (!stack_page) { err = -ENOMEM; goto out_err; From patchwork Thu Sep 3 09:17:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11752969 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 31849618 for ; Thu, 3 Sep 2020 09:18:57 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C971B208C7 for ; Thu, 3 Sep 2020 09:18:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="pNlTrZrc"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="YHJwj+2b"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="Z3AwNlaf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C971B208C7 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=k6lyqEEvJw48lqP/PamfNJ0dhQuIq46hNI9DbbHWCAM=; b=pNlTrZrcwlJMP9CdRKzsw1RK1 jRfuJrv370tEYzVRhMzKaPIPYFlE70KfIKaYkASltOCVMYiQQa4HerXaX0/056uz4GKrLTqoRpNzY PLY9/0wMOQ1xpH0Bw0vBYKhHnXbCVh6X5sa7HBf815io5kMB2luxPCBo0g81k/vmow06zFls77/ve K9kMZNIr/NzAkxEdEh1YsR7QrqktqQUo4RhOyOkSXm09hzDyfqkoOjNOpz/9XbyqBMQL4Fp3A85xC qvwaGm29nXlqR914JTwwUJhfJwxCyPiRp71SZaeWeJfDo/r8snZaQ2/3rje+cCfEVBMaUJNxn95fc vhVvkUsZA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlO6-0004Rc-AA; Thu, 03 Sep 2020 09:18:34 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNe-0004G3-Fd for linux-arm-kernel@merlin.infradead.org; Thu, 03 Sep 2020 09:18:06 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=kA/M5uo1pgexmorks7VPWASAyNAWS3vx6/WPRm8af9o=; b=YHJwj+2buFt5SQlZTBUNCvpwW4 HPa1KFVHkOYpfEYCmPADEIk5MWBLGlg7sBI/Lrzewvn3Er0uRzlLn5q36j2WLK2ITqawD3DxHTuoq +fHbwucd9S7ctq7RJazsL97uBPiNO4E0+XtuMlZ7X5Kfw9HY4A6OI3acI7pWgWBL1VVOojdTUoc7U qoTLlmu5xnHqsedLzCfQzgrwtoUy8nn/GphaDCFGnUfxFUSuH0hmvvxNUFcANTCn/2FrqGVTwQYJw QsUa6M5xl2gK3y/bT+vpNZU3+PA6BOrEttD+bLdjki0DCy6KUGPZOeyVJVeAiKGg7MxOwIz4aTnQl 6Lx9Fg+A==; Received: from mail-ej1-x629.google.com ([2a00:1450:4864:20::629]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNY-0004Ki-Ow for linux-arm-kernel@lists.infradead.org; Thu, 03 Sep 2020 09:18:05 +0000 Received: by mail-ej1-x629.google.com with SMTP id z22so2816596ejl.7 for ; Thu, 03 Sep 2020 02:17:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kA/M5uo1pgexmorks7VPWASAyNAWS3vx6/WPRm8af9o=; b=Z3AwNlaf6J5hW5xgtHRnIRpOFoT8tF/HLwZ8GJ0e5cvG+dJo2ZV8Td+XYGpSz0PDPU yZ/Eba/KurKSLu2xTfCSNv8w3CSqYugtsRgbJhG+wLL490SlO28pS2CRnJpJnDP/DwCo +E/F0+gL8wXjD44TOvVWyXQD/GZW/uPbFgKF9ERR7IXDZMkzS0sEOUJoNgA1H6/Byx4y NSwW2yoSQkQeLoA3uSwlSJJ/rckcuNL/6FNJHtZa16XVwYykjh/5UAxiCobCQ1amihKK R0W7G3p7XxI5OMHuVyUdK3LZNxzxpOJDnETrkKBJ1UwfzsUT0SGPh94z2Picpt7x6ZPk rD2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kA/M5uo1pgexmorks7VPWASAyNAWS3vx6/WPRm8af9o=; b=CUolicU6wSB5bypaTvCJruR4ku86DPeZ0vR/Zx7V3ITMcJfCvOlLYknhWavE/YzPrx aGRN3elrhPSefEWEZAf3N0PTn4GJJWGO8dKIXpEHMBRTMLFP0J/mZNpehYce7F6J2iWg dTt7KpdbNtK04Q+eL2fepGOWhi6B5hZrXFnsppiVLj3p724rd3NddnF239iPvd1XB2zK lyP1Q+8nFDIIJTo7mPorgSdNBJk1GgL3YchOOfcj6GOnAaZdhoa3grvN7qpqUAwfk0BD M8sOpIri2BDBJTqQRmZ4CmHqQKycElyLgwpXxrM02PSjXWARo1gIWnMiOOQ4UKwg1UYm fyug== X-Gm-Message-State: AOAM531Eb7OnQm+wXh6izXUGtWNT5fygvW08UDEcvvkmUqXIkvSyeIFR FMd9q0M2nRf0hfesCobmjrHWyA== X-Google-Smtp-Source: ABdhPJxmXDw453SzS5U6zfFul53RqVj1EsU1USDwBl8Q/g0xulot4lLcRQNNUqR52/MOFV2rbWsbIQ== X-Received: by 2002:a17:906:2f02:: with SMTP id v2mr1041974eji.465.1599124678549; Thu, 03 Sep 2020 02:17:58 -0700 (PDT) Received: from localhost (dynamic-2a00-1028-919a-a06e-64ac-0036-822c-68d3.ipv6.broadband.iol.cz. [2a00:1028:919a:a06e:64ac:36:822c:68d3]) by smtp.gmail.com with ESMTPSA id z23sm2614737eja.29.2020.09.03.02.17.56 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 03 Sep 2020 02:17:57 -0700 (PDT) From: David Brazdil To: Marc Zyngier , Catalin Marinas , Will Deacon , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH v2 09/10] kvm: arm64: Set up hyp percpu data for nVHE Date: Thu, 3 Sep 2020 11:17:11 +0200 Message-Id: <20200903091712.46456-10-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200903091712.46456-1-dbrazdil@google.com> References: <20200903091712.46456-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200903_101801_180492_4E581357 X-CRM114-Status: GOOD ( 24.08 ) X-Spam-Score: -18.1 (------------------) X-Spam-Report: SpamAssassin version 3.4.4 on casper.infradead.org summary: Content analysis details: (-18.1 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:629 listed in] [list.dnswl.org] -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] -0.0 SPF_PASS SPF: sender matches SPF record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.5 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, James Morse , Julien Thierry , David Brazdil , kernel-team@android.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Add hyp percpu section to linker script and rename the corresponding ELF sections of hyp/nvhe object files. This moves all nVHE-specific percpu variables to the new hyp percpu section. Allocate sufficient amount of memory for all percpu hyp regions at global KVM init time and create corresponding hyp mappings. The base addresses of hyp percpu regions are kept in a dynamically allocated array in the kernel. Add NULL checks in PMU event-reset code as it may run before KVM memory is initialized. Signed-off-by: David Brazdil --- arch/arm64/include/asm/kvm_asm.h | 19 +++++++++-- arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/vmlinux.lds.S | 10 ++++++ arch/arm64/kvm/arm.c | 56 +++++++++++++++++++++++++++++-- arch/arm64/kvm/hyp/nvhe/hyp.lds.S | 5 +++ arch/arm64/kvm/pmu.c | 5 ++- 6 files changed, 90 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h index 2b89817cdb01..c87111c25d9e 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -72,8 +72,23 @@ #define CHOOSE_VHE_SYM(sym) sym #define CHOOSE_NVHE_SYM(sym) kvm_nvhe_sym(sym) -#define this_cpu_ptr_nvhe(sym) this_cpu_ptr(&kvm_nvhe_sym(sym)) -#define per_cpu_ptr_nvhe(sym, cpu) per_cpu_ptr(&kvm_nvhe_sym(sym), cpu) +/* Array of percpu base addresses. Length of the array is nr_cpu_ids. */ +extern unsigned long *kvm_arm_hyp_percpu_base; + +/* + * Compute pointer to a symbol defined in nVHE percpu region. + * Returns NULL if percpu memory has not been allocated yet. + */ +#define this_cpu_ptr_nvhe(sym) per_cpu_ptr_nvhe(sym, smp_processor_id()) +#define per_cpu_ptr_nvhe(sym, cpu) \ + ({ \ + unsigned long base, off; \ + base = kvm_arm_hyp_percpu_base \ + ? kvm_arm_hyp_percpu_base[cpu] : 0; \ + off = (unsigned long)&kvm_nvhe_sym(sym) - \ + (unsigned long)&kvm_nvhe_sym(__per_cpu_start); \ + base ? (typeof(kvm_nvhe_sym(sym))*)(base + off) : NULL; \ + }) #ifndef __KVM_NVHE_HYPERVISOR__ /* diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index 3994169985ef..5062553a6847 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -18,5 +18,6 @@ extern char __exittext_begin[], __exittext_end[]; extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; +extern char __kvm_nvhe___per_cpu_start[], __kvm_nvhe___per_cpu_end[]; #endif /* __ASM_SECTIONS_H */ diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 7cba7623fcec..5904a4de9f40 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -15,6 +15,9 @@ #include "image.h" +#define __CONCAT3(x, y, z) x ## y ## z +#define CONCAT3(x, y, z) __CONCAT3(x, y, z) + OUTPUT_ARCH(aarch64) ENTRY(_text) @@ -191,6 +194,13 @@ SECTIONS PERCPU_SECTION(L1_CACHE_BYTES) + /* KVM nVHE per-cpu section */ + #undef PERCPU_SECTION_NAME + #undef PERCPU_SYMBOL_NAME + #define PERCPU_SECTION_NAME(suffix) CONCAT3(.hyp, PERCPU_SECTION_BASE_NAME, suffix) + #define PERCPU_SYMBOL_NAME(name) __kvm_nvhe_ ## name + PERCPU_SECTION(L1_CACHE_BYTES) + .rela.dyn : ALIGN(8) { *(.rela .rela*) } diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 8a1fcf4dffca..df7d133056ce 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -47,6 +47,7 @@ __asm__(".arch_extension virt"); #endif static DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); +unsigned long *kvm_arm_hyp_percpu_base; /* The VMID used in the VTTBR */ static atomic64_t kvm_vmid_gen = ATOMIC64_INIT(1); @@ -1255,6 +1256,15 @@ long kvm_arch_vm_ioctl(struct file *filp, } } +#define kvm_hyp_percpu_base(cpu) ((unsigned long)per_cpu_ptr_nvhe(__per_cpu_start, cpu)) +#define kvm_hyp_percpu_array_size (nr_cpu_ids * sizeof(*kvm_arm_hyp_percpu_base)) +#define kvm_hyp_percpu_array_order (get_order(kvm_hyp_percpu_array_size)) +#define kvm_hyp_percpu_start (CHOOSE_NVHE_SYM(__per_cpu_start)) +#define kvm_hyp_percpu_end (CHOOSE_NVHE_SYM(__per_cpu_end)) +#define kvm_hyp_percpu_size ((unsigned long)kvm_hyp_percpu_end - \ + (unsigned long)kvm_hyp_percpu_start) +#define kvm_hyp_percpu_order (kvm_hyp_percpu_size ? get_order(kvm_hyp_percpu_size) : 0) + static void cpu_init_hyp_mode(void) { phys_addr_t pgd_ptr; @@ -1270,8 +1280,8 @@ static void cpu_init_hyp_mode(void) * kernel's mapping to the linear mapping, and store it in tpidr_el2 * so that we can use adr_l to access per-cpu variables in EL2. */ - tpidr_el2 = ((unsigned long)this_cpu_ptr(&kvm_host_data) - - (unsigned long)kvm_ksym_ref(&kvm_host_data)); + tpidr_el2 = (unsigned long)this_cpu_ptr_nvhe(__per_cpu_start) - + (unsigned long)kvm_ksym_ref(kvm_hyp_percpu_start); pgd_ptr = kvm_mmu_get_httbr(); hyp_stack_ptr = __this_cpu_read(kvm_arm_hyp_stack_page) + PAGE_SIZE; @@ -1503,8 +1513,11 @@ static void teardown_hyp_mode(void) int cpu; free_hyp_pgds(); - for_each_possible_cpu(cpu) + for_each_possible_cpu(cpu) { free_hyp_page(per_cpu(kvm_arm_hyp_stack_page, cpu)); + free_hyp_pages(kvm_hyp_percpu_base(cpu), kvm_hyp_percpu_order); + } + free_hyp_pages((unsigned long)kvm_arm_hyp_percpu_base, kvm_hyp_percpu_array_order); } /** @@ -1537,6 +1550,28 @@ static int init_hyp_mode(void) per_cpu(kvm_arm_hyp_stack_page, cpu) = stack_page; } + /* + * Allocate and initialize pages for Hypervisor-mode percpu regions. + */ + kvm_arm_hyp_percpu_base = (unsigned long *)alloc_hyp_pages( + GFP_KERNEL | __GFP_ZERO, kvm_hyp_percpu_array_order); + if (!kvm_arm_hyp_percpu_base) { + err = -ENOMEM; + goto out_err; + } + for_each_possible_cpu(cpu) { + unsigned long percpu_base; + + percpu_base = alloc_hyp_pages(GFP_KERNEL, kvm_hyp_percpu_order); + if (!percpu_base) { + err = -ENOMEM; + goto out_err; + } + + memcpy((void *)percpu_base, kvm_hyp_percpu_start, kvm_hyp_percpu_size); + kvm_arm_hyp_percpu_base[cpu] = percpu_base; + } + /* * Map the Hyp-code called directly from the host */ @@ -1581,6 +1616,21 @@ static int init_hyp_mode(void) } } + /* + * Map Hyp percpu pages + */ + for_each_possible_cpu(cpu) { + char *percpu_begin = (char *)kvm_arm_hyp_percpu_base[cpu]; + char *percpu_end = percpu_begin + PAGE_ALIGN(kvm_hyp_percpu_size); + + err = create_hyp_mappings(percpu_begin, percpu_end, PAGE_HYP); + + if (err) { + kvm_err("Cannot map hyp percpu region\n"); + goto out_err; + } + } + for_each_possible_cpu(cpu) { kvm_host_data_t *cpu_data; diff --git a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S index aaa0ce133a32..7d8c3fa004f4 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp.lds.S +++ b/arch/arm64/kvm/hyp/nvhe/hyp.lds.S @@ -11,4 +11,9 @@ SECTIONS { HYP_SECTION(.text) + HYP_SECTION(.data..percpu) + HYP_SECTION(.data..percpu..first) + HYP_SECTION(.data..percpu..page_aligned) + HYP_SECTION(.data..percpu..read_mostly) + HYP_SECTION(.data..percpu..shared_aligned) } diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c index 6d80ffe1ebfc..cd653091f213 100644 --- a/arch/arm64/kvm/pmu.c +++ b/arch/arm64/kvm/pmu.c @@ -33,7 +33,7 @@ void kvm_set_pmu_events(u32 set, struct perf_event_attr *attr) { struct kvm_host_data *ctx = this_cpu_ptr_hyp(kvm_host_data); - if (!kvm_pmu_switch_needed(attr)) + if (!ctx || !kvm_pmu_switch_needed(attr)) return; if (!attr->exclude_host) @@ -49,6 +49,9 @@ void kvm_clr_pmu_events(u32 clr) { struct kvm_host_data *ctx = this_cpu_ptr_hyp(kvm_host_data); + if (!ctx) + return; + ctx->pmu_events.events_host &= ~clr; ctx->pmu_events.events_guest &= ~clr; } From patchwork Thu Sep 3 09:17:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Brazdil X-Patchwork-Id: 11752975 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 500F3739 for ; Thu, 3 Sep 2020 09:20:30 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2628520716 for ; Thu, 3 Sep 2020 09:20:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="F8eYqAxh"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="WGeAJLDM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2628520716 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=uyBqNnFiqFcBY/nxkb2ttM7OsHYNO/+jijdWzbdTZ+g=; b=F8eYqAxhJVCKzwwMB6SOklJ8B u8zQUU6sgMLEGwpl3UU7flLPkIqU9Fw+4pCLx5VM59V4kcNd7i/bERtcDW0w154Or8z0BtYug52S8 kwgJeeMymhbUoAbDha80xtX065TgfKNsmVGaQsBZ+oZLlVvdKxIcTjt6ayqZU+1jXpe7/Dsu4FsGB UT91x8EPvDNKosNLYa0Ru/SE04TJe77x7u6Blb8EXecY0qVwlc6603fx4t59Nci67c/ndlAucv3oc XhUwd/45xkh5ntaAaB2A4bYSj9qt+bgoVCnMbLHmk+HU5BUvC9YbC8j4hno4o59SP72TzigKhKl3/ zidVOi8Gg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlOM-0004ZO-4Y; Thu, 03 Sep 2020 09:18:50 +0000 Received: from mail-ej1-x642.google.com ([2a00:1450:4864:20::642]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kDlNa-0004E5-Ky for linux-arm-kernel@lists.infradead.org; Thu, 03 Sep 2020 09:18:11 +0000 Received: by mail-ej1-x642.google.com with SMTP id p9so2824556ejf.6 for ; Thu, 03 Sep 2020 02:18:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NQv+tl4uppeFkb8OE5qw190d9LgQJYKQj0zpEykuMCM=; b=WGeAJLDMsRVumT9Q9ux6Nast3r0JswgFOC70ZutZNZBpf4GDOxF1Vq6GaxB6cpDZ45 Ubq1CjFd9RyDJye5zoVo0+QI21IJuR8D5K5NTSCX0bfI9NOKTkGvpRSdqjbbeU8lOgnD tJC8vPxb//wpf1qUbLuSg0l8hrW/1r/jrioS74dOFucWmTNYEf6D0j0FI538lwVoZMfU 6nmWYHKuc7tfuY983RwJ7bm/dKm2mVyjPAIdqZoonp/ofsHcYNliONbyDgPpLGYp45Oa CHQHOnDIGMvH+AX91mPbZluIWwNyPlz47HjPqLMi0Ot2VIwY+bVwfHvG9y4br0BODeiC d9rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NQv+tl4uppeFkb8OE5qw190d9LgQJYKQj0zpEykuMCM=; b=oQdFoFcUCmipX1OfyEa6OZRswDEocgntxVXBIGwhZAjbDTFPypod2t6HAJUVgitjIq ka2Ll/YIT1vxTJnJhktIxT8MZXXCGd8/DN08wsr2T3WFWGBSipy4QnNoC1vewIEf3SpG d9P5fxN+FqDywsHDwNq1+grqRUX2l15TwMv/2tpMGgBiI1JpYVEtA6zuFB+ev3K/yjul wFz1K+qKVPifQRa57mwEgRka0sRBFLs7GwC1RjVTEZIgeqgzpQz8S5YKWtj3gAiaJhbq f8SWBbTbPGZ79T8iZztU5E1QoWGk+Y0gY0baUVTe4+IfKE6KMXsoMK3QL46ohiBjJyWz ZG2g== X-Gm-Message-State: AOAM530q+kl0MU8SX+39FEHWmXlukNJo/m9QOblH+gha5T4JFNsX86TZ QG2TDbY6NAm4SJOxxx0MiCu7xQ== X-Google-Smtp-Source: ABdhPJwKWw1WUu0SdgKqUcKkconNR0zceARFGVkjySt2UpxXg4FXus55KxeqDNsvsTsSugbaugIAoQ== X-Received: by 2002:a17:906:3ac4:: with SMTP id z4mr1065800ejd.65.1599124681204; Thu, 03 Sep 2020 02:18:01 -0700 (PDT) Received: from localhost (dynamic-2a00-1028-919a-a06e-64ac-0036-822c-68d3.ipv6.broadband.iol.cz. [2a00:1028:919a:a06e:64ac:36:822c:68d3]) by smtp.gmail.com with ESMTPSA id x25sm2429769edv.42.2020.09.03.02.17.59 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 03 Sep 2020 02:18:00 -0700 (PDT) From: David Brazdil To: Marc Zyngier , Catalin Marinas , Will Deacon , Dennis Zhou , Tejun Heo , Christoph Lameter , Arnd Bergmann Subject: [PATCH v2 10/10] kvm: arm64: Remove unnecessary hyp mappings Date: Thu, 3 Sep 2020 11:17:12 +0200 Message-Id: <20200903091712.46456-11-dbrazdil@google.com> X-Mailer: git-send-email 2.28.0 In-Reply-To: <20200903091712.46456-1-dbrazdil@google.com> References: <20200903091712.46456-1-dbrazdil@google.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200903_051802_791693_4899A4D9 X-CRM114-Status: GOOD ( 17.75 ) X-Spam-Score: -16.2 (----------------) X-Spam-Report: SpamAssassin version 3.4.4 on merlin.infradead.org summary: Content analysis details: (-16.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2a00:1450:4864:20:0:0:0:642 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -7.5 USER_IN_DEF_SPF_WL From: address is in the default SPF white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.5 ENV_AND_HDR_SPF_MATCH Env and Hdr From used in default SPF WL Match -0.5 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, Suzuki K Poulose , linux-kernel@vger.kernel.org, James Morse , Julien Thierry , David Brazdil , kernel-team@android.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org With all nVHE per-CPU variables being part of the hyp per-CPU region, mapping them individual is not necessary any longer. They are mapped to hyp as part of the overall per-CPU region. Signed-off-by: David Brazdil Acked-by: Andrew Scull --- arch/arm64/include/asm/kvm_mmu.h | 25 +++++++------------------ arch/arm64/kvm/arm.c | 17 +---------------- 2 files changed, 8 insertions(+), 34 deletions(-) diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 9db93da35606..bbe9df76ff42 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -531,28 +531,17 @@ static inline int kvm_map_vectors(void) DECLARE_PER_CPU_READ_MOSTLY(u64, arm64_ssbd_callback_required); DECLARE_KVM_NVHE_PER_CPU(u64, arm64_ssbd_callback_required); -static inline int hyp_init_aux_data(void) +static inline void hyp_init_aux_data(void) { - int cpu, err; + int cpu; - for_each_possible_cpu(cpu) { - u64 *ptr; - - ptr = per_cpu_ptr_nvhe(arm64_ssbd_callback_required, cpu); - err = create_hyp_mappings(ptr, ptr + 1, PAGE_HYP); - if (err) - return err; - - /* Copy value from kernel to hyp. */ - *ptr = per_cpu(arm64_ssbd_callback_required, cpu); - } - return 0; + /* Copy arm64_ssbd_callback_required values from kernel to hyp. */ + for_each_possible_cpu(cpu) + *(per_cpu_ptr_nvhe(arm64_ssbd_callback_required, cpu)) = + per_cpu(arm64_ssbd_callback_required, cpu); } #else -static inline int hyp_init_aux_data(void) -{ - return 0; -} +static inline void hyp_init_aux_data(void) {} #endif #define kvm_phys_to_vttbr(addr) phys_to_ttbr(addr) diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index df7d133056ce..dfe1baa5bbb7 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1631,22 +1631,7 @@ static int init_hyp_mode(void) } } - for_each_possible_cpu(cpu) { - kvm_host_data_t *cpu_data; - - cpu_data = per_cpu_ptr_hyp(kvm_host_data, cpu); - err = create_hyp_mappings(cpu_data, cpu_data + 1, PAGE_HYP); - - if (err) { - kvm_err("Cannot map host CPU state: %d\n", err); - goto out_err; - } - } - - err = hyp_init_aux_data(); - if (err) - kvm_err("Cannot map host auxiliary data: %d\n", err); - + hyp_init_aux_data(); return 0; out_err: