| Message ID | 20230324055127.2228330-1-anshuman.khandual@arm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | arm64: Disable EL2 traps for BRBE instructions executed in EL1 | expand |
On Fri, Mar 24, 2023 at 11:21:27AM +0530, Anshuman Khandual wrote: > This disables EL2 traps for BRBE instructions executed in EL1. This would > enable BRBE to be configured and used successfully in the guest kernel. We should probably update booting.rst too.
Hi Anshuman,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on arm64/for-next/core]
[also build test ERROR on arm/for-next arm/fixes kvmarm/next soc/for-next xilinx-xlnx/master linus/master v6.3-rc3 next-20230324]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Anshuman-Khandual/arm64-Disable-EL2-traps-for-BRBE-instructions-executed-in-EL1/20230324-135312
base: https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core
patch link: https://lore.kernel.org/r/20230324055127.2228330-1-anshuman.khandual%40arm.com
patch subject: [PATCH] arm64: Disable EL2 traps for BRBE instructions executed in EL1
config: arm64-allnoconfig (https://download.01.org/0day-ci/archive/20230325/202303250356.K6CxtK2u-lkp@intel.com/config)
compiler: aarch64-linux-gcc (GCC) 12.1.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/457bbac085c7f83be1a63b54f6caf32b820503eb
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Anshuman-Khandual/arm64-Disable-EL2-traps-for-BRBE-instructions-executed-in-EL1/20230324-135312
git checkout 457bbac085c7f83be1a63b54f6caf32b820503eb
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=arm64 olddefconfig
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=arm64 SHELL=/bin/bash arch/arm64/
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
| Link: https://lore.kernel.org/oe-kbuild-all/202303250356.K6CxtK2u-lkp@intel.com/
All errors (new ones prefixed by >>):
In file included from arch/arm64/kernel/head.S:27:
arch/arm64/include/asm/kernel-pgtable.h:126:41: warning: "PUD_SHIFT" is not defined, evaluates to 0 [-Wundef]
126 | #define ARM64_MEMSTART_SHIFT PUD_SHIFT
| ^~~~~~~~~
arch/arm64/include/asm/kernel-pgtable.h:139:5: note: in expansion of macro 'ARM64_MEMSTART_SHIFT'
139 | #if ARM64_MEMSTART_SHIFT < SECTION_SIZE_BITS
| ^~~~~~~~~~~~~~~~~~~~
arch/arm64/kernel/head.S: Assembler messages:
>> arch/arm64/kernel/head.S:577: Error: undefined symbol HFGITR_EL2_nBRBIALL used as an immediate value
>> arch/arm64/kernel/head.S:577: Error: undefined symbol HFGITR_EL2_nBRBINJ used as an immediate value
vim +577 arch/arm64/kernel/head.S
034edabe6cf1d0 Laura Abbott 2014-11-21 526
034edabe6cf1d0 Laura Abbott 2014-11-21 527 /*
034edabe6cf1d0 Laura Abbott 2014-11-21 528 * end early head section, begin head code that is also used for
034edabe6cf1d0 Laura Abbott 2014-11-21 529 * hotplug and needs to have the same protections as the text region
034edabe6cf1d0 Laura Abbott 2014-11-21 530 */
d54170812ef1c8 Mark Rutland 2023-02-20 531 .section ".idmap.text","a"
f80fb3a3d50843 Ard Biesheuvel 2016-01-26 532
9703d9d7f77ce1 Catalin Marinas 2012-03-05 533 /*
ecbb11ab3ebc02 Mark Rutland 2020-11-13 534 * Starting from EL2 or EL1, configure the CPU to execute at the highest
ecbb11ab3ebc02 Mark Rutland 2020-11-13 535 * reachable EL supported by the kernel in a chosen default state. If dropping
ecbb11ab3ebc02 Mark Rutland 2020-11-13 536 * from EL2 to EL1, configure EL2 before configuring EL1.
828e9834e9a5b7 Matthew Leach 2013-10-11 537 *
d87a8e65b51011 Mark Rutland 2020-11-13 538 * Since we cannot always rely on ERET synchronizing writes to sysregs (e.g. if
d87a8e65b51011 Mark Rutland 2020-11-13 539 * SCTLR_ELx.EOS is clear), we place an ISB prior to ERET.
828e9834e9a5b7 Matthew Leach 2013-10-11 540 *
b65e411d6cc2f1 Marc Zyngier 2022-06-30 541 * Returns either BOOT_CPU_MODE_EL1 or BOOT_CPU_MODE_EL2 in x0 if
b65e411d6cc2f1 Marc Zyngier 2022-06-30 542 * booted in EL1 or EL2 respectively, with the top 32 bits containing
b65e411d6cc2f1 Marc Zyngier 2022-06-30 543 * potential context flags. These flags are *not* stored in __boot_cpu_mode.
3dcf60bbfd284e Ard Biesheuvel 2023-01-11 544 *
3dcf60bbfd284e Ard Biesheuvel 2023-01-11 545 * x0: whether we are being called from the primary boot path with the MMU on
9703d9d7f77ce1 Catalin Marinas 2012-03-05 546 */
ecbb11ab3ebc02 Mark Rutland 2020-11-13 547 SYM_FUNC_START(init_kernel_el)
3dcf60bbfd284e Ard Biesheuvel 2023-01-11 548 mrs x1, CurrentEL
3dcf60bbfd284e Ard Biesheuvel 2023-01-11 549 cmp x1, #CurrentEL_EL2
d87a8e65b51011 Mark Rutland 2020-11-13 550 b.eq init_el2
d87a8e65b51011 Mark Rutland 2020-11-13 551
d87a8e65b51011 Mark Rutland 2020-11-13 552 SYM_INNER_LABEL(init_el1, SYM_L_LOCAL)
31a32b49b80f79 Marc Zyngier 2021-04-08 553 mov_q x0, INIT_SCTLR_EL1_MMU_OFF
9d7c13e5dde312 Ard Biesheuvel 2023-01-11 554 pre_disable_mmu_workaround
31a32b49b80f79 Marc Zyngier 2021-04-08 555 msr sctlr_el1, x0
9cf71728931a40 Matthew Leach 2013-10-11 556 isb
d87a8e65b51011 Mark Rutland 2020-11-13 557 mov_q x0, INIT_PSTATE_EL1
d87a8e65b51011 Mark Rutland 2020-11-13 558 msr spsr_el1, x0
d87a8e65b51011 Mark Rutland 2020-11-13 559 msr elr_el1, lr
d87a8e65b51011 Mark Rutland 2020-11-13 560 mov w0, #BOOT_CPU_MODE_EL1
d87a8e65b51011 Mark Rutland 2020-11-13 561 eret
9703d9d7f77ce1 Catalin Marinas 2012-03-05 562
d87a8e65b51011 Mark Rutland 2020-11-13 563 SYM_INNER_LABEL(init_el2, SYM_L_LOCAL)
3dcf60bbfd284e Ard Biesheuvel 2023-01-11 564 msr elr_el2, lr
3dcf60bbfd284e Ard Biesheuvel 2023-01-11 565
3dcf60bbfd284e Ard Biesheuvel 2023-01-11 566 // clean all HYP code to the PoC if we booted at EL2 with the MMU on
3dcf60bbfd284e Ard Biesheuvel 2023-01-11 567 cbz x0, 0f
3dcf60bbfd284e Ard Biesheuvel 2023-01-11 568 adrp x0, __hyp_idmap_text_start
3dcf60bbfd284e Ard Biesheuvel 2023-01-11 569 adr_l x1, __hyp_text_end
d54170812ef1c8 Mark Rutland 2023-02-20 570 adr_l x2, dcache_clean_poc
d54170812ef1c8 Mark Rutland 2023-02-20 571 blr x2
3dcf60bbfd284e Ard Biesheuvel 2023-01-11 572 0:
78869f0f0552d0 David Brazdil 2020-12-02 573 mov_q x0, HCR_HOST_NVHE_FLAGS
78869f0f0552d0 David Brazdil 2020-12-02 574 msr hcr_el2, x0
22043a3c082a58 Dave Martin 2017-10-31 575 isb
78869f0f0552d0 David Brazdil 2020-12-02 576
e2df464173f0b5 Marc Zyngier 2021-02-08 @577 init_el2_state
22043a3c082a58 Dave Martin 2017-10-31 578
712c6ff4dba491 Marc Zyngier 2012-10-19 579 /* Hypervisor stub */
78869f0f0552d0 David Brazdil 2020-12-02 580 adr_l x0, __hyp_stub_vectors
712c6ff4dba491 Marc Zyngier 2012-10-19 581 msr vbar_el2, x0
d87a8e65b51011 Mark Rutland 2020-11-13 582 isb
78869f0f0552d0 David Brazdil 2020-12-02 583
ae4b7e38e9a947 Marc Zyngier 2022-06-30 584 mov_q x1, INIT_SCTLR_EL1_MMU_OFF
ae4b7e38e9a947 Marc Zyngier 2022-06-30 585
31a32b49b80f79 Marc Zyngier 2021-04-08 586 /*
31a32b49b80f79 Marc Zyngier 2021-04-08 587 * Fruity CPUs seem to have HCR_EL2.E2H set to RES1,
31a32b49b80f79 Marc Zyngier 2021-04-08 588 * making it impossible to start in nVHE mode. Is that
31a32b49b80f79 Marc Zyngier 2021-04-08 589 * compliant with the architecture? Absolutely not!
31a32b49b80f79 Marc Zyngier 2021-04-08 590 */
31a32b49b80f79 Marc Zyngier 2021-04-08 591 mrs x0, hcr_el2
31a32b49b80f79 Marc Zyngier 2021-04-08 592 and x0, x0, #HCR_E2H
31a32b49b80f79 Marc Zyngier 2021-04-08 593 cbz x0, 1f
31a32b49b80f79 Marc Zyngier 2021-04-08 594
ae4b7e38e9a947 Marc Zyngier 2022-06-30 595 /* Set a sane SCTLR_EL1, the VHE way */
9d7c13e5dde312 Ard Biesheuvel 2023-01-11 596 pre_disable_mmu_workaround
ae4b7e38e9a947 Marc Zyngier 2022-06-30 597 msr_s SYS_SCTLR_EL12, x1
ae4b7e38e9a947 Marc Zyngier 2022-06-30 598 mov x2, #BOOT_CPU_FLAG_E2H
ae4b7e38e9a947 Marc Zyngier 2022-06-30 599 b 2f
31a32b49b80f79 Marc Zyngier 2021-04-08 600
31a32b49b80f79 Marc Zyngier 2021-04-08 601 1:
9d7c13e5dde312 Ard Biesheuvel 2023-01-11 602 pre_disable_mmu_workaround
ae4b7e38e9a947 Marc Zyngier 2022-06-30 603 msr sctlr_el1, x1
ae4b7e38e9a947 Marc Zyngier 2022-06-30 604 mov x2, xzr
ae4b7e38e9a947 Marc Zyngier 2022-06-30 605 2:
d87a8e65b51011 Mark Rutland 2020-11-13 606 mov w0, #BOOT_CPU_MODE_EL2
ae4b7e38e9a947 Marc Zyngier 2022-06-30 607 orr x0, x0, x2
9703d9d7f77ce1 Catalin Marinas 2012-03-05 608 eret
ecbb11ab3ebc02 Mark Rutland 2020-11-13 609 SYM_FUNC_END(init_kernel_el)
9703d9d7f77ce1 Catalin Marinas 2012-03-05 610
9703d9d7f77ce1 Catalin Marinas 2012-03-05 611 /*
9703d9d7f77ce1 Catalin Marinas 2012-03-05 612 * This provides a "holding pen" for platforms to hold all secondary
9703d9d7f77ce1 Catalin Marinas 2012-03-05 613 * cores are held until we're ready for them to initialise.
9703d9d7f77ce1 Catalin Marinas 2012-03-05 614 */
c63d9f82db9439 Mark Brown 2020-02-18 615 SYM_FUNC_START(secondary_holding_pen)
3dcf60bbfd284e Ard Biesheuvel 2023-01-11 616 mov x0, xzr
ecbb11ab3ebc02 Mark Rutland 2020-11-13 617 bl init_kernel_el // w0=cpu_boot_mode
005e12676af09a Ard Biesheuvel 2022-06-24 618 mrs x2, mpidr_el1
b03cc885328e3c Ard Biesheuvel 2016-04-18 619 mov_q x1, MPIDR_HWID_BITMASK
005e12676af09a Ard Biesheuvel 2022-06-24 620 and x2, x2, x1
b1c98297fe0c6e Ard Biesheuvel 2015-03-10 621 adr_l x3, secondary_holding_pen_release
9703d9d7f77ce1 Catalin Marinas 2012-03-05 622 pen: ldr x4, [x3]
005e12676af09a Ard Biesheuvel 2022-06-24 623 cmp x4, x2
9703d9d7f77ce1 Catalin Marinas 2012-03-05 624 b.eq secondary_startup
9703d9d7f77ce1 Catalin Marinas 2012-03-05 625 wfe
9703d9d7f77ce1 Catalin Marinas 2012-03-05 626 b pen
c63d9f82db9439 Mark Brown 2020-02-18 627 SYM_FUNC_END(secondary_holding_pen)
652af899799354 Mark Rutland 2013-10-24 628
652af899799354 Mark Rutland 2013-10-24 629 /*
652af899799354 Mark Rutland 2013-10-24 630 * Secondary entry point that jumps straight into the kernel. Only to
652af899799354 Mark Rutland 2013-10-24 631 * be used where CPUs are brought online dynamically by the kernel.
652af899799354 Mark Rutland 2013-10-24 632 */
c63d9f82db9439 Mark Brown 2020-02-18 633 SYM_FUNC_START(secondary_entry)
3dcf60bbfd284e Ard Biesheuvel 2023-01-11 634 mov x0, xzr
ecbb11ab3ebc02 Mark Rutland 2020-11-13 635 bl init_kernel_el // w0=cpu_boot_mode
652af899799354 Mark Rutland 2013-10-24 636 b secondary_startup
c63d9f82db9439 Mark Brown 2020-02-18 637 SYM_FUNC_END(secondary_entry)
9703d9d7f77ce1 Catalin Marinas 2012-03-05 638
Hi Anshuman,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on arm64/for-next/core]
[also build test ERROR on arm/for-next arm/fixes kvmarm/next soc/for-next xilinx-xlnx/master linus/master v6.3-rc3 next-20230324]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Anshuman-Khandual/arm64-Disable-EL2-traps-for-BRBE-instructions-executed-in-EL1/20230324-135312
base: https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core
patch link: https://lore.kernel.org/r/20230324055127.2228330-1-anshuman.khandual%40arm.com
patch subject: [PATCH] arm64: Disable EL2 traps for BRBE instructions executed in EL1
config: arm64-randconfig-r001-20230322 (https://download.01.org/0day-ci/archive/20230325/202303250802.KytExIHw-lkp@intel.com/config)
compiler: clang version 17.0.0 (https://github.com/llvm/llvm-project 67409911353323ca5edf2049ef0df54132fa1ca7)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install arm64 cross compiling tool for clang build
# apt-get install binutils-aarch64-linux-gnu
# https://github.com/intel-lab-lkp/linux/commit/457bbac085c7f83be1a63b54f6caf32b820503eb
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Anshuman-Khandual/arm64-Disable-EL2-traps-for-BRBE-instructions-executed-in-EL1/20230324-135312
git checkout 457bbac085c7f83be1a63b54f6caf32b820503eb
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=arm64 olddefconfig
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=arm64 SHELL=/bin/bash
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
| Link: https://lore.kernel.org/oe-kbuild-all/202303250802.KytExIHw-lkp@intel.com/
All errors (new ones prefixed by >>):
>> <instantiation>:36:14: error: expected compatible register or logical immediate
orr x0, x0, #HFGITR_EL2_nBRBIALL
^
<instantiation>:10:2: note: while in macro instantiation
__init_el2_fgt
^
arch/arm64/kernel/head.S:577:2: note: while in macro instantiation
init_el2_state
^
<instantiation>:37:14: error: expected compatible register or logical immediate
orr x0, x0, #HFGITR_EL2_nBRBINJ
^
<instantiation>:10:2: note: while in macro instantiation
__init_el2_fgt
^
arch/arm64/kernel/head.S:577:2: note: while in macro instantiation
init_el2_state
^
On 3/24/23 17:25, Mark Brown wrote: > On Fri, Mar 24, 2023 at 11:21:27AM +0530, Anshuman Khandual wrote: > >> This disables EL2 traps for BRBE instructions executed in EL1. This would >> enable BRBE to be configured and used successfully in the guest kernel. > > We should probably update booting.rst too. Sure, will updated as required, for SYS_HFGITR_EL2's BRBE fields.
On 3/25/23 06:07, kernel test robot wrote: > All errors (new ones prefixed by >>): > >>> <instantiation>:36:14: error: expected compatible register or logical immediate > orr x0, x0, #HFGITR_EL2_nBRBIALL > ^ > <instantiation>:10:2: note: while in macro instantiation > __init_el2_fgt > ^ > arch/arm64/kernel/head.S:577:2: note: while in macro instantiation > init_el2_state > ^ > <instantiation>:37:14: error: expected compatible register or logical immediate > orr x0, x0, #HFGITR_EL2_nBRBINJ > ^ > <instantiation>:10:2: note: while in macro instantiation > __init_el2_fgt > ^ > arch/arm64/kernel/head.S:577:2: note: while in macro instantiation > init_el2_state > ^ I guess these build errors are caused by unavailability of the pre-requisite patches from Mark Brown, in this test environment ? https://lore.kernel.org/all/20230306-arm64-fgt-reg-gen-v3-0-decba93cbaab@kernel.org/
On Mon, 2023-03-27 at 11:38 +0530, Anshuman Khandual wrote: > On 3/25/23 06:07, kernel test robot wrote: > > All errors (new ones prefixed by >>): > > > > > > <instantiation>:36:14: error: expected compatible register or logical immediate > > orr x0, x0, #HFGITR_EL2_nBRBIALL > > ^ > > <instantiation>:10:2: note: while in macro instantiation > > __init_el2_fgt > > ^ > > arch/arm64/kernel/head.S:577:2: note: while in macro instantiation > > init_el2_state > > ^ > > <instantiation>:37:14: error: expected compatible register or logical immediate > > orr x0, x0, #HFGITR_EL2_nBRBINJ > > ^ > > <instantiation>:10:2: note: while in macro instantiation > > __init_el2_fgt > > ^ > > arch/arm64/kernel/head.S:577:2: note: while in macro instantiation > > init_el2_state > > ^ > > I guess these build errors are caused by unavailability of the pre-requisite > patches from Mark Brown, in this test environment ? > > https://lore.kernel.org/all/20230306-arm64-fgt-reg-gen-v3-0-decba93cbaab@kernel.org/ Thanks for the information. We noticed that the base and prerequisite patches are mentioned in the commit message: --- This patch applies on v6.3-rc3, after the following patch series from Mark https://lore.kernel.org/all/20230306-arm64-fgt-reg-gen-v3-0-decba93cbaab@kernel.org/ The robot couldn't understand above description, and it is recommended to use `git format-patch --base` to generate base info. Please refer to the documents in: https://git-scm.com/docs/git-format-patch#_base_tree_information For this case, the correct order of patches would be: arm64: Disable EL2 traps for BRBE instructions executed in EL1 arm64/sysreg: Convert HFGITR_EL2 to automatic generation arm64/sysreg: Convert HFG[RW]TR_EL2 to automatic generation e8d018dd0257 Linux 6.3-rc3 Please checkout to the patch and do `git format-patch --base e8d018dd0257 -1`, it will generate base info in the footer of the patch: base-commit: e8d018dd0257f744ca50a729e3d042cf2ec9da65 prerequisite-patch-id: 6f448444c7a40a44146d4ecbfa57190ae30a257f prerequisite-patch-id: 484c605fa9d35d776eda3c47ab95a24841712885 Then the robot can parse this info to select base and apply prerequisite patches as expected. -- Best Regards, Yujie
diff --git a/arch/arm64/include/asm/el2_setup.h b/arch/arm64/include/asm/el2_setup.h index 037724b19c5c..06bf321a17be 100644 --- a/arch/arm64/include/asm/el2_setup.h +++ b/arch/arm64/include/asm/el2_setup.h @@ -161,6 +161,16 @@ msr_s SYS_HFGWTR_EL2, x0 msr_s SYS_HFGITR_EL2, xzr + mrs x1, id_aa64dfr0_el1 + ubfx x1, x1, #ID_AA64DFR0_EL1_BRBE_SHIFT, #4 + cbz x1, .Lskip_brbe_\@ + + mov x0, xzr + orr x0, x0, #HFGITR_EL2_nBRBIALL + orr x0, x0, #HFGITR_EL2_nBRBINJ + msr_s SYS_HFGITR_EL2, x0 + +.Lskip_brbe_\@: mrs x1, id_aa64pfr0_el1 // AMU traps UNDEF without AMU ubfx x1, x1, #ID_AA64PFR0_EL1_AMU_SHIFT, #4 cbz x1, .Lskip_fgt_\@
This disables EL2 traps for BRBE instructions executed in EL1. This would enable BRBE to be configured and used successfully in the guest kernel. Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Will Deacon <will@kernel.org> Cc: Mark Brown <broonie@kernel.org> Cc: Marc Zyngier <maz@kernel.org> Cc: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com> --- This patch applies on v6.3-rc3, after the following patch series from Mark https://lore.kernel.org/all/20230306-arm64-fgt-reg-gen-v3-0-decba93cbaab@kernel.org/ arch/arm64/include/asm/el2_setup.h | 10 ++++++++++ 1 file changed, 10 insertions(+)