[v5,11/17] arm64: docs: document pointer authentication

Commit Message

Kristina Martsenko Oct. 5, 2018, 8:47 a.m. UTC

From: Mark Rutland <[e-mail]>

Now that we've added code to support pointer authentication, add some
documentation so that people can figure out if/how to use it.

Signed-off-by: Mark Rutland <[e-mail]>
[kristina: update cpu-feature-registers.txt]
Signed-off-by: Kristina Martsenko <[e-mail]>
Cc: Andrew Jones <[e-mail]>
Cc: Catalin Marinas <[e-mail]>
Cc: Ramana Radhakrishnan <[e-mail]>
Cc: Will Deacon <[e-mail]>
---
 Documentation/arm64/booting.txt                |  8 +++
 Documentation/arm64/cpu-feature-registers.txt  |  4 ++
 Documentation/arm64/elf_hwcaps.txt             |  5 ++
 Documentation/arm64/pointer-authentication.txt | 84 ++++++++++++++++++++++++++
 4 files changed, 101 insertions(+)
 create mode 100644 Documentation/arm64/pointer-authentication.txt

Comments

Ramana Radhakrishnan Oct. 5, 2018, 9:04 a.m. UTC | #1

On 05/10/2018 09:47, Kristina Martsenko wrote:
> From: Mark Rutland <[e-mail]>
> 
> Now that we've added code to support pointer authentication, add some
> documentation so that people can figure out if/how to use it.
> 
> Signed-off-by: Mark Rutland <[e-mail]>
> [kristina: update cpu-feature-registers.txt]
> Signed-off-by: Kristina Martsenko <[e-mail]>
> Cc: Andrew Jones <[e-mail]>
> Cc: Catalin Marinas <[e-mail]>
> Cc: Ramana Radhakrishnan <[e-mail]>
> Cc: Will Deacon <[e-mail]>
> ---
>   Documentation/arm64/booting.txt                |  8 +++
>   Documentation/arm64/cpu-feature-registers.txt  |  4 ++
>   Documentation/arm64/elf_hwcaps.txt             |  5 ++
>   Documentation/arm64/pointer-authentication.txt | 84 ++++++++++++++++++++++++++
>   4 files changed, 101 insertions(+)
>   create mode 100644 Documentation/arm64/pointer-authentication.txt
> 
> diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt
> index 8d0df62c3fe0..8df9f4658d6f 100644
> --- a/Documentation/arm64/booting.txt
> +++ b/Documentation/arm64/booting.txt
> @@ -205,6 +205,14 @@ Before jumping into the kernel, the following conditions must be met:
>       ICC_SRE_EL2.SRE (bit 0) must be initialised to 0b0.
>     - The DT or ACPI tables must describe a GICv2 interrupt controller.
>   
> +  For CPUs with pointer authentication functionality:
> +  - If EL3 is present:
> +    SCR_EL3.APK (bit 16) must be initialised to 0b1
> +    SCR_EL3.API (bit 17) must be initialised to 0b1
> +  - If the kernel is entered at EL1:
> +    HCR_EL2.APK (bit 40) must be initialised to 0b1
> +    HCR_EL2.API (bit 41) must be initialised to 0b1
> +
>   The requirements described above for CPU mode, caches, MMUs, architected
>   timers, coherency and system registers apply to all CPUs.  All CPUs must
>   enter the kernel in the same exception level.
> diff --git a/Documentation/arm64/cpu-feature-registers.txt b/Documentation/arm64/cpu-feature-registers.txt
> index 7964f03846b1..b165677ffab9 100644
> --- a/Documentation/arm64/cpu-feature-registers.txt
> +++ b/Documentation/arm64/cpu-feature-registers.txt
> @@ -190,6 +190,10 @@ infrastructure:
>        |--------------------------------------------------|
>        | JSCVT                        | [15-12] |    y    |
>        |--------------------------------------------------|
> +     | API                          | [11-8]  |    y    |
> +     |--------------------------------------------------|
> +     | APA                          | [7-4]   |    y    |
> +     |--------------------------------------------------|
>        | DPB                          | [3-0]   |    y    |
>        x--------------------------------------------------x
>   
> diff --git a/Documentation/arm64/elf_hwcaps.txt b/Documentation/arm64/elf_hwcaps.txt
> index d6aff2c5e9e2..95509a7b0ffe 100644
> --- a/Documentation/arm64/elf_hwcaps.txt
> +++ b/Documentation/arm64/elf_hwcaps.txt
> @@ -178,3 +178,8 @@ HWCAP_ILRCPC
>   HWCAP_FLAGM
>   
>       Functionality implied by ID_AA64ISAR0_EL1.TS == 0b0001.
> +
> +HWCAP_APIA
> +
> +    EL0 AddPac and Auth functionality using APIAKey_EL1 is enabled, as
> +    described by Documentation/arm64/pointer-authentication.txt.
> diff --git a/Documentation/arm64/pointer-authentication.txt b/Documentation/arm64/pointer-authentication.txt
> new file mode 100644
> index 000000000000..8a9cb5713770
> --- /dev/null
> +++ b/Documentation/arm64/pointer-authentication.txt
> @@ -0,0 +1,84 @@
> +Pointer authentication in AArch64 Linux
> +=======================================
> +
> +Author: Mark Rutland <[e-mail]>
> +Date: 2017-07-19
> +
> +This document briefly describes the provision of pointer authentication
> +functionality in AArch64 Linux.
> +
> +
> +Architecture overview
> +---------------------
> +
> +The ARMv8.3 Pointer Authentication extension adds primitives that can be
> +used to mitigate certain classes of attack where an attacker can corrupt
> +the contents of some memory (e.g. the stack).
> +
> +The extension uses a Pointer Authentication Code (PAC) to determine
> +whether pointers have been modified unexpectedly. A PAC is derived from
> +a pointer, another value (such as the stack pointer), and a secret key
> +held in system registers.
> +
> +The extension adds instructions to insert a valid PAC into a pointer,
> +and to verify/remove the PAC from a pointer. The PAC occupies a number
> +of high-order bits of the pointer, which varies dependent on the
> +configured virtual address size and whether pointer tagging is in use.

s/pointer tagging/top byte ignore unless that's the terminology in the 
rest of the kernel documentation ?

> +
> +A subset of these instructions have been allocated from the HINT
> +encoding space. In the absence of the extension (or when disabled),
> +these instructions behave as NOPs. Applications and libraries using
> +these instructions operate correctly regardless of the presence of the
> +extension.
> +
> +
> +Basic support
> +-------------
> +
> +When CONFIG_ARM64_PTR_AUTH is selected, and relevant HW support is
> +present, the kernel will assign a random APIAKey value to each process
> +at exec*() time. This key is shared by all threads within the process,
> +and the key is preserved across fork(). Presence of functionality using
> +APIAKey is advertised via HWCAP_APIA.
> +
> +Recent versions of GCC can compile code with APIAKey-based return
> +address protection when passed the -msign-return-address option. This
> +uses instructions in the HINT space, and such code can run on systems
> +without the pointer authentication extension.

Just a clarification.

This uses instructions in the hint space for architecture levels less 
than armv8.3-a by default. If folks use -march=armv8.3-a you will start 
seeing the combined forms of retaa appear.

> +
> +The remaining instruction and data keys (APIBKey, APDAKey, APDBKey) are
> +reserved for future use, and instructions using these keys must not be
> +used by software until a purpose and scope for their use has been
> +decided. To enable future software using these keys to function on
> +contemporary kernels, where possible, instructions using these keys are
> +made to behave as NOPs.
> +
> +The generic key (APGAKey) is currently unsupported. Instructions using
> +the generic key must not be used by software.
> +
> +
> +Debugging
> +---------
> +
> +When CONFIG_ARM64_PTR_AUTH is selected, and relevant HW support is
> +present, the kernel will expose the position of TTBR0 PAC bits in the
> +NT_ARM_PAC_MASK regset (struct user_pac_mask), which userspace can
> +acqure via PTRACE_GETREGSET.
> +
> +Separate masks are exposed for data pointers and instruction pointers,
> +as the set of PAC bits can vary between the two. Debuggers should not
> +expect that HWCAP_APIA implies the presence (or non-presence) of this
> +regset -- in future the kernel may support the use of APIBKey, APDAKey,
> +and/or APBAKey, even in the absence of APIAKey.
> +
> +Note that the masks apply to TTBR0 addresses, and are not valid to apply
> +to TTBR1 addresses (e.g. kernel pointers).
> +
> +
> +Virtualization
> +--------------
> +
> +Pointer authentication is not currently supported in KVM guests. KVM
> +will mask the feature bits from ID_AA64ISAR1_EL1, and attempted use of
> +the feature will result in an UNDEFINED exception being injected into
> +the guest.

However applications using instructions from the hint space will 
continue to work albeit without any protection (as they would just be 
nops) ?


regards,
Ramana


Reviewed-by: Ramana Radhakrishnan  <[e-mail]>

>

Kees Cook Oct. 15, 2018, 10:35 p.m. UTC | #2

On Fri, Oct 5, 2018 at 1:47 AM, Kristina Martsenko
<[e-mail]> wrote:
> From: Mark Rutland <[e-mail]>
>
> Now that we've added code to support pointer authentication, add some
> documentation so that people can figure out if/how to use it.
>
> Signed-off-by: Mark Rutland <[e-mail]>
> [kristina: update cpu-feature-registers.txt]
> Signed-off-by: Kristina Martsenko <[e-mail]>
> Cc: Andrew Jones <[e-mail]>
> Cc: Catalin Marinas <[e-mail]>
> Cc: Ramana Radhakrishnan <[e-mail]>
> Cc: Will Deacon <[e-mail]>
> ---
>  Documentation/arm64/booting.txt                |  8 +++
>  Documentation/arm64/cpu-feature-registers.txt  |  4 ++
>  Documentation/arm64/elf_hwcaps.txt             |  5 ++
>  Documentation/arm64/pointer-authentication.txt | 84 ++++++++++++++++++++++++++
>  4 files changed, 101 insertions(+)
>  create mode 100644 Documentation/arm64/pointer-authentication.txt
>
> diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt
> index 8d0df62c3fe0..8df9f4658d6f 100644
> --- a/Documentation/arm64/booting.txt
> +++ b/Documentation/arm64/booting.txt
> @@ -205,6 +205,14 @@ Before jumping into the kernel, the following conditions must be met:
>      ICC_SRE_EL2.SRE (bit 0) must be initialised to 0b0.
>    - The DT or ACPI tables must describe a GICv2 interrupt controller.
>
> +  For CPUs with pointer authentication functionality:
> +  - If EL3 is present:
> +    SCR_EL3.APK (bit 16) must be initialised to 0b1
> +    SCR_EL3.API (bit 17) must be initialised to 0b1
> +  - If the kernel is entered at EL1:
> +    HCR_EL2.APK (bit 40) must be initialised to 0b1
> +    HCR_EL2.API (bit 41) must be initialised to 0b1
> +
>  The requirements described above for CPU mode, caches, MMUs, architected
>  timers, coherency and system registers apply to all CPUs.  All CPUs must
>  enter the kernel in the same exception level.
> diff --git a/Documentation/arm64/cpu-feature-registers.txt b/Documentation/arm64/cpu-feature-registers.txt
> index 7964f03846b1..b165677ffab9 100644
> --- a/Documentation/arm64/cpu-feature-registers.txt
> +++ b/Documentation/arm64/cpu-feature-registers.txt
> @@ -190,6 +190,10 @@ infrastructure:
>       |--------------------------------------------------|
>       | JSCVT                        | [15-12] |    y    |
>       |--------------------------------------------------|
> +     | API                          | [11-8]  |    y    |
> +     |--------------------------------------------------|
> +     | APA                          | [7-4]   |    y    |
> +     |--------------------------------------------------|
>       | DPB                          | [3-0]   |    y    |
>       x--------------------------------------------------x
>
> diff --git a/Documentation/arm64/elf_hwcaps.txt b/Documentation/arm64/elf_hwcaps.txt
> index d6aff2c5e9e2..95509a7b0ffe 100644
> --- a/Documentation/arm64/elf_hwcaps.txt
> +++ b/Documentation/arm64/elf_hwcaps.txt
> @@ -178,3 +178,8 @@ HWCAP_ILRCPC
>  HWCAP_FLAGM
>
>      Functionality implied by ID_AA64ISAR0_EL1.TS == 0b0001.
> +
> +HWCAP_APIA
> +
> +    EL0 AddPac and Auth functionality using APIAKey_EL1 is enabled, as
> +    described by Documentation/arm64/pointer-authentication.txt.
> diff --git a/Documentation/arm64/pointer-authentication.txt b/Documentation/arm64/pointer-authentication.txt
> new file mode 100644
> index 000000000000..8a9cb5713770
> --- /dev/null
> +++ b/Documentation/arm64/pointer-authentication.txt
> @@ -0,0 +1,84 @@
> +Pointer authentication in AArch64 Linux
> +=======================================
> +
> +Author: Mark Rutland <[e-mail]>
> +Date: 2017-07-19
> +
> +This document briefly describes the provision of pointer authentication
> +functionality in AArch64 Linux.
> +
> +
> +Architecture overview
> +---------------------
> +
> +The ARMv8.3 Pointer Authentication extension adds primitives that can be
> +used to mitigate certain classes of attack where an attacker can corrupt
> +the contents of some memory (e.g. the stack).
> +
> +The extension uses a Pointer Authentication Code (PAC) to determine
> +whether pointers have been modified unexpectedly. A PAC is derived from
> +a pointer, another value (such as the stack pointer), and a secret key
> +held in system registers.
> +
> +The extension adds instructions to insert a valid PAC into a pointer,
> +and to verify/remove the PAC from a pointer. The PAC occupies a number
> +of high-order bits of the pointer, which varies dependent on the
> +configured virtual address size and whether pointer tagging is in use.
> +
> +A subset of these instructions have been allocated from the HINT
> +encoding space. In the absence of the extension (or when disabled),
> +these instructions behave as NOPs. Applications and libraries using
> +these instructions operate correctly regardless of the presence of the
> +extension.
> +
> +
> +Basic support
> +-------------
> +
> +When CONFIG_ARM64_PTR_AUTH is selected, and relevant HW support is
> +present, the kernel will assign a random APIAKey value to each process
> +at exec*() time. This key is shared by all threads within the process,
> +and the key is preserved across fork(). Presence of functionality using
> +APIAKey is advertised via HWCAP_APIA.

It might be useful to include documentation here on how many bits of
the address are being used for the PAC bits (I'm assuming it's 7?)

-Kees

Kristina Martsenko Oct. 16, 2018, 4:14 p.m. UTC | #3

On 05/10/2018 10:04, Ramana Radhakrishnan wrote:
> On 05/10/2018 09:47, Kristina Martsenko wrote:
>> From: Mark Rutland <[e-mail]>
>>
>> Now that we've added code to support pointer authentication, add some
>> documentation so that people can figure out if/how to use it.
>>
>> Signed-off-by: Mark Rutland <[e-mail]>
>> [kristina: update cpu-feature-registers.txt]
>> Signed-off-by: Kristina Martsenko <[e-mail]>
>> Cc: Andrew Jones <[e-mail]>
>> Cc: Catalin Marinas <[e-mail]>
>> Cc: Ramana Radhakrishnan <[e-mail]>
>> Cc: Will Deacon <[e-mail]>
>> ---
>>   Documentation/arm64/booting.txt                |  8 +++
>>   Documentation/arm64/cpu-feature-registers.txt  |  4 ++
>>   Documentation/arm64/elf_hwcaps.txt             |  5 ++
>>   Documentation/arm64/pointer-authentication.txt | 84
>> ++++++++++++++++++++++++++
>>   4 files changed, 101 insertions(+)
>>   create mode 100644 Documentation/arm64/pointer-authentication.txt
>>
>> diff --git a/Documentation/arm64/booting.txt
>> b/Documentation/arm64/booting.txt
>> index 8d0df62c3fe0..8df9f4658d6f 100644
>> --- a/Documentation/arm64/booting.txt
>> +++ b/Documentation/arm64/booting.txt
>> @@ -205,6 +205,14 @@ Before jumping into the kernel, the following
>> conditions must be met:
>>       ICC_SRE_EL2.SRE (bit 0) must be initialised to 0b0.
>>     - The DT or ACPI tables must describe a GICv2 interrupt controller.
>>   +  For CPUs with pointer authentication functionality:
>> +  - If EL3 is present:
>> +    SCR_EL3.APK (bit 16) must be initialised to 0b1
>> +    SCR_EL3.API (bit 17) must be initialised to 0b1
>> +  - If the kernel is entered at EL1:
>> +    HCR_EL2.APK (bit 40) must be initialised to 0b1
>> +    HCR_EL2.API (bit 41) must be initialised to 0b1
>> +
>>   The requirements described above for CPU mode, caches, MMUs,
>> architected
>>   timers, coherency and system registers apply to all CPUs.  All CPUs
>> must
>>   enter the kernel in the same exception level.
>> diff --git a/Documentation/arm64/cpu-feature-registers.txt
>> b/Documentation/arm64/cpu-feature-registers.txt
>> index 7964f03846b1..b165677ffab9 100644
>> --- a/Documentation/arm64/cpu-feature-registers.txt
>> +++ b/Documentation/arm64/cpu-feature-registers.txt
>> @@ -190,6 +190,10 @@ infrastructure:
>>        |--------------------------------------------------|
>>        | JSCVT                        | [15-12] |    y    |
>>        |--------------------------------------------------|
>> +     | API                          | [11-8]  |    y    |
>> +     |--------------------------------------------------|
>> +     | APA                          | [7-4]   |    y    |
>> +     |--------------------------------------------------|
>>        | DPB                          | [3-0]   |    y    |
>>        x--------------------------------------------------x
>>   diff --git a/Documentation/arm64/elf_hwcaps.txt
>> b/Documentation/arm64/elf_hwcaps.txt
>> index d6aff2c5e9e2..95509a7b0ffe 100644
>> --- a/Documentation/arm64/elf_hwcaps.txt
>> +++ b/Documentation/arm64/elf_hwcaps.txt
>> @@ -178,3 +178,8 @@ HWCAP_ILRCPC
>>   HWCAP_FLAGM
>>         Functionality implied by ID_AA64ISAR0_EL1.TS == 0b0001.
>> +
>> +HWCAP_APIA
>> +
>> +    EL0 AddPac and Auth functionality using APIAKey_EL1 is enabled, as
>> +    described by Documentation/arm64/pointer-authentication.txt.
>> diff --git a/Documentation/arm64/pointer-authentication.txt
>> b/Documentation/arm64/pointer-authentication.txt
>> new file mode 100644
>> index 000000000000..8a9cb5713770
>> --- /dev/null
>> +++ b/Documentation/arm64/pointer-authentication.txt
>> @@ -0,0 +1,84 @@
>> +Pointer authentication in AArch64 Linux
>> +=======================================
>> +
>> +Author: Mark Rutland <[e-mail]>
>> +Date: 2017-07-19
>> +
>> +This document briefly describes the provision of pointer authentication
>> +functionality in AArch64 Linux.
>> +
>> +
>> +Architecture overview
>> +---------------------
>> +
>> +The ARMv8.3 Pointer Authentication extension adds primitives that can be
>> +used to mitigate certain classes of attack where an attacker can corrupt
>> +the contents of some memory (e.g. the stack).
>> +
>> +The extension uses a Pointer Authentication Code (PAC) to determine
>> +whether pointers have been modified unexpectedly. A PAC is derived from
>> +a pointer, another value (such as the stack pointer), and a secret key
>> +held in system registers.
>> +
>> +The extension adds instructions to insert a valid PAC into a pointer,
>> +and to verify/remove the PAC from a pointer. The PAC occupies a number
>> +of high-order bits of the pointer, which varies dependent on the
>> +configured virtual address size and whether pointer tagging is in use.
> 
> s/pointer tagging/top byte ignore unless that's the terminology in the
> rest of the kernel documentation ?

The rest of the kernel documentation calls them "tagged pointers", and
doesn't use "top byte ignore", for example
Documentation/arm64/tagged-pointers.txt:
https://elixir.bootlin.com/linux/latest/source/Documentation/arm64/tagged-pointers.txt

> 
>> +
>> +A subset of these instructions have been allocated from the HINT
>> +encoding space. In the absence of the extension (or when disabled),
>> +these instructions behave as NOPs. Applications and libraries using
>> +these instructions operate correctly regardless of the presence of the
>> +extension.
>> +
>> +
>> +Basic support
>> +-------------
>> +
>> +When CONFIG_ARM64_PTR_AUTH is selected, and relevant HW support is
>> +present, the kernel will assign a random APIAKey value to each process
>> +at exec*() time. This key is shared by all threads within the process,
>> +and the key is preserved across fork(). Presence of functionality using
>> +APIAKey is advertised via HWCAP_APIA.
>> +
>> +Recent versions of GCC can compile code with APIAKey-based return
>> +address protection when passed the -msign-return-address option. This
>> +uses instructions in the HINT space, and such code can run on systems
>> +without the pointer authentication extension.
> 
> Just a clarification.
> 
> This uses instructions in the hint space for architecture levels less
> than armv8.3-a by default. If folks use -march=armv8.3-a you will start
> seeing the combined forms of retaa appear.

I'll amend this to:

"This uses instructions in the HINT space (unless -march=armv8.3-a or
higher is also passed), and such code can run on systems without the
pointer authentication extension."

> 
>> +
>> +The remaining instruction and data keys (APIBKey, APDAKey, APDBKey) are
>> +reserved for future use, and instructions using these keys must not be
>> +used by software until a purpose and scope for their use has been
>> +decided. To enable future software using these keys to function on
>> +contemporary kernels, where possible, instructions using these keys are
>> +made to behave as NOPs.
>> +
>> +The generic key (APGAKey) is currently unsupported. Instructions using
>> +the generic key must not be used by software.
>> +
>> +
>> +Debugging
>> +---------
>> +
>> +When CONFIG_ARM64_PTR_AUTH is selected, and relevant HW support is
>> +present, the kernel will expose the position of TTBR0 PAC bits in the
>> +NT_ARM_PAC_MASK regset (struct user_pac_mask), which userspace can
>> +acqure via PTRACE_GETREGSET.
>> +
>> +Separate masks are exposed for data pointers and instruction pointers,
>> +as the set of PAC bits can vary between the two. Debuggers should not
>> +expect that HWCAP_APIA implies the presence (or non-presence) of this
>> +regset -- in future the kernel may support the use of APIBKey, APDAKey,
>> +and/or APBAKey, even in the absence of APIAKey.
>> +
>> +Note that the masks apply to TTBR0 addresses, and are not valid to apply
>> +to TTBR1 addresses (e.g. kernel pointers).
>> +
>> +
>> +Virtualization
>> +--------------
>> +
>> +Pointer authentication is not currently supported in KVM guests. KVM
>> +will mask the feature bits from ID_AA64ISAR1_EL1, and attempted use of
>> +the feature will result in an UNDEFINED exception being injected into
>> +the guest.
> 
> However applications using instructions from the hint space will
> continue to work albeit without any protection (as they would just be
> nops) ?

Mostly, yes. If the guest leaves SCTLR_EL1.EnIA unset (and
EnIB/EnDA/EnDB), then PAC* and AUT* instructions in the HINT space will
execute as NOPs. If the guest sets EnIA, then PAC*/AUT* instructions
will trap and KVM will inject an "Unknown reason" exception into the
guest (which will cause a Linux guest to send a SIGILL to the application).

In the latter case we could instead pretend the instruction was a NOP
and not inject an exception, but trapping twice per every function would
probably be terrible for performance. The guest shouldn't be setting
EnIA anyway if ID_AA64ISAR1_EL1 reports that pointer authentication is
not present (because KVM has hidden it).

The other special case is the XPACLRI instruction, which is also in the
HINT space. Currently it will trap and KVM will inject an exception into
the guest. We should probably change this to NOP instead, as that's what
applications will expect. Unfortunately there is no EnIA-like control to
make it NOP.

One option is for KVM to pretend the instruction was a NOP and return to
the guest. But if XPACLRI gets executed frequently, then the constant
trapping might hurt performance. I don't know how frequently it might
get used, as I don't know of any applications currently using it. From
what I understand, it may be used by userspace stack unwinders.

(Also worth noting - as far as I can tell there is no easy way for KVM
to know which pointer authentication instruction caused the trap, so we
may have to do something unusual like use "at s12e1r" to read guest
memory and check for XPACLRI.)

The other option is to turn off trapping entirely. However then on a
big.LITTLE system with mismatched pointer authentication support
instructions will work intermittently on some CPUs but not others.

Thoughts?

> 
> Reviewed-by: Ramana Radhakrishnan  <[e-mail]>

Thanks!

Kristina

Catalin Marinas Oct. 19, 2018, 11:35 a.m. UTC | #4

On Tue, Oct 16, 2018 at 05:14:39PM +0100, Kristina Martsenko wrote:
> On 05/10/2018 10:04, Ramana Radhakrishnan wrote:
> > On 05/10/2018 09:47, Kristina Martsenko wrote:
> >> +Virtualization
> >> +--------------
> >> +
> >> +Pointer authentication is not currently supported in KVM guests. KVM
> >> +will mask the feature bits from ID_AA64ISAR1_EL1, and attempted use of
> >> +the feature will result in an UNDEFINED exception being injected into
> >> +the guest.
> > 
> > However applications using instructions from the hint space will
> > continue to work albeit without any protection (as they would just be
> > nops) ?
> 
> Mostly, yes. If the guest leaves SCTLR_EL1.EnIA unset (and
> EnIB/EnDA/EnDB), then PAC* and AUT* instructions in the HINT space will
> execute as NOPs. If the guest sets EnIA, then PAC*/AUT* instructions
> will trap and KVM will inject an "Unknown reason" exception into the
> guest (which will cause a Linux guest to send a SIGILL to the application).

I think that part is fine. If KVM (a fairly recent version with CPUID
sanitisation) does not enable ptr auth, the CPUID should not advertise
this feature either so the guest kernel should not enable it. For the
above instructions in the HINT space, they will just be NOPs. If the
guest kernel enables the feature regardless of the CPUID information, it
deserves to get an "Unknown reason" exception.

> In the latter case we could instead pretend the instruction was a NOP
> and not inject an exception, but trapping twice per every function would
> probably be terrible for performance. The guest shouldn't be setting
> EnIA anyway if ID_AA64ISAR1_EL1 reports that pointer authentication is
> not present (because KVM has hidden it).

I don't think we should. The SCTLR_EL1 bits are RES0 unless you know
that the feature is present via CPUID.

> The other special case is the XPACLRI instruction, which is also in the
> HINT space. Currently it will trap and KVM will inject an exception into
> the guest. We should probably change this to NOP instead, as that's what
> applications will expect. Unfortunately there is no EnIA-like control to
> make it NOP.

Very good catch. Basically if EL2 doesn't know about ptr auth (older
distro), EL1 may or may not know but leaves SCTLR_EL1 disabled (based on
CPUID), the default HCR_EL2 is to trap (I'm ignoring EL3 as that's like
to have ptr auth enabled, being built for the specific HW). So a user
app considering XPACLRI a NOP (or inoffensive) will get a SIGILL
(injected by the guest kernel following the injection of "Unknown
reason" exception by KVM).

Ramana, is XPACLRI commonly generated by gcc and expects it to be a NOP?
Could we restrict it to only being used at run-time if the corresponding
HWCAP is set? This means redefining this instruction as no longer in the
NOP space.

> One option is for KVM to pretend the instruction was a NOP and return to
> the guest. But if XPACLRI gets executed frequently, then the constant
> trapping might hurt performance. I don't know how frequently it might
> get used, as I don't know of any applications currently using it. From
> what I understand, it may be used by userspace stack unwinders.
> 
> (Also worth noting - as far as I can tell there is no easy way for KVM
> to know which pointer authentication instruction caused the trap, so we
> may have to do something unusual like use "at s12e1r" to read guest
> memory and check for XPACLRI.)

Indeed, it's not an easy fix. As discussed (in the office), we can't
even guarantee that the guest stage 1 translation is stable and points
to the actual XPACLRI instruction.

> The other option is to turn off trapping entirely. However then on a
> big.LITTLE system with mismatched pointer authentication support
> instructions will work intermittently on some CPUs but not others.

That's another case but let's assume we never see such configurations ;).

Marc Zyngier Oct. 19, 2018, 11:47 a.m. UTC | #5

On 19/10/18 12:35, Catalin Marinas wrote:
> On Tue, Oct 16, 2018 at 05:14:39PM +0100, Kristina Martsenko wrote:
>> On 05/10/2018 10:04, Ramana Radhakrishnan wrote:
>>> On 05/10/2018 09:47, Kristina Martsenko wrote:

[...]

>> The other option is to turn off trapping entirely. However then on a
>> big.LITTLE system with mismatched pointer authentication support
>> instructions will work intermittently on some CPUs but not others.
> 
> That's another case but let's assume we never see such configurations ;).

I'd like to put it on the record that I'm not willing to support such a
configuration. So my ask is that if we detect a system where only some
of the CPUs have pointer authentication support, we either:

1) prevent some of the CPUs from booting (that's harsh)
2) disable KVM (that's easy)

I'm perfectly happy with (2).

Thanks,

	M.

Will Deacon Oct. 19, 2018, 12:22 p.m. UTC | #6

On Fri, Oct 19, 2018 at 12:35:56PM +0100, Catalin Marinas wrote:
> On Tue, Oct 16, 2018 at 05:14:39PM +0100, Kristina Martsenko wrote:
> > On 05/10/2018 10:04, Ramana Radhakrishnan wrote:
> > > On 05/10/2018 09:47, Kristina Martsenko wrote:
> > The other special case is the XPACLRI instruction, which is also in the
> > HINT space. Currently it will trap and KVM will inject an exception into
> > the guest. We should probably change this to NOP instead, as that's what
> > applications will expect. Unfortunately there is no EnIA-like control to
> > make it NOP.
> 
> Very good catch. Basically if EL2 doesn't know about ptr auth (older
> distro), EL1 may or may not know but leaves SCTLR_EL1 disabled (based on
> CPUID), the default HCR_EL2 is to trap (I'm ignoring EL3 as that's like
> to have ptr auth enabled, being built for the specific HW). So a user
> app considering XPACLRI a NOP (or inoffensive) will get a SIGILL
> (injected by the guest kernel following the injection of "Unknown
> reason" exception by KVM).
> 
> Ramana, is XPACLRI commonly generated by gcc and expects it to be a NOP?
> Could we restrict it to only being used at run-time if the corresponding
> HWCAP is set? This means redefining this instruction as no longer in the
> NOP space.

My main worry is that this instruction is used when unwinding C++
exceptions, so I think we'll see it fairly often.

Effectively, the architecture means these instructions can result in a
SIGILL if they are used under an OS/hypervisor that doesn't know about
the feature (i.e. any mainline kernel release so far). I think that's a
massive problem for the current implementation in GCC. Worse, if
distributions are currently shipping binaries built with this, they
basically have a ticking bomb in their applications where things will
start crashing when they encounter CPUs that implement pointer
authentication.

Ramana: do you know whether people are building binaries with this stuff
enabled by default?

Will

Kristina Martsenko Oct. 19, 2018, 2:42 p.m. UTC | #7

On 19/10/2018 12:35, Catalin Marinas wrote:
> On Tue, Oct 16, 2018 at 05:14:39PM +0100, Kristina Martsenko wrote:
>> On 05/10/2018 10:04, Ramana Radhakrishnan wrote:
>>> On 05/10/2018 09:47, Kristina Martsenko wrote:
>>>> +Virtualization
>>>> +--------------
>>>> +
>>>> +Pointer authentication is not currently supported in KVM guests. KVM
>>>> +will mask the feature bits from ID_AA64ISAR1_EL1, and attempted use of
>>>> +the feature will result in an UNDEFINED exception being injected into
>>>> +the guest.
>>>
>>> However applications using instructions from the hint space will
>>> continue to work albeit without any protection (as they would just be
>>> nops) ?
>>
>> Mostly, yes. If the guest leaves SCTLR_EL1.EnIA unset (and
>> EnIB/EnDA/EnDB), then PAC* and AUT* instructions in the HINT space will
>> execute as NOPs. If the guest sets EnIA, then PAC*/AUT* instructions
>> will trap and KVM will inject an "Unknown reason" exception into the
>> guest (which will cause a Linux guest to send a SIGILL to the application).
> 
> I think that part is fine. If KVM (a fairly recent version with CPUID
> sanitisation) does not enable ptr auth, the CPUID should not advertise
> this feature either so the guest kernel should not enable it. For the
> above instructions in the HINT space, they will just be NOPs. If the
> guest kernel enables the feature regardless of the CPUID information, it
> deserves to get an "Unknown reason" exception.
> 
>> In the latter case we could instead pretend the instruction was a NOP
>> and not inject an exception, but trapping twice per every function would
>> probably be terrible for performance. The guest shouldn't be setting
>> EnIA anyway if ID_AA64ISAR1_EL1 reports that pointer authentication is
>> not present (because KVM has hidden it).
> 
> I don't think we should. The SCTLR_EL1 bits are RES0 unless you know
> that the feature is present via CPUID.
> 
>> The other special case is the XPACLRI instruction, which is also in the
>> HINT space. Currently it will trap and KVM will inject an exception into
>> the guest. We should probably change this to NOP instead, as that's what
>> applications will expect. Unfortunately there is no EnIA-like control to
>> make it NOP.
> 
> Very good catch. Basically if EL2 doesn't know about ptr auth (older
> distro), EL1 may or may not know but leaves SCTLR_EL1 disabled (based on
> CPUID), the default HCR_EL2 is to trap (I'm ignoring EL3 as that's like
> to have ptr auth enabled, being built for the specific HW). So a user
> app considering XPACLRI a NOP (or inoffensive) will get a SIGILL
> (injected by the guest kernel following the injection of "Unknown
> reason" exception by KVM).
> 
> Ramana, is XPACLRI commonly generated by gcc and expects it to be a NOP?
> Could we restrict it to only being used at run-time if the corresponding
> HWCAP is set? This means redefining this instruction as no longer in the
> NOP space.

I think an alternative solution is to just disable trapping of pointer
auth instructions in KVM. This will mean that the instructions will
behave the same in the guest as they do in the host. HINT-space
instructions (including XPACLRI) will behave as NOPs (or perform their
function, if enabled by the guest), and will not trap.

A side effect of disabling trapping is that keys may effectively leak
from one guest to another, since one guest may set a key and another
guest may use an instruction that uses that key. But this can be fixed
by zeroing the keys every time we enter a guest. We can additionally
trap key accesses (which is separate from instruction trapping), to have
guests fail more reliably and avoid restoring host keys on guest exit.

Things still won't work well on big.LITTLE systems with mismatched
pointer auth support between CPUs, but as Marc pointed out in the other
email, we can just disable KVM on such systems when we detect a pointer
auth mismatch.

If we want current stable kernels to support guests that use HINT-space
pointer auth instructions, we'll need to backport the above changes to
stable kernels as well.

Even if we restricted userspace to only use XPACLRI if the HWCAP is set,
current stable kernels would still not be able to handle the HINT-space
PAC/AUT instructions that GCC generates, if the guest is pointer auth
aware. None of the stable kernels have the CPUID sanitisation patches,
so the guest would enable pointer auth, which would cause the PAC/AUT
instructions to trap.

>> One option is for KVM to pretend the instruction was a NOP and return to
>> the guest. But if XPACLRI gets executed frequently, then the constant
>> trapping might hurt performance. I don't know how frequently it might
>> get used, as I don't know of any applications currently using it. From
>> what I understand, it may be used by userspace stack unwinders.
>>
>> (Also worth noting - as far as I can tell there is no easy way for KVM
>> to know which pointer authentication instruction caused the trap, so we
>> may have to do something unusual like use "at s12e1r" to read guest
>> memory and check for XPACLRI.)
> 
> Indeed, it's not an easy fix. As discussed (in the office), we can't
> even guarantee that the guest stage 1 translation is stable and points
> to the actual XPACLRI instruction.
> 
>> The other option is to turn off trapping entirely. However then on a
>> big.LITTLE system with mismatched pointer authentication support
>> instructions will work intermittently on some CPUs but not others.
> 
> That's another case but let's assume we never see such configurations ;).

Kristina

Catalin Marinas Oct. 19, 2018, 3:10 p.m. UTC | #8

On Fri, Oct 19, 2018 at 03:42:23PM +0100, Kristina Martsenko wrote:
> On 19/10/2018 12:35, Catalin Marinas wrote:
> > On Tue, Oct 16, 2018 at 05:14:39PM +0100, Kristina Martsenko wrote:
> >> On 05/10/2018 10:04, Ramana Radhakrishnan wrote:
> >>> On 05/10/2018 09:47, Kristina Martsenko wrote:
> >>>> +Virtualization
> >>>> +--------------
> >>>> +
> >>>> +Pointer authentication is not currently supported in KVM guests. KVM
> >>>> +will mask the feature bits from ID_AA64ISAR1_EL1, and attempted use of
> >>>> +the feature will result in an UNDEFINED exception being injected into
> >>>> +the guest.
> >>>
> >>> However applications using instructions from the hint space will
> >>> continue to work albeit without any protection (as they would just be
> >>> nops) ?
> >>
> >> Mostly, yes. If the guest leaves SCTLR_EL1.EnIA unset (and
> >> EnIB/EnDA/EnDB), then PAC* and AUT* instructions in the HINT space will
> >> execute as NOPs. If the guest sets EnIA, then PAC*/AUT* instructions
> >> will trap and KVM will inject an "Unknown reason" exception into the
> >> guest (which will cause a Linux guest to send a SIGILL to the application).
> > 
> > I think that part is fine. If KVM (a fairly recent version with CPUID
> > sanitisation) does not enable ptr auth, the CPUID should not advertise
> > this feature either so the guest kernel should not enable it. For the
> > above instructions in the HINT space, they will just be NOPs. If the
> > guest kernel enables the feature regardless of the CPUID information, it
> > deserves to get an "Unknown reason" exception.
> > 
> >> In the latter case we could instead pretend the instruction was a NOP
> >> and not inject an exception, but trapping twice per every function would
> >> probably be terrible for performance. The guest shouldn't be setting
> >> EnIA anyway if ID_AA64ISAR1_EL1 reports that pointer authentication is
> >> not present (because KVM has hidden it).
> > 
> > I don't think we should. The SCTLR_EL1 bits are RES0 unless you know
> > that the feature is present via CPUID.
> > 
> >> The other special case is the XPACLRI instruction, which is also in the
> >> HINT space. Currently it will trap and KVM will inject an exception into
> >> the guest. We should probably change this to NOP instead, as that's what
> >> applications will expect. Unfortunately there is no EnIA-like control to
> >> make it NOP.
> > 
> > Very good catch. Basically if EL2 doesn't know about ptr auth (older
> > distro), EL1 may or may not know but leaves SCTLR_EL1 disabled (based on
> > CPUID), the default HCR_EL2 is to trap (I'm ignoring EL3 as that's like
> > to have ptr auth enabled, being built for the specific HW). So a user
> > app considering XPACLRI a NOP (or inoffensive) will get a SIGILL
> > (injected by the guest kernel following the injection of "Unknown
> > reason" exception by KVM).
> > 
> > Ramana, is XPACLRI commonly generated by gcc and expects it to be a NOP?
> > Could we restrict it to only being used at run-time if the corresponding
> > HWCAP is set? This means redefining this instruction as no longer in the
> > NOP space.
> 
> I think an alternative solution is to just disable trapping of pointer
> auth instructions in KVM. This will mean that the instructions will
> behave the same in the guest as they do in the host. HINT-space
> instructions (including XPACLRI) will behave as NOPs (or perform their
> function, if enabled by the guest), and will not trap.

OK, so this means disabling the trap (during early EL2 setup) but still
sanitizing the CPUID not to report the feature to EL1 unless fully
supported on all CPUs.

> A side effect of disabling trapping is that keys may effectively leak
> from one guest to another, since one guest may set a key and another
> guest may use an instruction that uses that key. But this can be fixed
> by zeroing the keys every time we enter a guest. We can additionally
> trap key accesses (which is separate from instruction trapping), to have
> guests fail more reliably and avoid restoring host keys on guest exit.

Actually, if the CPUID doesn't report the feature present, do we care
that a stupid guest writes some register and expects it to be preserved
and not leaked?

> Things still won't work well on big.LITTLE systems with mismatched
> pointer auth support between CPUs, but as Marc pointed out in the other
> email, we can just disable KVM on such systems when we detect a pointer
> auth mismatch.

If we ever see such system, we could follow the approach above - disable
trapping at EL2 if the feature is present but do not report it to EL1
via CPUID. A guest should not expect it to work properly.

> If we want current stable kernels to support guests that use HINT-space
> pointer auth instructions, we'll need to backport the above changes to
> stable kernels as well.

I agree.

> Even if we restricted userspace to only use XPACLRI if the HWCAP is set,
> current stable kernels would still not be able to handle the HINT-space
> PAC/AUT instructions that GCC generates, if the guest is pointer auth
> aware. None of the stable kernels have the CPUID sanitisation patches,
> so the guest would enable pointer auth, which would cause the PAC/AUT
> instructions to trap.

Ah, CPUID sanitisation would have to be backported as well. We can
probably get away with something simpler which does not sanitise
big.LITTLE (and I would not expect "enterprise" big.LITTLE machines) but
simply cap/mask the CPUID to what is known to the respective kernel
version. That shouldn't be too intrusive.

Will Deacon Oct. 19, 2018, 5:45 p.m. UTC | #9

On Fri, Oct 19, 2018 at 04:10:29PM +0100, Catalin Marinas wrote:
> On Fri, Oct 19, 2018 at 03:42:23PM +0100, Kristina Martsenko wrote:
> > On 19/10/2018 12:35, Catalin Marinas wrote:
> > > On Tue, Oct 16, 2018 at 05:14:39PM +0100, Kristina Martsenko wrote:
> > >> On 05/10/2018 10:04, Ramana Radhakrishnan wrote:
> > >>> On 05/10/2018 09:47, Kristina Martsenko wrote:
> > >>>> +Virtualization
> > >>>> +--------------
> > >>>> +
> > >>>> +Pointer authentication is not currently supported in KVM guests. KVM
> > >>>> +will mask the feature bits from ID_AA64ISAR1_EL1, and attempted use of
> > >>>> +the feature will result in an UNDEFINED exception being injected into
> > >>>> +the guest.
> > >>>
> > >>> However applications using instructions from the hint space will
> > >>> continue to work albeit without any protection (as they would just be
> > >>> nops) ?
> > >>
> > >> Mostly, yes. If the guest leaves SCTLR_EL1.EnIA unset (and
> > >> EnIB/EnDA/EnDB), then PAC* and AUT* instructions in the HINT space will
> > >> execute as NOPs. If the guest sets EnIA, then PAC*/AUT* instructions
> > >> will trap and KVM will inject an "Unknown reason" exception into the
> > >> guest (which will cause a Linux guest to send a SIGILL to the application).
> > > 
> > > I think that part is fine. If KVM (a fairly recent version with CPUID
> > > sanitisation) does not enable ptr auth, the CPUID should not advertise
> > > this feature either so the guest kernel should not enable it. For the
> > > above instructions in the HINT space, they will just be NOPs. If the
> > > guest kernel enables the feature regardless of the CPUID information, it
> > > deserves to get an "Unknown reason" exception.
> > > 
> > >> In the latter case we could instead pretend the instruction was a NOP
> > >> and not inject an exception, but trapping twice per every function would
> > >> probably be terrible for performance. The guest shouldn't be setting
> > >> EnIA anyway if ID_AA64ISAR1_EL1 reports that pointer authentication is
> > >> not present (because KVM has hidden it).
> > > 
> > > I don't think we should. The SCTLR_EL1 bits are RES0 unless you know
> > > that the feature is present via CPUID.
> > > 
> > >> The other special case is the XPACLRI instruction, which is also in the
> > >> HINT space. Currently it will trap and KVM will inject an exception into
> > >> the guest. We should probably change this to NOP instead, as that's what
> > >> applications will expect. Unfortunately there is no EnIA-like control to
> > >> make it NOP.
> > > 
> > > Very good catch. Basically if EL2 doesn't know about ptr auth (older
> > > distro), EL1 may or may not know but leaves SCTLR_EL1 disabled (based on
> > > CPUID), the default HCR_EL2 is to trap (I'm ignoring EL3 as that's like
> > > to have ptr auth enabled, being built for the specific HW). So a user
> > > app considering XPACLRI a NOP (or inoffensive) will get a SIGILL
> > > (injected by the guest kernel following the injection of "Unknown
> > > reason" exception by KVM).
> > > 
> > > Ramana, is XPACLRI commonly generated by gcc and expects it to be a NOP?
> > > Could we restrict it to only being used at run-time if the corresponding
> > > HWCAP is set? This means redefining this instruction as no longer in the
> > > NOP space.
> > 
> > I think an alternative solution is to just disable trapping of pointer
> > auth instructions in KVM. This will mean that the instructions will
> > behave the same in the guest as they do in the host. HINT-space
> > instructions (including XPACLRI) will behave as NOPs (or perform their
> > function, if enabled by the guest), and will not trap.
> 
> OK, so this means disabling the trap (during early EL2 setup) but still
> sanitizing the CPUID not to report the feature to EL1 unless fully
> supported on all CPUs.

... which is perfectly sensible, but not actually my main concern here.
I'm worried about the possibility of distributions shipping *now* with
userspace that's built with these instructions. That stuff is going to
break if/when it encounters v8.3 hardware, and I don't think we can do
much about it other than alert them to the potential issue.

Will

Ramana Radhakrishnan Oct. 24, 2018, 10:56 a.m. UTC | #10

On 19/10/2018 12:35, Catalin Marinas wrote:
> On Tue, Oct 16, 2018 at 05:14:39PM +0100, Kristina Martsenko wrote:
>> On 05/10/2018 10:04, Ramana Radhakrishnan wrote:
>>> On 05/10/2018 09:47, Kristina Martsenko wrote:
>>>> +Virtualization
>>>> +--------------
>>>> +
>>>> +Pointer authentication is not currently supported in KVM guests. KVM
>>>> +will mask the feature bits from ID_AA64ISAR1_EL1, and attempted use of
>>>> +the feature will result in an UNDEFINED exception being injected into
>>>> +the guest.
>>>
>>> However applications using instructions from the hint space will
>>> continue to work albeit without any protection (as they would just be
>>> nops) ?
>>
>> Mostly, yes. If the guest leaves SCTLR_EL1.EnIA unset (and
>> EnIB/EnDA/EnDB), then PAC* and AUT* instructions in the HINT space will
>> execute as NOPs. If the guest sets EnIA, then PAC*/AUT* instructions
>> will trap and KVM will inject an "Unknown reason" exception into the
>> guest (which will cause a Linux guest to send a SIGILL to the application).
> 
> I think that part is fine. If KVM (a fairly recent version with CPUID
> sanitisation) does not enable ptr auth, the CPUID should not advertise
> this feature either so the guest kernel should not enable it. For the
> above instructions in the HINT space, they will just be NOPs. If the
> guest kernel enables the feature regardless of the CPUID information, it
> deserves to get an "Unknown reason" exception.
> 
>> In the latter case we could instead pretend the instruction was a NOP
>> and not inject an exception, but trapping twice per every function would
>> probably be terrible for performance. The guest shouldn't be setting
>> EnIA anyway if ID_AA64ISAR1_EL1 reports that pointer authentication is
>> not present (because KVM has hidden it).
> 
> I don't think we should. The SCTLR_EL1 bits are RES0 unless you know
> that the feature is present via CPUID.
> 
>> The other special case is the XPACLRI instruction, which is also in the
>> HINT space. Currently it will trap and KVM will inject an exception into
>> the guest. We should probably change this to NOP instead, as that's what
>> applications will expect. Unfortunately there is no EnIA-like control to
>> make it NOP.
> 
> Very good catch. Basically if EL2 doesn't know about ptr auth (older
> distro), EL1 may or may not know but leaves SCTLR_EL1 disabled (based on
> CPUID), the default HCR_EL2 is to trap (I'm ignoring EL3 as that's like
> to have ptr auth enabled, being built for the specific HW). So a user
> app considering XPACLRI a NOP (or inoffensive) will get a SIGILL
> (injected by the guest kernel following the injection of "Unknown
> reason" exception by KVM).
> 
> Ramana, is XPACLRI commonly generated by gcc and expects it to be a NOP?
> Could we restrict it to only being used at run-time if the corresponding
> HWCAP is set? This means redefining this instruction as no longer in the
> NOP space.

Sorry to have missed this - I'm still catching up on email.

XPACLRI is used in the unwinder in exactly 2 places but not for 
unwinding itself but for storing the actual return address in the data 
structures, its not something I expect to be used very commonly so a 
check there seems reasonable. The xpaclri is considered a nop in the 
architecture as it is defined today. I don't like the idea of redefining 
instructions as not in the nop space after it's been defined as being 
so. We could investigate guarding the XPACLRI with a check with the 
HWCAP. How many unwinders would you like us to fix ?



> 
>> One option is for KVM to pretend the instruction was a NOP and return to
>> the guest. But if XPACLRI gets executed frequently, then the constant
>> trapping might hurt performance. I don't know how frequently it might
>> get used, as I don't know of any applications currently using it. From
>> what I understand, it may be used by userspace stack unwinders.

Yep. Probably one instruction per frame being unwound during exception 
unwinding. And no trapping will be expensive even though it's *only* in 
the exception unwind case.

>>
>> (Also worth noting - as far as I can tell there is no easy way for KVM
>> to know which pointer authentication instruction caused the trap, so we
>> may have to do something unusual like use "at s12e1r" to read guest
>> memory and check for XPACLRI.)
> 
> Indeed, it's not an easy fix. As discussed (in the office), we can't
> even guarantee that the guest stage 1 translation is stable and points
> to the actual XPACLRI instruction.
> 
>> The other option is to turn off trapping entirely. However then on a
>> big.LITTLE system with mismatched pointer authentication support
>> instructions will work intermittently on some CPUs but not others.
> 
> That's another case but let's assume we never see such configurations ;).

That's a broken system by design :) !

Ramana
>

Jon Masters Nov. 2, 2018, 6:02 a.m. UTC | #11

On 10/19/18 1:45 PM, Will Deacon wrote:

>>> I think an alternative solution is to just disable trapping of pointer
>>> auth instructions in KVM. This will mean that the instructions will
>>> behave the same in the guest as they do in the host. HINT-space
>>> instructions (including XPACLRI) will behave as NOPs (or perform their
>>> function, if enabled by the guest), and will not trap.
>>
>> OK, so this means disabling the trap (during early EL2 setup) but still
>> sanitizing the CPUID not to report the feature to EL1 unless fully
>> supported on all CPUs.
> 
> ... which is perfectly sensible, but not actually my main concern here.
> I'm worried about the possibility of distributions shipping *now* with
> userspace that's built with these instructions. That stuff is going to
> break if/when it encounters v8.3 hardware, and I don't think we can do
> much about it other than alert them to the potential issue.

FYI tracking this for RHEL. It's not a problem currently. I'll alert our
tools teams to hold off on any PAC work until this is figured out.

Jon.

Ramana Radhakrishnan Nov. 2, 2018, 9:46 a.m. UTC | #12

>> +
>> +When CONFIG_ARM64_PTR_AUTH is selected, and relevant HW support is
>> +present, the kernel will assign a random APIAKey value to each process
>> +at exec*() time. This key is shared by all threads within the process,
>> +and the key is preserved across fork(). Presence of functionality using
>> +APIAKey is advertised via HWCAP_APIA.
> 
> It might be useful to include documentation here on how many bits of
> the address are being used for the PAC bits (I'm assuming it's 7?)


the number of bits available depends on the VA size, so you can't really 
document a number.

Ramana

> 
> -Kees
>

Patch

diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt
index 8d0df62c3fe0..8df9f4658d6f 100644
--- a/Documentation/arm64/booting.txt
+++ b/Documentation/arm64/booting.txt
@@ -205,6 +205,14 @@  Before jumping into the kernel, the following conditions must be met:
     ICC_SRE_EL2.SRE (bit 0) must be initialised to 0b0.
   - The DT or ACPI tables must describe a GICv2 interrupt controller.
 
+  For CPUs with pointer authentication functionality:
+  - If EL3 is present:
+    SCR_EL3.APK (bit 16) must be initialised to 0b1
+    SCR_EL3.API (bit 17) must be initialised to 0b1
+  - If the kernel is entered at EL1:
+    HCR_EL2.APK (bit 40) must be initialised to 0b1
+    HCR_EL2.API (bit 41) must be initialised to 0b1
+
 The requirements described above for CPU mode, caches, MMUs, architected
 timers, coherency and system registers apply to all CPUs.  All CPUs must
 enter the kernel in the same exception level.
diff --git a/Documentation/arm64/cpu-feature-registers.txt b/Documentation/arm64/cpu-feature-registers.txt
index 7964f03846b1..b165677ffab9 100644
--- a/Documentation/arm64/cpu-feature-registers.txt
+++ b/Documentation/arm64/cpu-feature-registers.txt
@@ -190,6 +190,10 @@  infrastructure:
      |--------------------------------------------------|
      | JSCVT                        | [15-12] |    y    |
      |--------------------------------------------------|
+     | API                          | [11-8]  |    y    |
+     |--------------------------------------------------|
+     | APA                          | [7-4]   |    y    |
+     |--------------------------------------------------|
      | DPB                          | [3-0]   |    y    |
      x--------------------------------------------------x
 
diff --git a/Documentation/arm64/elf_hwcaps.txt b/Documentation/arm64/elf_hwcaps.txt
index d6aff2c5e9e2..95509a7b0ffe 100644
--- a/Documentation/arm64/elf_hwcaps.txt
+++ b/Documentation/arm64/elf_hwcaps.txt
@@ -178,3 +178,8 @@  HWCAP_ILRCPC
 HWCAP_FLAGM
 
     Functionality implied by ID_AA64ISAR0_EL1.TS == 0b0001.
+
+HWCAP_APIA
+
+    EL0 AddPac and Auth functionality using APIAKey_EL1 is enabled, as
+    described by Documentation/arm64/pointer-authentication.txt.
diff --git a/Documentation/arm64/pointer-authentication.txt b/Documentation/arm64/pointer-authentication.txt
new file mode 100644
index 000000000000..8a9cb5713770
--- /dev/null
+++ b/Documentation/arm64/pointer-authentication.txt
@@ -0,0 +1,84 @@ 
+Pointer authentication in AArch64 Linux
+=======================================
+
+Author: Mark Rutland <[e-mail]>
+Date: 2017-07-19
+
+This document briefly describes the provision of pointer authentication
+functionality in AArch64 Linux.
+
+
+Architecture overview
+---------------------
+
+The ARMv8.3 Pointer Authentication extension adds primitives that can be
+used to mitigate certain classes of attack where an attacker can corrupt
+the contents of some memory (e.g. the stack).
+
+The extension uses a Pointer Authentication Code (PAC) to determine
+whether pointers have been modified unexpectedly. A PAC is derived from
+a pointer, another value (such as the stack pointer), and a secret key
+held in system registers.
+
+The extension adds instructions to insert a valid PAC into a pointer,
+and to verify/remove the PAC from a pointer. The PAC occupies a number
+of high-order bits of the pointer, which varies dependent on the
+configured virtual address size and whether pointer tagging is in use.
+
+A subset of these instructions have been allocated from the HINT
+encoding space. In the absence of the extension (or when disabled),
+these instructions behave as NOPs. Applications and libraries using
+these instructions operate correctly regardless of the presence of the
+extension.
+
+
+Basic support
+-------------
+
+When CONFIG_ARM64_PTR_AUTH is selected, and relevant HW support is
+present, the kernel will assign a random APIAKey value to each process
+at exec*() time. This key is shared by all threads within the process,
+and the key is preserved across fork(). Presence of functionality using
+APIAKey is advertised via HWCAP_APIA.
+
+Recent versions of GCC can compile code with APIAKey-based return
+address protection when passed the -msign-return-address option. This
+uses instructions in the HINT space, and such code can run on systems
+without the pointer authentication extension.
+
+The remaining instruction and data keys (APIBKey, APDAKey, APDBKey) are
+reserved for future use, and instructions using these keys must not be
+used by software until a purpose and scope for their use has been
+decided. To enable future software using these keys to function on
+contemporary kernels, where possible, instructions using these keys are
+made to behave as NOPs.
+
+The generic key (APGAKey) is currently unsupported. Instructions using
+the generic key must not be used by software.
+
+
+Debugging
+---------
+
+When CONFIG_ARM64_PTR_AUTH is selected, and relevant HW support is
+present, the kernel will expose the position of TTBR0 PAC bits in the
+NT_ARM_PAC_MASK regset (struct user_pac_mask), which userspace can
+acqure via PTRACE_GETREGSET.
+
+Separate masks are exposed for data pointers and instruction pointers,
+as the set of PAC bits can vary between the two. Debuggers should not
+expect that HWCAP_APIA implies the presence (or non-presence) of this
+regset -- in future the kernel may support the use of APIBKey, APDAKey,
+and/or APBAKey, even in the absence of APIAKey.
+
+Note that the masks apply to TTBR0 addresses, and are not valid to apply
+to TTBR1 addresses (e.g. kernel pointers).
+
+
+Virtualization
+--------------
+
+Pointer authentication is not currently supported in KVM guests. KVM
+will mask the feature bits from ID_AA64ISAR1_EL1, and attempted use of
+the feature will result in an UNDEFINED exception being injected into
+the guest.