mbox series

[v2,00/25] Apple M1 SoC platform bring-up

Message ID 20210215121713.57687-1-marcan@marcan.st (mailing list archive)
Headers show
Series Apple M1 SoC platform bring-up | expand

Message

Hector Martin Feb. 15, 2021, 12:16 p.m. UTC
This series brings up initial support for the Apple M1 SoC, used in the
2020 Mac Mini, MacBook Pro, and MacBook Air models.

The following features are supported in this initial port:

- UART (samsung-style) with earlycon support
- Interrupts, including affinity and IPIs (Apple Interrupt Controller)
- SMP (through standard spin-table support)
- simplefb-based framebuffer
- Devicetree for the Mac Mini (should work for the others too at this
  stage)

See below for an overview of changes since v1.

== Patch overview ==

- 01-02 Core platform DT bindings
- 03-04 CPU DT bindings and MIDR defines
- 05-06 Add interrupt-names support to the ARM timer driver
        This is used to cleanly express the lack of a secure timer;
        platforms in the past have used various hacks like dummy
        IRQs here.
- 07-09 FIQ support
        These platforms require FIQ support, as some interrupt sources
        are hard-wired to FIQ. We settled on always keeping the FIQ/IRQ
        mask state synced, and only redirecting the vectors using
        alternatives on these CPUs. Other platforms retain the existing
        panic-on-FIQ behavior.
- 10-12 ioremap_np() (nGnRnE) support
        The fabric in these SoCs only supports nGnRnE accesses for
        standard MMIO, except for PCI ranges which use nGnRE. Linux
        currently defaults to the latter on ARM64, so this adds a new
        ioremap type and DT properties to automatically select it for
        drivers using OF and devm abstractions, under buses specified
        in DT.
- 13-15 AIC (Apple Interrupt Controller) driver and support defines
        This also embeds FIQ handling for this platform.
- 16    Introduce CONFIG_ARCH_APPLE & add it to defconfig
- 17-23 Add Apple SoC support to the samsung_tty driver
        This includes several refactoring patches to be able to more
        cleanly introduce this, as well as a switch to
        devm_ioremap_resource to be able to use the nGnRnE support
        introduced above. Earlycon support is included with a
        special-case nGnRnE hack, as earlycon is too early to use the
        generic infrastructure.
- 24    simple-framebuffer bindings for Apple (trivial)
- 25    Add the initial M1 Mac Mini (j274) devicetree

== About the hardware ==

These machines officially support booting unsigned/user-provided
XNU-like kernels, with a very different boot protocol and devicetree
format. We are developing an initial bootloader, m1n1 [1], to take care
of as many hardware peculiarities as possible and present a standard
Linux arm64 boot protocol and device tree. In the future, I expect that
production setups will add U-Boot and perhaps GRUB into the boot chain,
to make the boot process similar to other ARM64 platforms.

The machines expose their debug UART over USB Type C, triggered with
vendor-specific USB-PD commands. Currently, the easiest way to get a
serial console on these machines is to use a second M1 box and a simple
USB C cable [2]. You can also build a DIY interface using an Arduino, a
FUSB302 chip or board, and a 1.2V UART-TTL adapter [3]. In the coming
weeks we will be designing an open hardware project to provide
serial/debug connectivity to these machines (and, hopefully, also
support other UART-over-Type C setups from other vendors). Please
contact me privately if you are interested in getting an early prototype
version of one of these devices.

A quickstart guide to booting Linux kernels on these machines is
available at [4], and we are documenting the hardware at [5].

[1] https://github.com/AsahiLinux/m1n1/
[2] https://github.com/AsahiLinux/macvdmtool/
[3] https://github.com/AsahiLinux/vdmtool/
[4] https://github.com/AsahiLinux/docs/wiki/Developer-Quickstart
[5] https://github.com/AsahiLinux/docs/wiki

== Project Blurb ==

Asahi Linux is an open community project dedicated to developing and
maintaining mainline support for Apple Silicon on Linux. Feel free to
drop by #asahi and #asahi-dev on freenode to chat with us, or check
our website for more information on the project:

https://asahilinux.org/

== Changes since v1 ==

* DT vendor prefix AAPL -> apple
* Added interrupt-names support to the ARMv8 timer driver, to better
  represent the lack of secure timer
* Added sysreg_apple.h to define Apple vendor specific CPU system
  registers
* No longer gating the DAIF.F bit handling via alternatives; now we
  always keep it in sync with DAIF.I on all platforms.
* ARCH_APPLE selects APPLE_AIC
* Fixed copyright of DT (GPL-2.0+ OR MIT)
* Made all the copyright lines for new files
  `Copyright The Asahi Linux Contributors`
* Split DT into apple-m1.dtsi and apple-j274.dts
* Cleaner solution for FIQ alternatives in entry.S from Marc
* Implemented proper nGnRnE solution using the `[non]posted-mmio`
  DT properties and ioremap_np()
* Added (tentative) support for guest timer FIQs in AIC
* Added masked initialization and basic fallback handling for other FIQs
* Reworked AIC to not use chained IRQ domains, and split up FIQ irqchip
* More refactoring of the samsung_tty driver to cleanly introduce Apple
  support
* Fixed many style issues, bugs, and other nits

Note: this keeps the `apple,arm-platform` compatible, which is now used
to gate the OF nonposted-mmio logic to Apple platforms only.

Hector Martin (24):
  dt-bindings: vendor-prefixes: Add apple prefix
  dt-bindings: arm: apple: Add bindings for Apple ARM platforms
  dt-bindings: arm: cpus: Add apple,firestorm & icestorm compatibles
  arm64: cputype: Add CPU implementor & types for the Apple M1 cores
  dt-bindings: timer: arm,arch_timer: Add interrupt-names support
  arm64: arch_timer: implement support for interrupt-names
  arm64: cpufeature: Add a feature for FIQ support
  arm64: Always keep DAIF.[IF] in sync
  asm-generic/io.h:  Add a non-posted variant of ioremap()
  arm64: Implement ioremap_np() to map MMIO as nGnRnE
  of/address: Add infrastructure to declare MMIO as non-posted
  arm64: Add Apple vendor-specific system registers
  dt-bindings: interrupt-controller: Add DT bindings for apple-aic
  irqchip/apple-aic: Add support for the Apple Interrupt Controller
  arm64: Kconfig: Introduce CONFIG_ARCH_APPLE
  tty: serial: samsung_tty: Separate S3C64XX ops structure
  tty: serial: samsung_tty: add s3c24xx_port_type
  tty: serial: samsung_tty: IRQ rework
  tty: serial: samsung_tty: Use devm_ioremap_resource
  dt-bindings: serial: samsung: Add apple,s5l-uart compatible
  tty: serial: samsung_tty: Add support for Apple UARTs
  tty: serial: samsung_tty: Add earlycon support for Apple UARTs
  dt-bindings: display: Add apple,simple-framebuffer
  arm64: apple: Add initial Mac Mini 2020 (M1) devicetree

Marc Zyngier (1):
  arm64: entry: Map the FIQ vector to IRQ on NEEDS_FIQ platforms

 .../devicetree/bindings/arm/apple.yaml        |  36 +
 .../devicetree/bindings/arm/cpus.yaml         |   2 +
 .../bindings/display/simple-framebuffer.yaml  |   5 +
 .../interrupt-controller/apple,aic.yaml       |  88 +++
 .../bindings/serial/samsung_uart.yaml         |   4 +-
 .../bindings/timer/arm,arch_timer.yaml        |  14 +
 .../devicetree/bindings/vendor-prefixes.yaml  |   2 +
 MAINTAINERS                                   |  15 +
 arch/arm64/Kconfig                            |  11 +
 arch/arm64/Kconfig.platforms                  |   8 +
 arch/arm64/boot/dts/Makefile                  |   1 +
 arch/arm64/boot/dts/apple/Makefile            |   2 +
 arch/arm64/boot/dts/apple/apple-j274.dts      |  41 ++
 arch/arm64/boot/dts/apple/apple-m1.dtsi       | 124 ++++
 arch/arm64/configs/defconfig                  |   1 +
 arch/arm64/include/asm/assembler.h            |   6 +-
 arch/arm64/include/asm/cpucaps.h              |   3 +-
 arch/arm64/include/asm/cputype.h              |   6 +
 arch/arm64/include/asm/daifflags.h            |   4 +-
 arch/arm64/include/asm/io.h                   |   1 +
 arch/arm64/include/asm/irqflags.h             |  19 +-
 arch/arm64/include/asm/sysreg_apple.h         |  69 ++
 arch/arm64/kernel/cpufeature.c                |  14 +
 arch/arm64/kernel/entry.S                     |  27 +-
 drivers/clocksource/arm_arch_timer.c          |  25 +-
 drivers/irqchip/Kconfig                       |  10 +
 drivers/irqchip/Makefile                      |   1 +
 drivers/irqchip/irq-apple-aic.c               | 647 ++++++++++++++++++
 drivers/of/address.c                          |  72 +-
 drivers/tty/serial/Kconfig                    |   2 +-
 drivers/tty/serial/samsung_tty.c              | 499 +++++++++++---
 include/asm-generic/io.h                      |   8 +-
 include/clocksource/arm_arch_timer.h          |   1 +
 .../interrupt-controller/apple-aic.h          |  15 +
 include/linux/cpuhotplug.h                    |   1 +
 include/linux/io.h                            |   2 +
 include/linux/ioport.h                        |   1 +
 include/linux/of_address.h                    |   1 +
 include/linux/serial_s3c.h                    |  16 +
 lib/devres.c                                  |  22 +
 40 files changed, 1685 insertions(+), 141 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/arm/apple.yaml
 create mode 100644 Documentation/devicetree/bindings/interrupt-controller/apple,aic.yaml
 create mode 100644 arch/arm64/boot/dts/apple/Makefile
 create mode 100644 arch/arm64/boot/dts/apple/apple-j274.dts
 create mode 100644 arch/arm64/boot/dts/apple/apple-m1.dtsi
 create mode 100644 arch/arm64/include/asm/sysreg_apple.h
 create mode 100644 drivers/irqchip/irq-apple-aic.c
 create mode 100644 include/dt-bindings/interrupt-controller/apple-aic.h

--
2.30.0

Comments

Arnd Bergmann Feb. 15, 2021, 12:57 p.m. UTC | #1
On Mon, Feb 15, 2021 at 1:16 PM Hector Martin <marcan@marcan.st> wrote:
>
> This series brings up initial support for the Apple M1 SoC, used in the
> 2020 Mac Mini, MacBook Pro, and MacBook Air models.
>
> The following features are supported in this initial port:
>
> - UART (samsung-style) with earlycon support
> - Interrupts, including affinity and IPIs (Apple Interrupt Controller)
> - SMP (through standard spin-table support)
> - simplefb-based framebuffer
> - Devicetree for the Mac Mini (should work for the others too at this
>   stage)

I am essentially happy with the state of this series, the comments I had
in v1 by email and IRC are all addressed, but of course with the timing
during the merge window, it is not going to be in v5.12.

(adding maintainers for the serial/irqchip/clocksource drivers and
arch/arm64 to cc)

I would suggest merging it together as a series through the soc tree for
v5.13, once each patch has been reviewed by the respective subsystem
maintainers, with possible add-on patches on the same branch for
additional drivers that may become ready during the 5.12-rc cycle.
After the initial merge, driver patches will of course go through subsystem
trees as normal.

Let me know if that works for everyone.

     Arnd
Greg KH Feb. 15, 2021, 1:22 p.m. UTC | #2
On Mon, Feb 15, 2021 at 01:57:39PM +0100, Arnd Bergmann wrote:
> On Mon, Feb 15, 2021 at 1:16 PM Hector Martin <marcan@marcan.st> wrote:
> >
> > This series brings up initial support for the Apple M1 SoC, used in the
> > 2020 Mac Mini, MacBook Pro, and MacBook Air models.
> >
> > The following features are supported in this initial port:
> >
> > - UART (samsung-style) with earlycon support
> > - Interrupts, including affinity and IPIs (Apple Interrupt Controller)
> > - SMP (through standard spin-table support)
> > - simplefb-based framebuffer
> > - Devicetree for the Mac Mini (should work for the others too at this
> >   stage)
> 
> I am essentially happy with the state of this series, the comments I had
> in v1 by email and IRC are all addressed, but of course with the timing
> during the merge window, it is not going to be in v5.12.
> 
> (adding maintainers for the serial/irqchip/clocksource drivers and
> arch/arm64 to cc)
> 
> I would suggest merging it together as a series through the soc tree for
> v5.13, once each patch has been reviewed by the respective subsystem
> maintainers, with possible add-on patches on the same branch for
> additional drivers that may become ready during the 5.12-rc cycle.
> After the initial merge, driver patches will of course go through subsystem
> trees as normal.
> 
> Let me know if that works for everyone.

Sure, as long as the maintainers get to see the patches, I don't think
I've seen the serial ones at all...
Hector Martin Feb. 15, 2021, 3:57 p.m. UTC | #3
On 15/02/2021 22.22, gregkh wrote:
> On Mon, Feb 15, 2021 at 01:57:39PM +0100, Arnd Bergmann wrote:
>> (adding maintainers for the serial/irqchip/clocksource drivers and
>> arch/arm64 to cc)
>>
>> I would suggest merging it together as a series through the soc tree for
>> v5.13, once each patch has been reviewed by the respective subsystem
>> maintainers, with possible add-on patches on the same branch for
>> additional drivers that may become ready during the 5.12-rc cycle.
>> After the initial merge, driver patches will of course go through subsystem
>> trees as normal.
>>
>> Let me know if that works for everyone.
> 
> Sure, as long as the maintainers get to see the patches, I don't think
> I've seen the serial ones at all...

Sorry, I figured Krzysztof would take a look at it first and I didn't 
want to spam too much. I'm still getting used to figuring out who to CC...

Do you want to take a look at v2, or wait for v3?
Greg KH Feb. 15, 2021, 4:12 p.m. UTC | #4
On Tue, Feb 16, 2021 at 12:57:27AM +0900, Hector Martin wrote:
> On 15/02/2021 22.22, gregkh wrote:
> > On Mon, Feb 15, 2021 at 01:57:39PM +0100, Arnd Bergmann wrote:
> > > (adding maintainers for the serial/irqchip/clocksource drivers and
> > > arch/arm64 to cc)
> > > 
> > > I would suggest merging it together as a series through the soc tree for
> > > v5.13, once each patch has been reviewed by the respective subsystem
> > > maintainers, with possible add-on patches on the same branch for
> > > additional drivers that may become ready during the 5.12-rc cycle.
> > > After the initial merge, driver patches will of course go through subsystem
> > > trees as normal.
> > > 
> > > Let me know if that works for everyone.
> > 
> > Sure, as long as the maintainers get to see the patches, I don't think
> > I've seen the serial ones at all...
> 
> Sorry, I figured Krzysztof would take a look at it first and I didn't want
> to spam too much. I'm still getting used to figuring out who to CC...

scripts/get_maintainer.pl is your friend.

> Do you want to take a look at v2, or wait for v3?

v3 is fine, I can't do anything until after 5.12-rc1 is out anyway.

thanks,

greg k-h
Hector Martin Feb. 15, 2021, 4:54 p.m. UTC | #5
On 16/02/2021 01.12, gregkh wrote:
> On Tue, Feb 16, 2021 at 12:57:27AM +0900, Hector Martin wrote:
>> On 15/02/2021 22.22, gregkh wrote:
>>> On Mon, Feb 15, 2021 at 01:57:39PM +0100, Arnd Bergmann wrote:
>>>> (adding maintainers for the serial/irqchip/clocksource drivers and
>>>> arch/arm64 to cc)
>>>>
>>>> I would suggest merging it together as a series through the soc tree for
>>>> v5.13, once each patch has been reviewed by the respective subsystem
>>>> maintainers, with possible add-on patches on the same branch for
>>>> additional drivers that may become ready during the 5.12-rc cycle.
>>>> After the initial merge, driver patches will of course go through subsystem
>>>> trees as normal.
>>>>
>>>> Let me know if that works for everyone.
>>>
>>> Sure, as long as the maintainers get to see the patches, I don't think
>>> I've seen the serial ones at all...
>>
>> Sorry, I figured Krzysztof would take a look at it first and I didn't want
>> to spam too much. I'm still getting used to figuring out who to CC...
> 
> scripts/get_maintainer.pl is your friend.

It's the additional step of figuring out who to include from 
get_maintainer.pl output, and whether any subsetting is warranted at 
all, that I'm finding less well documented... :-)

(In particular for a bring-up series such as this one, where most people 
are only concerned with a few patches... but maybe I'm just overthinking 
things)

>> Do you want to take a look at v2, or wait for v3?
> 
> v3 is fine, I can't do anything until after 5.12-rc1 is out anyway.

Got it, thanks!
Krzysztof Kozlowski Feb. 15, 2021, 5:43 p.m. UTC | #6
On Tue, Feb 16, 2021 at 01:54:25AM +0900, Hector Martin wrote:
> On 16/02/2021 01.12, gregkh wrote:
> > On Tue, Feb 16, 2021 at 12:57:27AM +0900, Hector Martin wrote:
> > > On 15/02/2021 22.22, gregkh wrote:
> > > > On Mon, Feb 15, 2021 at 01:57:39PM +0100, Arnd Bergmann wrote:
> > > > > (adding maintainers for the serial/irqchip/clocksource drivers and
> > > > > arch/arm64 to cc)
> > > > > 
> > > > > I would suggest merging it together as a series through the soc tree for
> > > > > v5.13, once each patch has been reviewed by the respective subsystem
> > > > > maintainers, with possible add-on patches on the same branch for
> > > > > additional drivers that may become ready during the 5.12-rc cycle.
> > > > > After the initial merge, driver patches will of course go through subsystem
> > > > > trees as normal.
> > > > > 
> > > > > Let me know if that works for everyone.
> > > > 
> > > > Sure, as long as the maintainers get to see the patches, I don't think
> > > > I've seen the serial ones at all...
> > > 
> > > Sorry, I figured Krzysztof would take a look at it first and I didn't want
> > > to spam too much. I'm still getting used to figuring out who to CC...
> > 
> > scripts/get_maintainer.pl is your friend.
> 
> It's the additional step of figuring out who to include from
> get_maintainer.pl output, and whether any subsetting is warranted at all,
> that I'm finding less well documented... :-)
> 
> (In particular for a bring-up series such as this one, where most people are
> only concerned with a few patches... but maybe I'm just overthinking things)

For bigger patchsets, you should combine get_maintainer.pl with sending
emails so individual patches will go to all role-based entries from
get_maintainer.pl and to all mailing lists. You can (and sometimes even
worth to) skip proposals based on git-history.

Then the cover letter which should go to everyone... or most of people
and linked from individual patches.

For example the easiest I think:
1. Put all folks and lists as "Cc:" in cover letter.
2. Send everything:
   git send-email --cc-cmd "scripts/get_maintainer.pl --no-git --no-roles --no-rolestats" --to linux-kernel@vger.kernel.org 0*

Optionally you could add --no-git-fallback to get_maintainer if it
proposes some irrelevant addresses.

Other way is to send first cover letter and then reference it via
in-reply-to:

1. To get everyone for cover letter:
   scripts/get_maintainer.pl --no-multiline --interactive --separator=\'' --to '\' --no-git --no-roles --no-rolestats 00*

2. git send-email --cc linux-kernel@vger.kernel.org ... 0000*

3. Finally all patches with in-reply-to:
   git send-email --cc-cmd "scripts/get_maintainer.pl --no-git --no-roles --no-rolestats" --to linux-kernel@vger.kernel.org --no-thread --in-reply-to='foo-bar-xxxxxxx' 000[123456789]-* 00[123456789]*-*

Best regards,
Krzysztof
Marc Zyngier Feb. 15, 2021, 7:11 p.m. UTC | #7
On Mon, 15 Feb 2021 17:43:59 +0000,
Krzysztof Kozlowski <krzk@kernel.org> wrote:

> For bigger patchsets, you should combine get_maintainer.pl with sending
> emails so individual patches will go to all role-based entries from
> get_maintainer.pl and to all mailing lists. You can (and sometimes even
> worth to) skip proposals based on git-history.
> 
> Then the cover letter which should go to everyone... or most of people
> and linked from individual patches.
> 
> For example the easiest I think:
> 1. Put all folks and lists as "Cc:" in cover letter.
> 2. Send everything:
>    git send-email --cc-cmd "scripts/get_maintainer.pl --no-git --no-roles --no-rolestats" --to linux-kernel@vger.kernel.org 0*
> 
> Optionally you could add --no-git-fallback to get_maintainer if it
> proposes some irrelevant addresses.
> 
> Other way is to send first cover letter and then reference it via
> in-reply-to:
> 
> 1. To get everyone for cover letter:
>    scripts/get_maintainer.pl --no-multiline --interactive --separator=\'' --to '\' --no-git --no-roles --no-rolestats 00*
> 
> 2. git send-email --cc linux-kernel@vger.kernel.org ... 0000*
> 
> 3. Finally all patches with in-reply-to:
>    git send-email --cc-cmd "scripts/get_maintainer.pl --no-git --no-roles --no-rolestats" --to linux-kernel@vger.kernel.org --no-thread --in-reply-to='foo-bar-xxxxxxx' 000[123456789]-* 00[123456789]*-*

I personally dislike this method as a recipient. Context matters, and
seeing how an isolated patch fits in a bigger series is pretty
important. I can mark the patches I don't care about quicker than you
can say "Don't care", and it saves me having to drag the mbox via lore
and import it.

That's only me, but I suspect I'm not the only one with that
particular flow.

Thanks,

	M.
Mark Rutland Feb. 18, 2021, 2:36 p.m. UTC | #8
On Mon, Feb 15, 2021 at 09:16:48PM +0900, Hector Martin wrote:
> This series brings up initial support for the Apple M1 SoC, used in the
> 2020 Mac Mini, MacBook Pro, and MacBook Air models.
> 
> The following features are supported in this initial port:
> 
> - UART (samsung-style) with earlycon support
> - Interrupts, including affinity and IPIs (Apple Interrupt Controller)
> - SMP (through standard spin-table support)
> - simplefb-based framebuffer
> - Devicetree for the Mac Mini (should work for the others too at this
>   stage)

IIUC, the CPUs in these parts have some IMP-DEF instructions that can be
used at EL0 which might have some IMP-DEF state. Our general expectation
is that FW should configure such things to trap, but I don't know
whether the M1 FW does that, and I fear that this will end up being a
problem for us -- even if that doesn't affect EL1/EL2, IMP-DEF state is
an interesting covert channel between EL0 tasks, and not generally safe
to use thanks to context-switch and idle, so I'd like to make sure we
can catch usage and make it SIGILL.

Do you happen to know whether all of that is configured to trap, and if
not, is it possible to adjust the bootloader to ensure it is?

Thanks,
Mark.
Hector Martin Feb. 21, 2021, 3:20 p.m. UTC | #9
On 18/02/2021 23.36, Mark Rutland wrote:
> IIUC, the CPUs in these parts have some IMP-DEF instructions that can be
> used at EL0 which might have some IMP-DEF state. Our general expectation
> is that FW should configure such things to trap, but I don't know
> whether the M1 FW does that, and I fear that this will end up being a
> problem for us -- even if that doesn't affect EL1/EL2, IMP-DEF state is
> an interesting covert channel between EL0 tasks, and not generally safe
> to use thanks to context-switch and idle, so I'd like to make sure we
> can catch usage and make it SIGILL.
> 
> Do you happen to know whether all of that is configured to trap, and if
> not, is it possible to adjust the bootloader to ensure it is?

Very good point!

If only they were IMP-DEF... they're straight in Unallocated space. I 
spent some time the other day exhaustively searching the chunk of the 
encoding space where it looks like all these "fun" additions are,
at EL2, and I documented what I found here:

https://github.com/AsahiLinux/docs/wiki/HW:Apple-Instructions

I haven't tested things at EL0 yet, but it looks like the stateful 
instructions known to be usable in EL0 (AMX) already default to trap on 
this platform, so we should be safe there. Everything else looks like it 
probably either shouldn't work in EL0 (I sure hope the address 
translation one doesn't...) or is probably stateless. I'll dig deeper 
and test EL0 in the future, but so far things look OK (for some 
questionable values of OK :) ).
Hector Martin Feb. 23, 2021, 9:11 a.m. UTC | #10
On 22/02/2021 05.41, Andy Shevchenko wrote:
> Hector, I would like to be cc’ed in the next version

Noted, thanks!
Hector Martin Feb. 24, 2021, 3:55 p.m. UTC | #11
On 22/02/2021 00.20, Hector Martin wrote:
> I haven't tested things at EL0 yet, but it looks like the stateful
> instructions known to be usable in EL0 (AMX) already default to trap on
> this platform, so we should be safe there. Everything else looks like it
> probably either shouldn't work in EL0 (I sure hope the address
> translation one doesn't...) or is probably stateless. I'll dig deeper
> and test EL0 in the future, but so far things look OK (for some
> questionable values of OK :) ).

Follow-up: I have EL0 testing scaffolding now, and I found some more 
mutable state (an IMP-DEF, pre-standard version of FEAT_AFP, using a 
separate status register for the bits), but thankfully it traps at EL0 
by default.

And then I found some other mutable IMP-DEF state that does not trap at 
EL0. And which is a 0-day CVE in macOS, because it doesn't 
save/restore/clear it either, nor does it trap there.

E-mailing security@apple.com...