mbox series

[v13,00/12] Add AMD SEV guest live migration support

Message ID cover.1618498113.git.ashish.kalra@amd.com (mailing list archive)
Headers show
Series Add AMD SEV guest live migration support | expand

Message

Ashish Kalra April 15, 2021, 3:52 p.m. UTC
From: Ashish Kalra <ashish.kalra@amd.com>

The series add support for AMD SEV guest live migration commands. To protect the
confidentiality of an SEV protected guest memory while in transit we need to
use the SEV commands defined in SEV API spec [1].

SEV guest VMs have the concept of private and shared memory. Private memory
is encrypted with the guest-specific key, while shared memory may be encrypted
with hypervisor key. The commands provided by the SEV FW are meant to be used
for the private memory only. The patch series introduces a new hypercall.
The guest OS can use this hypercall to notify the page encryption status.
If the page is encrypted with guest specific-key then we use SEV command during
the migration. If page is not encrypted then fallback to default.

The patch uses the KVM_EXIT_HYPERCALL exitcode and hypercall to
userspace exit functionality as a common interface from the guest back to the
VMM and passing on the guest shared/unencrypted page information to the
userspace VMM/Qemu. Qemu can consult this information during migration to know 
whether the page is encrypted.

This section descibes how the SEV live migration feature is negotiated
between the host and guest, the host indicates this feature support via 
KVM_FEATURE_CPUID. The guest firmware (OVMF) detects this feature and
sets a UEFI enviroment variable indicating OVMF support for live
migration, the guest kernel also detects the host support for this
feature via cpuid and in case of an EFI boot verifies if OVMF also
supports this feature by getting the UEFI enviroment variable and if it
set then enables live migration feature on host by writing to a custom
MSR, if not booted under EFI, then it simply enables the feature by
again writing to the custom MSR. The MSR is also handled by the
userspace VMM/Qemu.

A branch containing these patches is available here:
https://github.com/AMDESE/linux/tree/sev-migration-v13

[1] https://developer.amd.com/wp-content/resources/55766.PDF

Changes since v12:
- Reset page encryption status during early boot instead of just 
  before the kexec to avoid SMP races during kvm_pv_guest_cpu_reboot().
- Remove incorrect log message in case of non-EFI boot and implicit
  enabling of SEV live migration feature.

Changes since v11:
- Clean up and remove kvm_x86_ops callback for page_enc_status_hc and
  instead add a new per-VM flag to support/enable the page encryption
  status hypercall.
- Remove KVM_EXIT_DMA_SHARE/KVM_EXIT_DMA_UNSHARE exitcodes and instead
  use the KVM_EXIT_HYPERCALL exitcode for page encryption status
  hypercall to userspace functionality. 

Changes since v10:
- Adds new KVM_EXIT_DMA_SHARE/KVM_EXIT_DMA_UNSHARE hypercall to
  userspace exit functionality as a common interface from the guest back to the
  KVM and passing on the guest shared/unencrypted region information to the
  userspace VMM/Qemu. KVM/host kernel does not maintain the guest shared
  memory regions information anymore. 
- Remove implicit enabling of SEV live migration feature for an SEV
  guest, now this is explicitly in control of the userspace VMM/Qemu.
- Custom MSR handling is also now moved into userspace VMM/Qemu.
- As KVM does not maintain the guest shared memory region information
  anymore, sev_dbg_crypt() cannot bypass unencrypted guest memory
  regions without support from userspace VMM/Qemu.

Changes since v9:
- Transitioning from page encryption bitmap to the shared pages list
  to keep track of guest's shared/unencrypted memory regions.
- Move back to marking the complete _bss_decrypted section as 
  decrypted in the shared pages list.
- Invoke a new function check_kvm_sev_migration() via kvm_init_platform()
  for guest to query for host-side support for SEV live migration 
  and to enable the SEV live migration feature, to avoid
  #ifdefs in code 
- Rename MSR_KVM_SEV_LIVE_MIG_EN to MSR_KVM_SEV_LIVE_MIGRATION.
- Invoke a new function handle_unencrypted_region() from 
  sev_dbg_crypt() to bypass unencrypted guest memory regions.

Changes since v8:
- Rebasing to kvm next branch.
- Fixed and added comments as per review feedback on v8 patches.
- Removed implicitly enabling live migration for incoming VMs in
  in KVM_SET_PAGE_ENC_BITMAP, it is now done via KVM_SET_MSR ioctl.
- Adds support for bypassing unencrypted guest memory regions for
  DBG_DECRYPT API calls, guest memory region encryption status in
  sev_dbg_decrypt() is referenced using the page encryption bitmap.

Changes since v7:
- Removed the hypervisor specific hypercall/paravirt callback for
  SEV live migration and moved back to calling kvm_sev_hypercall3 
  directly.
- Fix build errors as
  Reported-by: kbuild test robot <lkp@intel.com>, specifically fixed
  build error when CONFIG_HYPERVISOR_GUEST=y and
  CONFIG_AMD_MEM_ENCRYPT=n.
- Implicitly enabled live migration for incoming VM(s) to handle 
  A->B->C->... VM migrations.
- Fixed Documentation as per comments on v6 patches.
- Fixed error return path in sev_send_update_data() as per comments 
  on v6 patches. 

Changes since v6:
- Rebasing to mainline and refactoring to the new split SVM
  infrastructre.
- Move to static allocation of the unified Page Encryption bitmap
  instead of the dynamic resizing of the bitmap, the static allocation
  is done implicitly by extending kvm_arch_commit_memory_region() callack
  to add svm specific x86_ops which can read the userspace provided memory
  region/memslots and calculate the amount of guest RAM managed by the KVM
  and grow the bitmap.
- Fixed KVM_SET_PAGE_ENC_BITMAP ioctl to set the whole bitmap instead
  of simply clearing specific bits.
- Removed KVM_PAGE_ENC_BITMAP_RESET ioctl, which is now performed using
  KVM_SET_PAGE_ENC_BITMAP.
- Extended guest support for enabling Live Migration feature by adding a
  check for UEFI environment variable indicating OVMF support for Live
  Migration feature and additionally checking for KVM capability for the
  same feature. If not booted under EFI, then we simply check for KVM
  capability.
- Add hypervisor specific hypercall for SEV live migration by adding
  a new paravirt callback as part of x86_hyper_runtime.
  (x86 hypervisor specific runtime callbacks)
- Moving MSR handling for MSR_KVM_SEV_LIVE_MIG_EN into svm/sev code 
  and adding check for SEV live migration enabled by guest in the 
  KVM_GET_PAGE_ENC_BITMAP ioctl.
- Instead of the complete __bss_decrypted section, only specific variables
  such as hv_clock_boot and wall_clock are marked as decrypted in the
  page encryption bitmap

Changes since v5:
- Fix build errors as
  Reported-by: kbuild test robot <lkp@intel.com>

Changes since v4:
- Host support has been added to extend KVM capabilities/feature bits to 
  include a new KVM_FEATURE_SEV_LIVE_MIGRATION, which the guest can
  query for host-side support for SEV live migration and a new custom MSR
  MSR_KVM_SEV_LIVE_MIG_EN is added for guest to enable the SEV live
  migration feature.
- Ensure that _bss_decrypted section is marked as decrypted in the
  page encryption bitmap.
- Fixing KVM_GET_PAGE_ENC_BITMAP ioctl to return the correct bitmap
  as per the number of pages being requested by the user. Ensure that
  we only copy bmap->num_pages bytes in the userspace buffer, if
  bmap->num_pages is not byte aligned we read the trailing bits
  from the userspace and copy those bits as is. This fixes guest
  page(s) corruption issues observed after migration completion.
- Add kexec support for SEV Live Migration to reset the host's
  page encryption bitmap related to kernel specific page encryption
  status settings before we load a new kernel by kexec. We cannot
  reset the complete page encryption bitmap here as we need to
  retain the UEFI/OVMF firmware specific settings.

Changes since v3:
- Rebasing to mainline and testing.
- Adding a new KVM_PAGE_ENC_BITMAP_RESET ioctl, which resets the 
  page encryption bitmap on a guest reboot event.
- Adding a more reliable sanity check for GPA range being passed to
  the hypercall to ensure that guest MMIO ranges are also marked
  in the page encryption bitmap.

Changes since v2:
 - reset the page encryption bitmap on vcpu reboot

Changes since v1:
 - Add support to share the page encryption between the source and target
   machine.
 - Fix review feedbacks from Tom Lendacky.
 - Add check to limit the session blob length.
 - Update KVM_GET_PAGE_ENC_BITMAP icotl to use the base_gfn instead of
   the memory slot when querying the bitmap.

Ashish Kalra (4):
  KVM: X86: Introduce KVM_HC_PAGE_ENC_STATUS hypercall
  KVM: x86: Introduce new KVM_FEATURE_SEV_LIVE_MIGRATION feature &
    Custom MSR.
  EFI: Introduce the new AMD Memory Encryption GUID.
  x86/kvm: Add guest support for detecting and enabling SEV Live
    Migration feature.

Brijesh Singh (8):
  KVM: SVM: Add KVM_SEV SEND_START command
  KVM: SVM: Add KVM_SEND_UPDATE_DATA command
  KVM: SVM: Add KVM_SEV_SEND_FINISH command
  KVM: SVM: Add support for KVM_SEV_RECEIVE_START command
  KVM: SVM: Add KVM_SEV_RECEIVE_UPDATE_DATA command
  KVM: SVM: Add KVM_SEV_RECEIVE_FINISH command
  KVM: x86: Add AMD SEV specific Hypercall3
  mm: x86: Invoke hypercall when page encryption status is changed

 .../virt/kvm/amd-memory-encryption.rst        | 120 +++++
 Documentation/virt/kvm/cpuid.rst              |   5 +
 Documentation/virt/kvm/hypercalls.rst         |  15 +
 Documentation/virt/kvm/msr.rst                |  12 +
 arch/x86/include/asm/kvm_host.h               |   2 +
 arch/x86/include/asm/kvm_para.h               |  12 +
 arch/x86/include/asm/mem_encrypt.h            |   8 +
 arch/x86/include/asm/paravirt.h               |  10 +
 arch/x86/include/asm/paravirt_types.h         |   2 +
 arch/x86/include/uapi/asm/kvm_para.h          |   4 +
 arch/x86/kernel/kvm.c                         |  55 +++
 arch/x86/kernel/paravirt.c                    |   1 +
 arch/x86/kvm/cpuid.c                          |   3 +-
 arch/x86/kvm/svm/sev.c                        | 454 ++++++++++++++++++
 arch/x86/kvm/x86.c                            |  29 ++
 arch/x86/mm/mem_encrypt.c                     | 121 ++++-
 arch/x86/mm/pat/set_memory.c                  |   7 +
 include/linux/efi.h                           |   1 +
 include/linux/psp-sev.h                       |   8 +-
 include/uapi/linux/kvm.h                      |  39 ++
 include/uapi/linux/kvm_para.h                 |   1 +
 21 files changed, 903 insertions(+), 6 deletions(-)

Comments

Steve Rutherford April 16, 2021, 9:43 p.m. UTC | #1
On Thu, Apr 15, 2021 at 8:52 AM Ashish Kalra <Ashish.Kalra@amd.com> wrote:
>
> From: Ashish Kalra <ashish.kalra@amd.com>
>
> The series add support for AMD SEV guest live migration commands. To protect the
> confidentiality of an SEV protected guest memory while in transit we need to
> use the SEV commands defined in SEV API spec [1].
>
> SEV guest VMs have the concept of private and shared memory. Private memory
> is encrypted with the guest-specific key, while shared memory may be encrypted
> with hypervisor key. The commands provided by the SEV FW are meant to be used
> for the private memory only. The patch series introduces a new hypercall.
> The guest OS can use this hypercall to notify the page encryption status.
> If the page is encrypted with guest specific-key then we use SEV command during
> the migration. If page is not encrypted then fallback to default.
>
> The patch uses the KVM_EXIT_HYPERCALL exitcode and hypercall to
> userspace exit functionality as a common interface from the guest back to the
> VMM and passing on the guest shared/unencrypted page information to the
> userspace VMM/Qemu. Qemu can consult this information during migration to know
> whether the page is encrypted.
>
> This section descibes how the SEV live migration feature is negotiated
> between the host and guest, the host indicates this feature support via
> KVM_FEATURE_CPUID. The guest firmware (OVMF) detects this feature and
> sets a UEFI enviroment variable indicating OVMF support for live
> migration, the guest kernel also detects the host support for this
> feature via cpuid and in case of an EFI boot verifies if OVMF also
> supports this feature by getting the UEFI enviroment variable and if it
> set then enables live migration feature on host by writing to a custom
> MSR, if not booted under EFI, then it simply enables the feature by
> again writing to the custom MSR. The MSR is also handled by the
> userspace VMM/Qemu.
>
> A branch containing these patches is available here:
> https://github.com/AMDESE/linux/tree/sev-migration-v13
>
> [1] https://developer.amd.com/wp-content/resources/55766.PDF
>
> Changes since v12:
> - Reset page encryption status during early boot instead of just
>   before the kexec to avoid SMP races during kvm_pv_guest_cpu_reboot().

Does this series need to disable the MSR during kvm_pv_guest_cpu_reboot()?

I _think_ going into blackout during the window after restart, but
before the MSR is explicitly reenabled, would cause corruption. The
historical shared pages could be re-allocated as non-shared pages
during restart.

Steve
Ashish Kalra April 19, 2021, 2:40 p.m. UTC | #2
On Fri, Apr 16, 2021 at 02:43:48PM -0700, Steve Rutherford wrote:
> On Thu, Apr 15, 2021 at 8:52 AM Ashish Kalra <Ashish.Kalra@amd.com> wrote:
> >
> > From: Ashish Kalra <ashish.kalra@amd.com>
> >
> > The series add support for AMD SEV guest live migration commands. To protect the
> > confidentiality of an SEV protected guest memory while in transit we need to
> > use the SEV commands defined in SEV API spec [1].
> >
> > SEV guest VMs have the concept of private and shared memory. Private memory
> > is encrypted with the guest-specific key, while shared memory may be encrypted
> > with hypervisor key. The commands provided by the SEV FW are meant to be used
> > for the private memory only. The patch series introduces a new hypercall.
> > The guest OS can use this hypercall to notify the page encryption status.
> > If the page is encrypted with guest specific-key then we use SEV command during
> > the migration. If page is not encrypted then fallback to default.
> >
> > The patch uses the KVM_EXIT_HYPERCALL exitcode and hypercall to
> > userspace exit functionality as a common interface from the guest back to the
> > VMM and passing on the guest shared/unencrypted page information to the
> > userspace VMM/Qemu. Qemu can consult this information during migration to know
> > whether the page is encrypted.
> >
> > This section descibes how the SEV live migration feature is negotiated
> > between the host and guest, the host indicates this feature support via
> > KVM_FEATURE_CPUID. The guest firmware (OVMF) detects this feature and
> > sets a UEFI enviroment variable indicating OVMF support for live
> > migration, the guest kernel also detects the host support for this
> > feature via cpuid and in case of an EFI boot verifies if OVMF also
> > supports this feature by getting the UEFI enviroment variable and if it
> > set then enables live migration feature on host by writing to a custom
> > MSR, if not booted under EFI, then it simply enables the feature by
> > again writing to the custom MSR. The MSR is also handled by the
> > userspace VMM/Qemu.
> >
> > A branch containing these patches is available here:
> > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAMDESE%2Flinux%2Ftree%2Fsev-migration-v13&amp;data=04%7C01%7CAshish.Kalra%40amd.com%7C7bee6d5c907b46d0998508d90120ce2d%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637542063133830260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=FkKrciL41GDNyNDqrPMVblRa%2FaReogW4OzhbYaSYs04%3D&amp;reserved=0
> >
> > [1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper.amd.com%2Fwp-content%2Fresources%2F55766.PDF&amp;data=04%7C01%7CAshish.Kalra%40amd.com%7C7bee6d5c907b46d0998508d90120ce2d%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637542063133830260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=%2FLFBR9ean0acMmR8WTLUHZsAynYPRAa7%2FeZEVVdpCo8%3D&amp;reserved=0
> >
> > Changes since v12:
> > - Reset page encryption status during early boot instead of just
> >   before the kexec to avoid SMP races during kvm_pv_guest_cpu_reboot().
> 
> Does this series need to disable the MSR during kvm_pv_guest_cpu_reboot()?
> 

Yes, i think that will make sense, it will be similar to the first time
VM boot where the MSR will be disabled till it is enabled at early
kernel boot. I will add this to the current patch series.

Thanks,
Ashish

> I _think_ going into blackout during the window after restart, but
> before the MSR is explicitly reenabled, would cause corruption. The
> historical shared pages could be re-allocated as non-shared pages
> during restart.
> 
> Steve
Paolo Bonzini April 20, 2021, 11:11 a.m. UTC | #3
On 15/04/21 17:52, Ashish Kalra wrote:
> From: Ashish Kalra <ashish.kalra@amd.com>
> 
> The series add support for AMD SEV guest live migration commands. To protect the
> confidentiality of an SEV protected guest memory while in transit we need to
> use the SEV commands defined in SEV API spec [1].
> 
> SEV guest VMs have the concept of private and shared memory. Private memory
> is encrypted with the guest-specific key, while shared memory may be encrypted
> with hypervisor key. The commands provided by the SEV FW are meant to be used
> for the private memory only. The patch series introduces a new hypercall.
> The guest OS can use this hypercall to notify the page encryption status.
> If the page is encrypted with guest specific-key then we use SEV command during
> the migration. If page is not encrypted then fallback to default.
> 
> The patch uses the KVM_EXIT_HYPERCALL exitcode and hypercall to
> userspace exit functionality as a common interface from the guest back to the
> VMM and passing on the guest shared/unencrypted page information to the
> userspace VMM/Qemu. Qemu can consult this information during migration to know
> whether the page is encrypted.
> 
> This section descibes how the SEV live migration feature is negotiated
> between the host and guest, the host indicates this feature support via
> KVM_FEATURE_CPUID. The guest firmware (OVMF) detects this feature and
> sets a UEFI enviroment variable indicating OVMF support for live
> migration, the guest kernel also detects the host support for this
> feature via cpuid and in case of an EFI boot verifies if OVMF also
> supports this feature by getting the UEFI enviroment variable and if it
> set then enables live migration feature on host by writing to a custom
> MSR, if not booted under EFI, then it simply enables the feature by
> again writing to the custom MSR. The MSR is also handled by the
> userspace VMM/Qemu.
> 
> A branch containing these patches is available here:
> https://github.com/AMDESE/linux/tree/sev-migration-v13
> 
> [1] https://developer.amd.com/wp-content/resources/55766.PDF

I have queued patches 1-6.

For patches 8 and 10 I will post my own version based on my review and 
feedback.

For guest patches, please repost separately so that x86 maintainers will 
notice them and ack them.

Paolo

> Changes since v12:
> - Reset page encryption status during early boot instead of just
>    before the kexec to avoid SMP races during kvm_pv_guest_cpu_reboot().
> - Remove incorrect log message in case of non-EFI boot and implicit
>    enabling of SEV live migration feature.
> 
> Changes since v11:
> - Clean up and remove kvm_x86_ops callback for page_enc_status_hc and
>    instead add a new per-VM flag to support/enable the page encryption
>    status hypercall.
> - Remove KVM_EXIT_DMA_SHARE/KVM_EXIT_DMA_UNSHARE exitcodes and instead
>    use the KVM_EXIT_HYPERCALL exitcode for page encryption status
>    hypercall to userspace functionality.
> 
> Changes since v10:
> - Adds new KVM_EXIT_DMA_SHARE/KVM_EXIT_DMA_UNSHARE hypercall to
>    userspace exit functionality as a common interface from the guest back to the
>    KVM and passing on the guest shared/unencrypted region information to the
>    userspace VMM/Qemu. KVM/host kernel does not maintain the guest shared
>    memory regions information anymore.
> - Remove implicit enabling of SEV live migration feature for an SEV
>    guest, now this is explicitly in control of the userspace VMM/Qemu.
> - Custom MSR handling is also now moved into userspace VMM/Qemu.
> - As KVM does not maintain the guest shared memory region information
>    anymore, sev_dbg_crypt() cannot bypass unencrypted guest memory
>    regions without support from userspace VMM/Qemu.
> 
> Changes since v9:
> - Transitioning from page encryption bitmap to the shared pages list
>    to keep track of guest's shared/unencrypted memory regions.
> - Move back to marking the complete _bss_decrypted section as
>    decrypted in the shared pages list.
> - Invoke a new function check_kvm_sev_migration() via kvm_init_platform()
>    for guest to query for host-side support for SEV live migration
>    and to enable the SEV live migration feature, to avoid
>    #ifdefs in code
> - Rename MSR_KVM_SEV_LIVE_MIG_EN to MSR_KVM_SEV_LIVE_MIGRATION.
> - Invoke a new function handle_unencrypted_region() from
>    sev_dbg_crypt() to bypass unencrypted guest memory regions.
> 
> Changes since v8:
> - Rebasing to kvm next branch.
> - Fixed and added comments as per review feedback on v8 patches.
> - Removed implicitly enabling live migration for incoming VMs in
>    in KVM_SET_PAGE_ENC_BITMAP, it is now done via KVM_SET_MSR ioctl.
> - Adds support for bypassing unencrypted guest memory regions for
>    DBG_DECRYPT API calls, guest memory region encryption status in
>    sev_dbg_decrypt() is referenced using the page encryption bitmap.
> 
> Changes since v7:
> - Removed the hypervisor specific hypercall/paravirt callback for
>    SEV live migration and moved back to calling kvm_sev_hypercall3
>    directly.
> - Fix build errors as
>    Reported-by: kbuild test robot <lkp@intel.com>, specifically fixed
>    build error when CONFIG_HYPERVISOR_GUEST=y and
>    CONFIG_AMD_MEM_ENCRYPT=n.
> - Implicitly enabled live migration for incoming VM(s) to handle
>    A->B->C->... VM migrations.
> - Fixed Documentation as per comments on v6 patches.
> - Fixed error return path in sev_send_update_data() as per comments
>    on v6 patches.
> 
> Changes since v6:
> - Rebasing to mainline and refactoring to the new split SVM
>    infrastructre.
> - Move to static allocation of the unified Page Encryption bitmap
>    instead of the dynamic resizing of the bitmap, the static allocation
>    is done implicitly by extending kvm_arch_commit_memory_region() callack
>    to add svm specific x86_ops which can read the userspace provided memory
>    region/memslots and calculate the amount of guest RAM managed by the KVM
>    and grow the bitmap.
> - Fixed KVM_SET_PAGE_ENC_BITMAP ioctl to set the whole bitmap instead
>    of simply clearing specific bits.
> - Removed KVM_PAGE_ENC_BITMAP_RESET ioctl, which is now performed using
>    KVM_SET_PAGE_ENC_BITMAP.
> - Extended guest support for enabling Live Migration feature by adding a
>    check for UEFI environment variable indicating OVMF support for Live
>    Migration feature and additionally checking for KVM capability for the
>    same feature. If not booted under EFI, then we simply check for KVM
>    capability.
> - Add hypervisor specific hypercall for SEV live migration by adding
>    a new paravirt callback as part of x86_hyper_runtime.
>    (x86 hypervisor specific runtime callbacks)
> - Moving MSR handling for MSR_KVM_SEV_LIVE_MIG_EN into svm/sev code
>    and adding check for SEV live migration enabled by guest in the
>    KVM_GET_PAGE_ENC_BITMAP ioctl.
> - Instead of the complete __bss_decrypted section, only specific variables
>    such as hv_clock_boot and wall_clock are marked as decrypted in the
>    page encryption bitmap
> 
> Changes since v5:
> - Fix build errors as
>    Reported-by: kbuild test robot <lkp@intel.com>
> 
> Changes since v4:
> - Host support has been added to extend KVM capabilities/feature bits to
>    include a new KVM_FEATURE_SEV_LIVE_MIGRATION, which the guest can
>    query for host-side support for SEV live migration and a new custom MSR
>    MSR_KVM_SEV_LIVE_MIG_EN is added for guest to enable the SEV live
>    migration feature.
> - Ensure that _bss_decrypted section is marked as decrypted in the
>    page encryption bitmap.
> - Fixing KVM_GET_PAGE_ENC_BITMAP ioctl to return the correct bitmap
>    as per the number of pages being requested by the user. Ensure that
>    we only copy bmap->num_pages bytes in the userspace buffer, if
>    bmap->num_pages is not byte aligned we read the trailing bits
>    from the userspace and copy those bits as is. This fixes guest
>    page(s) corruption issues observed after migration completion.
> - Add kexec support for SEV Live Migration to reset the host's
>    page encryption bitmap related to kernel specific page encryption
>    status settings before we load a new kernel by kexec. We cannot
>    reset the complete page encryption bitmap here as we need to
>    retain the UEFI/OVMF firmware specific settings.
> 
> Changes since v3:
> - Rebasing to mainline and testing.
> - Adding a new KVM_PAGE_ENC_BITMAP_RESET ioctl, which resets the
>    page encryption bitmap on a guest reboot event.
> - Adding a more reliable sanity check for GPA range being passed to
>    the hypercall to ensure that guest MMIO ranges are also marked
>    in the page encryption bitmap.
> 
> Changes since v2:
>   - reset the page encryption bitmap on vcpu reboot
> 
> Changes since v1:
>   - Add support to share the page encryption between the source and target
>     machine.
>   - Fix review feedbacks from Tom Lendacky.
>   - Add check to limit the session blob length.
>   - Update KVM_GET_PAGE_ENC_BITMAP icotl to use the base_gfn instead of
>     the memory slot when querying the bitmap.
> 
> Ashish Kalra (4):
>    KVM: X86: Introduce KVM_HC_PAGE_ENC_STATUS hypercall
>    KVM: x86: Introduce new KVM_FEATURE_SEV_LIVE_MIGRATION feature &
>      Custom MSR.
>    EFI: Introduce the new AMD Memory Encryption GUID.
>    x86/kvm: Add guest support for detecting and enabling SEV Live
>      Migration feature.
> 
> Brijesh Singh (8):
>    KVM: SVM: Add KVM_SEV SEND_START command
>    KVM: SVM: Add KVM_SEND_UPDATE_DATA command
>    KVM: SVM: Add KVM_SEV_SEND_FINISH command
>    KVM: SVM: Add support for KVM_SEV_RECEIVE_START command
>    KVM: SVM: Add KVM_SEV_RECEIVE_UPDATE_DATA command
>    KVM: SVM: Add KVM_SEV_RECEIVE_FINISH command
>    KVM: x86: Add AMD SEV specific Hypercall3
>    mm: x86: Invoke hypercall when page encryption status is changed
> 
>   .../virt/kvm/amd-memory-encryption.rst        | 120 +++++
>   Documentation/virt/kvm/cpuid.rst              |   5 +
>   Documentation/virt/kvm/hypercalls.rst         |  15 +
>   Documentation/virt/kvm/msr.rst                |  12 +
>   arch/x86/include/asm/kvm_host.h               |   2 +
>   arch/x86/include/asm/kvm_para.h               |  12 +
>   arch/x86/include/asm/mem_encrypt.h            |   8 +
>   arch/x86/include/asm/paravirt.h               |  10 +
>   arch/x86/include/asm/paravirt_types.h         |   2 +
>   arch/x86/include/uapi/asm/kvm_para.h          |   4 +
>   arch/x86/kernel/kvm.c                         |  55 +++
>   arch/x86/kernel/paravirt.c                    |   1 +
>   arch/x86/kvm/cpuid.c                          |   3 +-
>   arch/x86/kvm/svm/sev.c                        | 454 ++++++++++++++++++
>   arch/x86/kvm/x86.c                            |  29 ++
>   arch/x86/mm/mem_encrypt.c                     | 121 ++++-
>   arch/x86/mm/pat/set_memory.c                  |   7 +
>   include/linux/efi.h                           |   1 +
>   include/linux/psp-sev.h                       |   8 +-
>   include/uapi/linux/kvm.h                      |  39 ++
>   include/uapi/linux/kvm_para.h                 |   1 +
>   21 files changed, 903 insertions(+), 6 deletions(-)
>
Borislav Petkov April 20, 2021, 6:51 p.m. UTC | #4
Hey Paolo,

On Tue, Apr 20, 2021 at 01:11:31PM +0200, Paolo Bonzini wrote:
> I have queued patches 1-6.
> 
> For patches 8 and 10 I will post my own version based on my review and
> feedback.

can you pls push that tree up to here to a branch somewhere so that ...
 
> For guest patches, please repost separately so that x86 maintainers will
> notice them and ack them.

... I can take a look at the guest bits in the full context of the
changes?

Thx.
Paolo Bonzini April 20, 2021, 7:08 p.m. UTC | #5
On 20/04/21 20:51, Borislav Petkov wrote:
> Hey Paolo,
> 
> On Tue, Apr 20, 2021 at 01:11:31PM +0200, Paolo Bonzini wrote:
>> I have queued patches 1-6.
>>
>> For patches 8 and 10 I will post my own version based on my review and
>> feedback.
> 
> can you pls push that tree up to here to a branch somewhere so that ...

Yup, for now it's all at kvm/queue and it will land in kvm/next tomorrow 
(hopefully).  The guest interface patches in KVM are very near the top.

Paolo

>> For guest patches, please repost separately so that x86 maintainers will
>> notice them and ack them.
> 
> ... I can take a look at the guest bits in the full context of the
> changes?
Borislav Petkov April 20, 2021, 8:28 p.m. UTC | #6
On Tue, Apr 20, 2021 at 09:08:26PM +0200, Paolo Bonzini wrote:
> Yup, for now it's all at kvm/queue and it will land in kvm/next tomorrow
> (hopefully).  The guest interface patches in KVM are very near the top.

Thx, I'll have a look tomorrow.