diff mbox series

[v5,05/17] KVM: Annotate -EFAULTs from kvm_vcpu_read/write_guest_page()

Message ID 20230908222905.1321305-6-amoorthy@google.com (mailing list archive)
State New, archived
Headers show
Series Improve KVM + userfaultfd live migration via annotated memory faults. | expand

Commit Message

Anish Moorthy Sept. 8, 2023, 10:28 p.m. UTC
Implement KVM_CAP_MEMORY_FAULT_INFO for uaccess failures in
kvm_vcpu_read/write_guest_page()

Signed-off-by: Anish Moorthy <amoorthy@google.com>
---
 virt/kvm/kvm_main.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

Comments

kernel test robot Sept. 14, 2023, 8:04 a.m. UTC | #1
Hello,

kernel test robot noticed "WARNING:at_include/linux/kvm_host.h:#kvm_vcpu_write_guest_page[kvm]" on:

commit: 00aaa25de7f10dfd5ac7afec09d6b4d72c379451 ("[PATCH v5 05/17] KVM: Annotate -EFAULTs from kvm_vcpu_read/write_guest_page()")
url: https://github.com/intel-lab-lkp/linux/commits/Anish-Moorthy/KVM-Clarify-documentation-of-hva_to_pfn-s-atomic-parameter/20230909-063310
base: https://git.kernel.org/cgit/virt/kvm/kvm.git queue
patch link: https://lore.kernel.org/all/20230908222905.1321305-6-amoorthy@google.com/
patch subject: [PATCH v5 05/17] KVM: Annotate -EFAULTs from kvm_vcpu_read/write_guest_page()

in testcase: kernel-selftests
version: kernel-selftests-x86_64-60acb023-1_20230329
with following parameters:

	group: kvm



compiler: gcc-12
test machine: 224 threads 2 sockets Intel(R) Xeon(R) Platinum 8480+ (Sapphire Rapids) with 256G memory

(please refer to attached dmesg/kmsg for entire log/backtrace)


If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <oliver.sang@intel.com>
| Closes: https://lore.kernel.org/oe-lkp/202309141107.30863e9d-oliver.sang@intel.com


The kernel config and materials to reproduce are available at:
https://download.01.org/0day-ci/archive/20230914/202309141107.30863e9d-oliver.sang@intel.com


[  216.317580][ T6089] ------------[ cut here ]------------
[  216.324543][ T6089] WARNING: CPU: 117 PID: 6089 at include/linux/kvm_host.h:2346 kvm_vcpu_write_guest_page+0x23b/0x2a0 [kvm]
[  216.338385][ T6089] Modules linked in: openvswitch nf_conncount nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 intel_rapl_msr intel_rapl_common btrfs x86_pkg_temp_thermal blake2b_generic intel_powerclamp xor coretemp raid6_pq kvm_intel zstd_compress libcrc32c kvm irqbypass crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel nvme sha512_ssse3 nvme_core rapl t10_pi intel_cstate mei_me ast dax_hmem crc64_rocksoft_generic crc64_rocksoft drm_shmem_helper i2c_i801 crc64 i2c_ismt mei i2c_smbus drm_kms_helper wmi ipmi_ssif acpi_ipmi joydev ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter acpi_pad binfmt_misc fuse drm ip_tables
[  216.406963][ T6089] CPU: 117 PID: 6089 Comm: mmio_warning_te Not tainted 6.5.0-00313-g00aaa25de7f1 #1
[  216.418660][ T6089] RIP: 0010:kvm_vcpu_write_guest_page+0x23b/0x2a0 [kvm]
[  216.427008][ T6089] Code: c1 8b 04 24 e9 d0 fe ff ff 89 04 24 e8 3e 3c 09 c1 8b 04 24 e9 1f ff ff ff 0f 1f 44 00 00 e9 5b fe ff ff 0f 0b e9 24 fe ff ff <0f> 0b e9 89 fe ff ff 48 89 df 48 89 34 24 e8 52 3c 09 c1 48 8b 34
[  216.450579][ T6089] RSP: 0018:ffa000001ad0f638 EFLAGS: 00010202
[  216.457880][ T6089] RAX: 00000000fffffff2 RBX: ff1100019e5a8040 RCX: 1fe2200033cb53c9
[  216.467703][ T6089] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffa00000177e1000
[  216.477457][ T6089] RBP: 000000000000fffc R08: 0000000000000ffc R09: 0000000000000002
[  216.487445][ T6089] R10: ffa00000177eafd3 R11: 0000000000000001 R12: ff1100019e5a9e48
[  216.497466][ T6089] R13: 0000000000000002 R14: ff11000500a72cd0 R15: 000000000000000f
[  216.507485][ T6089] FS:  00007fd4160036c0(0000) GS:ff110017fe680000(0000) knlGS:0000000000000000
[  216.518653][ T6089] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  216.526450][ T6089] CR2: 00007fd416002f78 CR3: 0000000154bb0003 CR4: 0000000000f73ee0
[  216.536508][ T6089] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  216.546584][ T6089] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
[  216.556682][ T6089] PKRU: 55555554
[  216.561031][ T6089] Call Trace:
[  216.565102][ T6089]  <TASK>
[  216.568851][ T6089]  ? __warn+0xcd/0x2b0
[  216.573882][ T6089]  ? kvm_vcpu_write_guest_page+0x23b/0x2a0 [kvm]
[  216.581406][ T6089]  ? report_bug+0x267/0x2d0
[  216.586914][ T6089]  ? handle_bug+0x3c/0x70
[  216.592099][ T6089]  ? exc_invalid_op+0x17/0x40
[  216.597749][ T6089]  ? asm_exc_invalid_op+0x1a/0x20
[  216.603751][ T6089]  ? kvm_vcpu_write_guest_page+0x23b/0x2a0 [kvm]
[  216.611219][ T6089]  ? kvm_vcpu_write_guest_page+0x5b/0x2a0 [kvm]
[  216.618594][ T6089]  kvm_vcpu_write_guest+0x4b/0x80 [kvm]
[  216.625453][ T6089]  write_emulate+0x23/0x50 [kvm]
[  216.631477][ T6089]  emulator_read_write_onepage+0x2ff/0x4a0 [kvm]
[  216.638944][ T6089]  ? vcpu_mmio_gva_to_gpa+0x730/0x730 [kvm]
[  216.645902][ T6089]  ? em_clflushopt+0x10/0x10 [kvm]
[  216.651976][ T6089]  emulator_read_write+0x149/0x510 [kvm]
[  216.658642][ T6089]  segmented_write+0xce/0x120 [kvm]
[  216.665492][ T6089]  ? em_sgdt+0x70/0x70 [kvm]
[  216.670959][ T6089]  ? vmx_read_guest_seg_selector+0x2c/0x290 [kvm_intel]
[  216.679008][ T6089]  push+0x316/0x5f0 [kvm]
[  216.684164][ T6089]  ? emulator_get_segment+0xbe/0x410 [kvm]
[  216.690978][ T6089]  ? load_state_from_tss16+0x940/0x940 [kvm]
[  216.697979][ T6089]  __emulate_int_real+0x306/0x690 [kvm]
[  216.704485][ T6089]  ? vmx_read_guest_seg_ar+0x2f/0x2b0 [kvm_intel]
[  216.711940][ T6089]  ? em_call+0x120/0x120 [kvm]
[  216.717583][ T6089]  ? kvm_guest_time_update+0x420/0xae0 [kvm]
[  216.724596][ T6089]  ? trace_event_raw_event_kvm_exit+0x2d0/0x2d0 [kvm]
[  216.732449][ T6089]  ? validate_chain+0x151/0xfe0
[  216.738101][ T6089]  ? slab_free_freelist_hook+0x11e/0x1e0
[  216.744690][ T6089]  emulate_int_real+0x79/0xc0 [kvm]
[  216.750867][ T6089]  kvm_inject_realmode_interrupt+0x102/0x260 [kvm]
[  216.758433][ T6089]  kvm_check_and_inject_events+0x805/0x1090 [kvm]
[  216.765924][ T6089]  vcpu_enter_guest+0xbd3/0x3780 [kvm]
[  216.773487][ T6089]  ? kvm_check_and_inject_events+0x1090/0x1090 [kvm]
[  216.781257][ T6089]  ? lock_acquire+0x193/0x4b0
[  216.786793][ T6089]  ? kvm_arch_vcpu_ioctl_run+0x12d/0x1630 [kvm]
[  216.794064][ T6089]  ? lock_sync+0x170/0x170
[  216.799259][ T6089]  ? mark_held_locks+0x9e/0xe0
[  216.804829][ T6089]  ? vcpu_run+0xb2/0xa00 [kvm]
[  216.810443][ T6089]  vcpu_run+0xb2/0xa00 [kvm]
[  216.815883][ T6089]  ? __local_bh_enable_ip+0xa6/0x110
[  216.822064][ T6089]  kvm_arch_vcpu_ioctl_run+0x39f/0x1630 [kvm]
[  216.829166][ T6089]  kvm_vcpu_ioctl+0x51c/0xcb0 [kvm]
[  216.835258][ T6089]  ? kvm_vcpu_kick+0x320/0x320 [kvm]
[  216.841460][ T6089]  ? find_held_lock+0x2d/0x110
[  216.847022][ T6089]  ? __lock_release+0x111/0x440
[  216.853383][ T6089]  ? __fget_files+0x1c5/0x380
[  216.858863][ T6089]  ? reacquire_held_locks+0x4e0/0x4e0
[  216.865134][ T6089]  ? __fget_files+0x1c5/0x380
[  216.870640][ T6089]  ? lock_release+0xe3/0x200
[  216.876005][ T6089]  ? __fget_files+0x1dd/0x380
[  216.881470][ T6089]  __x64_sys_ioctl+0x130/0x1a0
[  216.887004][ T6089]  do_syscall_64+0x59/0x80
[  216.892164][ T6089]  entry_SYSCALL_64_after_hwframe+0x5e/0xc8
[  216.898974][ T6089] RIP: 0033:0x7fd416905bab
[  216.904137][ T6089] Code: 00 48 89 44 24 18 31 c0 48 8d 44 24 60 c7 04 24 10 00 00 00 48 89 44 24 08 48 8d 44 24 20 48 89 44 24 10 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 1c 48 8b 44 24 18 64 48 2b 04 25 28 00 00
[  216.927031][ T6089] RSP: 002b:00007fd416002e70 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[  216.937130][ T6089] RAX: ffffffffffffffda RBX: 00007fd4169ef000 RCX: 00007fd416905bab
[  216.946672][ T6089] RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000007
[  216.956171][ T6089] RBP: 0000000000000000 R08: 0000000000000000 R09: 00007fff966831e7
[  216.965766][ T6089] R10: 0000000000000008 R11: 0000000000000246 R12: ffffffffffffff80
[  216.975342][ T6089] R13: 0000000000000000 R14: 00007fff966830f0 R15: 00007fd415803000
[  216.985071][ T6089]  </TASK>
[  216.988860][ T6089] irq event stamp: 1547
[  216.993931][ T6089] hardirqs last  enabled at (1561): [<ffffffff81385452>] __up_console_sem+0x52/0x60
[  217.005280][ T6089] hardirqs last disabled at (1580): [<ffffffff81385437>] __up_console_sem+0x37/0x60
[  217.016487][ T6089] softirqs last  enabled at (1574): [<ffffffff83a997a5>] __do_softirq+0x545/0x814
[  217.027532][ T6089] softirqs last disabled at (1569): [<ffffffff811eb372>] __irq_exit_rcu+0x132/0x180
[  217.038927][ T6089] ---[ end trace 0000000000000000 ]---
[  217.045467][ T6089] ------------[ cut here ]------------
[  217.051961][ T6089] WARNING: CPU: 117 PID: 6089 at include/linux/kvm_host.h:2346 kvm_vcpu_read_guest_page+0x21f/0x270 [kvm]
[  217.065501][ T6089] Modules linked in: openvswitch nf_conncount nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 intel_rapl_msr intel_rapl_common btrfs x86_pkg_temp_thermal blake2b_generic intel_powerclamp xor coretemp raid6_pq kvm_intel zstd_compress libcrc32c kvm irqbypass crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel nvme sha512_ssse3 nvme_core rapl t10_pi intel_cstate mei_me ast dax_hmem crc64_rocksoft_generic crc64_rocksoft drm_shmem_helper i2c_i801 crc64 i2c_ismt mei i2c_smbus drm_kms_helper wmi ipmi_ssif acpi_ipmi joydev ipmi_si ipmi_devintf ipmi_msghandler acpi_power_meter acpi_pad binfmt_misc fuse drm ip_tables
[  217.134216][ T6089] CPU: 117 PID: 6089 Comm: mmio_warning_te Tainted: G        W          6.5.0-00313-g00aaa25de7f1 #1
[  217.147426][ T6089] RIP: 0010:kvm_vcpu_read_guest_page+0x21f/0x270 [kvm]
[  217.155531][ T6089] Code: 24 04 e9 d0 fe ff ff 89 44 24 04 e8 db 38 09 c1 8b 44 24 04 e9 1d ff ff ff 0f 1f 44 00 00 e9 59 fe ff ff 0f 0b e9 22 fe ff ff <0f> 0b e9 87 fe ff ff 89 44 24 04 e8 91 39 09 c1 8b 44 24 04 e9 25
[  217.179210][ T6089] RSP: 0018:ffa000001ad0f8b8 EFLAGS: 00010202
[  217.186390][ T6089] RAX: 00000000fffffff2 RBX: ff1100019e5a8040 RCX: 1fe2200033cb53c9
[  217.196363][ T6089] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
[  217.206324][ T6089] RBP: 000000000000001a R08: 0000000000000002 R09: fff3fc0002efd5fa
[  217.216274][ T6089] R10: ffa00000177eafd3 R11: 0000000000000001 R12: ff1100019e5a9e48
[  217.226228][ T6089] R13: 0000000000000002 R14: ffa000001ad0f9a0 R15: ffa000001ad0f9a0
[  217.236138][ T6089] FS:  00007fd4160036c0(0000) GS:ff110017fe680000(0000) knlGS:0000000000000000
[  217.247194][ T6089] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  217.254995][ T6089] CR2: 00007fd416002f78 CR3: 0000000154bb0003 CR4: 0000000000f73ee0
[  217.264906][ T6089] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  217.274788][ T6089] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
[  217.284600][ T6089] PKRU: 55555554
[  217.289083][ T6089] Call Trace:
[  217.293161][ T6089]  <TASK>
[  217.296927][ T6089]  ? __warn+0xcd/0x2b0
[  217.301964][ T6089]  ? kvm_vcpu_read_guest_page+0x21f/0x270 [kvm]
[  217.309368][ T6089]  ? report_bug+0x267/0x2d0
[  217.314868][ T6089]  ? handle_bug+0x3c/0x70
[  217.320085][ T6089]  ? exc_invalid_op+0x17/0x40
[  217.325731][ T6089]  ? asm_exc_invalid_op+0x1a/0x20
[  217.331757][ T6089]  ? kvm_vcpu_read_guest_page+0x21f/0x270 [kvm]
[  217.339112][ T6089]  ? kvm_vcpu_read_guest_page+0x3d/0x270 [kvm]
[  217.346393][ T6089]  kvm_read_guest_virt_helper+0x97/0x150 [kvm]
[  217.353707][ T6089]  __emulate_int_real+0x478/0x690 [kvm]
[  217.360268][ T6089]  ? vmx_read_guest_seg_ar+0x2f/0x2b0 [kvm_intel]
[  217.367857][ T6089]  ? em_call+0x120/0x120 [kvm]
[  217.373544][ T6089]  ? kvm_guest_time_update+0x420/0xae0 [kvm]
[  217.380580][ T6089]  ? trace_event_raw_event_kvm_exit+0x2d0/0x2d0 [kvm]
[  217.388496][ T6089]  ? validate_chain+0x151/0xfe0
[  217.394204][ T6089]  ? slab_free_freelist_hook+0x11e/0x1e0
[  217.400874][ T6089]  emulate_int_real+0x79/0xc0 [kvm]
[  217.407014][ T6089]  kvm_inject_realmode_interrupt+0x102/0x260 [kvm]
[  217.414639][ T6089]  kvm_check_and_inject_events+0x805/0x1090 [kvm]
[  217.422140][ T6089]  vcpu_enter_guest+0xbd3/0x3780 [kvm]
[  217.429792][ T6089]  ? kvm_check_and_inject_events+0x1090/0x1090 [kvm]
[  217.437587][ T6089]  ? lock_acquire+0x193/0x4b0
[  217.443085][ T6089]  ? kvm_arch_vcpu_ioctl_run+0x12d/0x1630 [kvm]
[  217.450366][ T6089]  ? lock_sync+0x170/0x170
[  217.455537][ T6089]  ? mark_held_locks+0x9e/0xe0
[  217.461108][ T6089]  ? vcpu_run+0xb2/0xa00 [kvm]
[  217.466753][ T6089]  vcpu_run+0xb2/0xa00 [kvm]
[  217.472164][ T6089]  ? __local_bh_enable_ip+0xa6/0x110
[  217.478296][ T6089]  kvm_arch_vcpu_ioctl_run+0x39f/0x1630 [kvm]
[  217.485359][ T6089]  kvm_vcpu_ioctl+0x51c/0xcb0 [kvm]
[  217.491433][ T6089]  ? kvm_vcpu_kick+0x320/0x320 [kvm]
[  217.497653][ T6089]  ? find_held_lock+0x2d/0x110
[  217.503209][ T6089]  ? __lock_release+0x111/0x440
[  217.509571][ T6089]  ? __fget_files+0x1c5/0x380
[  217.515022][ T6089]  ? reacquire_held_locks+0x4e0/0x4e0
[  217.521289][ T6089]  ? __fget_files+0x1c5/0x380
[  217.526854][ T6089]  ? lock_release+0xe3/0x200
[  217.532235][ T6089]  ? __fget_files+0x1dd/0x380
[  217.537777][ T6089]  __x64_sys_ioctl+0x130/0x1a0
[  217.543329][ T6089]  do_syscall_64+0x59/0x80
[  217.548489][ T6089]  entry_SYSCALL_64_after_hwframe+0x5e/0xc8
[  217.555297][ T6089] RIP: 0033:0x7fd416905bab
[  217.560474][ T6089] Code: 00 48 89 44 24 18 31 c0 48 8d 44 24 60 c7 04 24 10 00 00 00 48 89 44 24 08 48 8d 44 24 20 48 89 44 24 10 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 1c 48 8b 44 24 18 64 48 2b 04 25 28 00 00
[  217.583524][ T6089] RSP: 002b:00007fd416002e70 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[  217.593465][ T6089] RAX: ffffffffffffffda RBX: 00007fd4169ef000 RCX: 00007fd416905bab
[  217.603087][ T6089] RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000007
[  217.612606][ T6089] RBP: 0000000000000000 R08: 0000000000000000 R09: 00007fff966831e7
[  217.622354][ T6089] R10: 0000000000000008 R11: 0000000000000246 R12: ffffffffffffff80
[  217.632027][ T6089] R13: 0000000000000000 R14: 00007fff966830f0 R15: 00007fd415803000
[  217.641658][ T6089]  </TASK>
[  217.645355][ T6089] irq event stamp: 2949
[  217.650299][ T6089] hardirqs last  enabled at (2961): [<ffffffff81385452>] __up_console_sem+0x52/0x60
[  217.661481][ T6089] hardirqs last disabled at (2978): [<ffffffff81385437>] __up_console_sem+0x37/0x60
[  217.672743][ T6089] softirqs last  enabled at (2974): [<ffffffff83a997a5>] __do_softirq+0x545/0x814
[  217.683899][ T6089] softirqs last disabled at (2969): [<ffffffff811eb372>] __irq_exit_rcu+0x132/0x180
Sean Christopherson Oct. 5, 2023, 1:53 a.m. UTC | #2
On Fri, Sep 08, 2023, Anish Moorthy wrote:
> Implement KVM_CAP_MEMORY_FAULT_INFO for uaccess failures in
> kvm_vcpu_read/write_guest_page()

Why?  (rhetorical question)
Anish Moorthy Oct. 5, 2023, 11:03 p.m. UTC | #3
Dropping as per
https://lore.kernel.org/kvm/ZR88w9W62qsZDro-@google.com/. Take that,
kernel test robot!
diff mbox series

Patch

diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index e31435179764..13aa2ed11d0d 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -3043,8 +3043,12 @@  int kvm_vcpu_read_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn, void *data,
 			     int offset, int len)
 {
 	struct kvm_memory_slot *slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn);
+	int r = __kvm_read_guest_page(slot, gfn, data, offset, len);
 
-	return __kvm_read_guest_page(slot, gfn, data, offset, len);
+	if (r)
+		kvm_handle_guest_uaccess_fault(vcpu, gfn * PAGE_SIZE + offset,
+					       len, KVM_MEMORY_FAULT_FLAG_READ);
+	return r;
 }
 EXPORT_SYMBOL_GPL(kvm_vcpu_read_guest_page);
 
@@ -3149,8 +3153,12 @@  int kvm_vcpu_write_guest_page(struct kvm_vcpu *vcpu, gfn_t gfn,
 			      const void *data, int offset, int len)
 {
 	struct kvm_memory_slot *slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn);
+	int r = __kvm_write_guest_page(vcpu->kvm, slot, gfn, data, offset, len);
 
-	return __kvm_write_guest_page(vcpu->kvm, slot, gfn, data, offset, len);
+	if (r)
+		kvm_handle_guest_uaccess_fault(vcpu, gfn * PAGE_SIZE + offset,
+					       len, KVM_MEMORY_FAULT_FLAG_WRITE);
+	return r;
 }
 EXPORT_SYMBOL_GPL(kvm_vcpu_write_guest_page);