diff mbox series

[v1,3/5] KVM: selftests: Dirty host pages in dirty_log_test

Message ID 20220819005601.198436-4-gshan@redhat.com (mailing list archive)
State New
Headers show
Series KVM: arm64: Enable ring-based dirty memory tracking | expand

Commit Message

Gavin Shan Aug. 19, 2022, 12:55 a.m. UTC
It's assumed that 1024 host pages, instead of guest pages, are dirtied
in each iteration in guest_code(). The current implementation misses
the case of mismatched page sizes in host and guest. For example,
ARM64 could have 64KB page size in guest, but 4KB page size in host.
(TEST_PAGES_PER_LOOP / 16), instead of TEST_PAGES_PER_LOOP, host pages
are dirtied in every iteration.

Fix the issue by touching all sub-pages when we have mismatched
page sizes in host and guest.

Signed-off-by: Gavin Shan <gshan@redhat.com>
---
 tools/testing/selftests/kvm/dirty_log_test.c | 50 +++++++++++++++-----
 1 file changed, 39 insertions(+), 11 deletions(-)

Comments

Andrew Jones Aug. 19, 2022, 5:28 a.m. UTC | #1
On Fri, Aug 19, 2022 at 08:55:59AM +0800, Gavin Shan wrote:
> It's assumed that 1024 host pages, instead of guest pages, are dirtied
> in each iteration in guest_code(). The current implementation misses
> the case of mismatched page sizes in host and guest. For example,
> ARM64 could have 64KB page size in guest, but 4KB page size in host.
> (TEST_PAGES_PER_LOOP / 16), instead of TEST_PAGES_PER_LOOP, host pages
> are dirtied in every iteration.
> 
> Fix the issue by touching all sub-pages when we have mismatched
> page sizes in host and guest.

I'll let the dirty-log test authors decide what's best to do for this
test, but I'd think we should let the guest continue dirtying its
pages without knowledge of the host pages. Then, adjust the host test
code to assert all sub-pages, other than the ones it expects the guest
to have written, remain untouched.

Thanks,
drew
Gavin Shan Aug. 22, 2022, 6:29 a.m. UTC | #2
Hi Drew,

On 8/19/22 3:28 PM, Andrew Jones wrote:
> On Fri, Aug 19, 2022 at 08:55:59AM +0800, Gavin Shan wrote:
>> It's assumed that 1024 host pages, instead of guest pages, are dirtied
>> in each iteration in guest_code(). The current implementation misses
>> the case of mismatched page sizes in host and guest. For example,
>> ARM64 could have 64KB page size in guest, but 4KB page size in host.
>> (TEST_PAGES_PER_LOOP / 16), instead of TEST_PAGES_PER_LOOP, host pages
>> are dirtied in every iteration.
>>
>> Fix the issue by touching all sub-pages when we have mismatched
>> page sizes in host and guest.
> 
> I'll let the dirty-log test authors decide what's best to do for this
> test, but I'd think we should let the guest continue dirtying its
> pages without knowledge of the host pages. Then, adjust the host test
> code to assert all sub-pages, other than the ones it expects the guest
> to have written, remain untouched.
> 

I don't think what is clarified in the change log is correct. The current
implementation already had the logic to handle the mismatched page sizes
in vm_dirty_log_verify() where 'step' is used for it by fetching value
from vm_num_host_pages(mode, 1). Please ignore this patch for now, as
explained below.

The issue I have is the 'dirty_log_test' hangs when I have 4KB host page size
and 64KB guest page size. It seems the vcpu doesn't exit due to full ring
buffer state or kick-off. I will have more investigations to figure out the
root cause.

# ./dirty_log_test -M dirty-ring -m 7
Setting log mode to: 'dirty-ring'
Test iterations: 32, interval: 10 (ms)
Testing guest mode: PA-bits:40,  VA-bits:48, 64K pages
guest physical test memory offset: 0xffbffc0000
vcpu stops because vcpu is kicked out...
Notifying vcpu to continue
vcpu continues now.
Iteration 1 collected 1903 pages
<no more output>

'dirty_lot_test' works well when both host and guest have 4KB page size.

# ./dirty_log_test -M dirty-ring -m 5
Setting log mode to: 'dirty-ring'
Test iterations: 32, interval: 10 (ms)
Testing guest mode: PA-bits:40,  VA-bits:48,  4K pages
guest physical test memory offset: 0xffbfffc000
vcpu stops because vcpu is kicked out...
Notifying vcpu to continue
vcpu continues now.
   :
Dirtied 1006592 pages
Total bits checked: dirty (1020487), clear (7106070), track_next (974104)

Thanks,
Gavin
Gavin Shan Aug. 23, 2022, 3:09 a.m. UTC | #3
Hi Drew,

On 8/22/22 4:29 PM, Gavin Shan wrote:
> On 8/19/22 3:28 PM, Andrew Jones wrote:
>> On Fri, Aug 19, 2022 at 08:55:59AM +0800, Gavin Shan wrote:
>>> It's assumed that 1024 host pages, instead of guest pages, are dirtied
>>> in each iteration in guest_code(). The current implementation misses
>>> the case of mismatched page sizes in host and guest. For example,
>>> ARM64 could have 64KB page size in guest, but 4KB page size in host.
>>> (TEST_PAGES_PER_LOOP / 16), instead of TEST_PAGES_PER_LOOP, host pages
>>> are dirtied in every iteration.
>>>
>>> Fix the issue by touching all sub-pages when we have mismatched
>>> page sizes in host and guest.
>>
>> I'll let the dirty-log test authors decide what's best to do for this
>> test, but I'd think we should let the guest continue dirtying its
>> pages without knowledge of the host pages. Then, adjust the host test
>> code to assert all sub-pages, other than the ones it expects the guest
>> to have written, remain untouched.
>>
> 
> I don't think what is clarified in the change log is correct. The current
> implementation already had the logic to handle the mismatched page sizes
> in vm_dirty_log_verify() where 'step' is used for it by fetching value
> from vm_num_host_pages(mode, 1). Please ignore this patch for now, as
> explained below.
> 
> The issue I have is the 'dirty_log_test' hangs when I have 4KB host page size
> and 64KB guest page size. It seems the vcpu doesn't exit due to full ring
> buffer state or kick-off. I will have more investigations to figure out the
> root cause.
> 

[...]

Please ignore this PATCH[3/5], I think this should be fixed by selecting
correct dirty ring count and the fix will be folded to PATCH[5/5] in next
revision.

In dirty_log_test, we have 1GB memory for guest to write and make them
dirty. When we have mismatch page sizes on host and guest, which is either
4kb-host-64kb-guest or 64kb-host-4kb-guest apart from 16kb case, 16384 host
pages are dirtied in each iteration. The default dirty ring count is 65536.
So the vcpu never exit due to full-dirty-ring-buffer state. This leads the
guest's code keep running and the dirty log isn't collected by the main
thread.

     #define TEST_DIRTY_RING_COUNT           65536

     dirty_pages_per_iteration = (0x40000000 / 0x10000)
                               = 0x4000
                               = 16384

Thanks,
Gavin
diff mbox series

Patch

diff --git a/tools/testing/selftests/kvm/dirty_log_test.c b/tools/testing/selftests/kvm/dirty_log_test.c
index 9c883c94d478..50b02186ce12 100644
--- a/tools/testing/selftests/kvm/dirty_log_test.c
+++ b/tools/testing/selftests/kvm/dirty_log_test.c
@@ -70,6 +70,7 @@ 
  * that may change.
  */
 static uint64_t host_page_size;
+static uint64_t host_num_pages;
 static uint64_t guest_page_size;
 static uint64_t guest_num_pages;
 static uint64_t random_array[TEST_PAGES_PER_LOOP];
@@ -94,8 +95,23 @@  static uint64_t guest_test_virt_mem = DEFAULT_GUEST_TEST_MEM;
  */
 static void guest_code(void)
 {
+	uint64_t num_pages, page_size, sub_page_size;
 	uint64_t addr;
-	int i;
+	int pages_per_loop, i, j;
+
+	/*
+	 * The page sizes on host and VM could be different. We need
+	 * to perform writing on all sub-pages.
+	 */
+	if (host_page_size >= guest_page_size) {
+		num_pages = host_num_pages;
+		page_size = host_page_size;
+		sub_page_size = host_page_size;
+	} else {
+		num_pages = guest_num_pages;
+		page_size = guest_page_size;
+		sub_page_size = host_page_size;
+	}
 
 	/*
 	 * On s390x, all pages of a 1M segment are initially marked as dirty
@@ -103,18 +119,29 @@  static void guest_code(void)
 	 * To compensate this specialty in this test, we need to touch all
 	 * pages during the first iteration.
 	 */
-	for (i = 0; i < guest_num_pages; i++) {
-		addr = guest_test_virt_mem + i * guest_page_size;
-		*(uint64_t *)addr = READ_ONCE(iteration);
+	for (i = 0; i < num_pages; i++) {
+		addr = guest_test_virt_mem + i * page_size;
+		addr = align_down(addr, page_size);
+
+		for (j = 0; j < page_size / sub_page_size; j++) {
+			*(uint64_t *)(addr + j * sub_page_size) =
+				READ_ONCE(iteration);
+		}
 	}
 
+	pages_per_loop = (TEST_PAGES_PER_LOOP * sub_page_size) / page_size;
+
 	while (true) {
-		for (i = 0; i < TEST_PAGES_PER_LOOP; i++) {
+		for (i = 0; i < pages_per_loop; i++) {
 			addr = guest_test_virt_mem;
-			addr += (READ_ONCE(random_array[i]) % guest_num_pages)
-				* guest_page_size;
-			addr = align_down(addr, host_page_size);
-			*(uint64_t *)addr = READ_ONCE(iteration);
+			addr += (READ_ONCE(random_array[i]) % num_pages)
+				* page_size;
+			addr = align_down(addr, page_size);
+
+			for (j = 0; j < page_size / sub_page_size; j++) {
+				*(uint64_t *)(addr + j * sub_page_size) =
+					READ_ONCE(iteration);
+			}
 		}
 
 		/* Tell the host that we need more random numbers */
@@ -713,14 +740,14 @@  static void run_test(enum vm_guest_mode mode, void *arg)
 		       2ul << (DIRTY_MEM_BITS - PAGE_SHIFT_4K), guest_code);
 
 	guest_page_size = vm->page_size;
+	host_page_size = getpagesize();
+
 	/*
 	 * A little more than 1G of guest page sized pages.  Cover the
 	 * case where the size is not aligned to 64 pages.
 	 */
 	guest_num_pages = (1ul << (DIRTY_MEM_BITS - vm->page_shift)) + 3;
 	guest_num_pages = vm_adjust_num_guest_pages(mode, guest_num_pages);
-
-	host_page_size = getpagesize();
 	host_num_pages = vm_num_host_pages(mode, guest_num_pages);
 
 	if (!p->phys_offset) {
@@ -760,6 +787,7 @@  static void run_test(enum vm_guest_mode mode, void *arg)
 	sync_global_to_guest(vm, host_page_size);
 	sync_global_to_guest(vm, guest_page_size);
 	sync_global_to_guest(vm, guest_test_virt_mem);
+	sync_global_to_guest(vm, host_num_pages);
 	sync_global_to_guest(vm, guest_num_pages);
 
 	/* Start the iterations */