| Message ID | 20250107-virtual_address_range-tests-v1-3-3834a2fb47fe@linutronix.de (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | selftests/mm: virtual_address_range: Two bugfixes and a cleanup | expand |
On Wed, Jan 08, 2025 at 11:39:40AM +0530, Dev Jain wrote: > > On 07/01/25 8:44 pm, Thomas Weißschuh wrote: > > During the execution of validate_complete_va_space() a lot of memory is > > on the VM subsystem. When running on a low memory subsystem an OOM may > > be triggered, when writing to the dump file as the filesystem may also > > require memory. > > > > On my test system with 1100MiB physical memory: > > > > Tasks state (memory values in pages): > > [ pid ] uid tgid total_vm rss rss_anon rss_file rss_shmem pgtables_bytes swapents oom_score_adj name > > [ 57] 0 57 34359215953 695 256 0 439 1064390656 0 0 virtual_address > > > > Out of memory: Killed process 57 (virtual_address) total-vm:137436863812kB, anon-rss:1024kB, file-rss:0kB, shmem-rss:1756kB, UID:0 pgtables:1039444kB oom_score_adj:0 > > <snip> > > fault_in_iov_iter_readable+0x4a/0xd0 > > generic_perform_write+0x9c/0x280 > > shmem_file_write_iter+0x86/0x90 > > vfs_write+0x29c/0x480 > > ksys_write+0x6c/0xe0 > > do_syscall_64+0x9e/0x1a0 > > entry_SYSCALL_64_after_hwframe+0x77/0x7f > > > > Write the dumped data into /dev/null instead which does not require > > additional memory during write(), making the code simpler as a > > side-effect. > > > > Signed-off-by: Thomas Weißschuh<thomas.weissschuh@linutronix.de> > > --- > > tools/testing/selftests/mm/virtual_address_range.c | 6 ++---- > > 1 file changed, 2 insertions(+), 4 deletions(-) > > > > diff --git a/tools/testing/selftests/mm/virtual_address_range.c b/tools/testing/selftests/mm/virtual_address_range.c > > index 484f82c7b7c871f82a7d9ec6d6c649f2ab1eb0cd..4042fd878acd702d23da2c3293292de33bd48143 100644 > > --- a/tools/testing/selftests/mm/virtual_address_range.c > > +++ b/tools/testing/selftests/mm/virtual_address_range.c > > @@ -103,10 +103,9 @@ static int validate_complete_va_space(void) > > FILE *file; > > int fd; > > - fd = open("va_dump", O_CREAT | O_WRONLY, 0600); > > - unlink("va_dump"); > > + fd = open("/dev/null", O_WRONLY); > > if (fd < 0) { > > - ksft_test_result_skip("cannot create or open dump file\n"); > > + ksft_test_result_skip("cannot create or open /dev/null\n"); > > ksft_finished(); > > } > > @@ -152,7 +151,6 @@ static int validate_complete_va_space(void) > > while (start_addr + hop < end_addr) { > > if (write(fd, (void *)(start_addr + hop), 1) != 1) > > return 1; > > - lseek(fd, 0, SEEK_SET); > > hop += MAP_CHUNK_SIZE; > > } > > > > The reason I had not used /dev/null was that write() was succeeding to /dev/null > even from an address not in my VA space. I was puzzled about this behaviour of > /dev/null and I chose to ignore it and just use a real file. That makes sense and I can reproduce your example. Switching to another dummy file which reads the written data like /dev/random also leads to OOM, so wouldn't help either. Thanks for the explanation. @Andrew, could you drop this patch? > To test this behaviour, run the following program: [..] PS: Your mail contained HTML and did not make it to the list archives. (And the text variant of the example program was corrupted)
On 08.01.25 07:09, Dev Jain wrote: > > On 07/01/25 8:44 pm, Thomas Weißschuh wrote: >> During the execution of validate_complete_va_space() a lot of memory is >> on the VM subsystem. When running on a low memory subsystem an OOM may >> be triggered, when writing to the dump file as the filesystem may also >> require memory. >> >> On my test system with 1100MiB physical memory: >> >> Tasks state (memory values in pages): >> [ pid ] uid tgid total_vm rss rss_anon rss_file rss_shmem pgtables_bytes swapents oom_score_adj name >> [ 57] 0 57 34359215953 695 256 0 439 1064390656 0 0 virtual_address >> >> Out of memory: Killed process 57 (virtual_address) total-vm:137436863812kB, anon-rss:1024kB, file-rss:0kB, shmem-rss:1756kB, UID:0 pgtables:1039444kB oom_score_adj:0 >> <snip> >> fault_in_iov_iter_readable+0x4a/0xd0 >> generic_perform_write+0x9c/0x280 >> shmem_file_write_iter+0x86/0x90 >> vfs_write+0x29c/0x480 >> ksys_write+0x6c/0xe0 >> do_syscall_64+0x9e/0x1a0 >> entry_SYSCALL_64_after_hwframe+0x77/0x7f >> >> Write the dumped data into /dev/null instead which does not require >> additional memory during write(), making the code simpler as a >> side-effect. >> >> Signed-off-by: Thomas Weißschuh<thomas.weissschuh@linutronix.de> >> --- >> tools/testing/selftests/mm/virtual_address_range.c | 6 ++---- >> 1 file changed, 2 insertions(+), 4 deletions(-) >> >> diff --git a/tools/testing/selftests/mm/virtual_address_range.c b/tools/testing/selftests/mm/virtual_address_range.c >> index 484f82c7b7c871f82a7d9ec6d6c649f2ab1eb0cd..4042fd878acd702d23da2c3293292de33bd48143 100644 >> --- a/tools/testing/selftests/mm/virtual_address_range.c >> +++ b/tools/testing/selftests/mm/virtual_address_range.c >> @@ -103,10 +103,9 @@ static int validate_complete_va_space(void) >> FILE *file; >> int fd; >> >> - fd = open("va_dump", O_CREAT | O_WRONLY, 0600); >> - unlink("va_dump"); >> + fd = open("/dev/null", O_WRONLY); >> if (fd < 0) { >> - ksft_test_result_skip("cannot create or open dump file\n"); >> + ksft_test_result_skip("cannot create or open /dev/null\n"); >> ksft_finished(); >> } >> >> @@ -152,7 +151,6 @@ static int validate_complete_va_space(void) >> while (start_addr + hop < end_addr) { >> if (write(fd, (void *)(start_addr + hop), 1) != 1) >> return 1; >> - lseek(fd, 0, SEEK_SET); >> >> hop += MAP_CHUNK_SIZE; >> } >> > > The reason I had not used /dev/null was that write() was succeeding to /dev/null > even from an address not in my VA space. I was puzzled about this behaviour of > /dev/null and I chose to ignore it and just use a real file. > > To test this behaviour, run the following program: > > #include <stdio.h> > #include <stdlib.h> > #include <unistd.h> > #include <fcntl.h> > #include <sys/mman.h> > intmain() > { > intfd; > fd = open("va_dump", O_CREAT| O_WRONLY, 0600); > unlink("va_dump"); > // fd = open("/dev/null", O_WRONLY); > intret = munmap((void*)(1UL<< 30), 100); > if(!ret) > printf("munmap succeeded\n"); > intres = write(fd, (void*)(1UL<< 30), 1); > if(res == 1) > printf("write succeeded\n"); > return0; > } > The write will fail as expected, but if you comment out the va_dump > lines and use /dev/null, the write will succeed. What exactly do we want to achieve with the write? Verify that the output of /proc/self/map is reasonable and we can actually resolve a fault / map a page? Why not access the memory directly+signal handler or using /proc/self/mem, so you can avoid the temp file completely?
On 08/01/25 7:00 pm, David Hildenbrand wrote: > On 08.01.25 07:09, Dev Jain wrote: >> >> On 07/01/25 8:44 pm, Thomas Weißschuh wrote: >>> During the execution of validate_complete_va_space() a lot of memory is >>> on the VM subsystem. When running on a low memory subsystem an OOM may >>> be triggered, when writing to the dump file as the filesystem may also >>> require memory. >>> >>> On my test system with 1100MiB physical memory: >>> >>> Tasks state (memory values in pages): >>> [ pid ] uid tgid total_vm rss rss_anon rss_file >>> rss_shmem pgtables_bytes swapents oom_score_adj name >>> [ 57] 0 57 34359215953 695 256 0 439 >>> 1064390656 0 0 virtual_address >>> >>> Out of memory: Killed process 57 (virtual_address) >>> total-vm:137436863812kB, anon-rss:1024kB, file-rss:0kB, >>> shmem-rss:1756kB, UID:0 pgtables:1039444kB oom_score_adj:0 >>> <snip> >>> fault_in_iov_iter_readable+0x4a/0xd0 >>> generic_perform_write+0x9c/0x280 >>> shmem_file_write_iter+0x86/0x90 >>> vfs_write+0x29c/0x480 >>> ksys_write+0x6c/0xe0 >>> do_syscall_64+0x9e/0x1a0 >>> entry_SYSCALL_64_after_hwframe+0x77/0x7f >>> >>> Write the dumped data into /dev/null instead which does not require >>> additional memory during write(), making the code simpler as a >>> side-effect. >>> >>> Signed-off-by: Thomas Weißschuh<thomas.weissschuh@linutronix.de> >>> --- >>> tools/testing/selftests/mm/virtual_address_range.c | 6 ++---- >>> 1 file changed, 2 insertions(+), 4 deletions(-) >>> >>> diff --git a/tools/testing/selftests/mm/virtual_address_range.c >>> b/tools/testing/selftests/mm/virtual_address_range.c >>> index >>> 484f82c7b7c871f82a7d9ec6d6c649f2ab1eb0cd..4042fd878acd702d23da2c3293292de33bd48143 >>> 100644 >>> --- a/tools/testing/selftests/mm/virtual_address_range.c >>> +++ b/tools/testing/selftests/mm/virtual_address_range.c >>> @@ -103,10 +103,9 @@ static int validate_complete_va_space(void) >>> FILE *file; >>> int fd; >>> - fd = open("va_dump", O_CREAT | O_WRONLY, 0600); >>> - unlink("va_dump"); >>> + fd = open("/dev/null", O_WRONLY); >>> if (fd < 0) { >>> - ksft_test_result_skip("cannot create or open dump file\n"); >>> + ksft_test_result_skip("cannot create or open /dev/null\n"); >>> ksft_finished(); >>> } > >> >> @@ -152,7 +151,6 @@ static int validate_complete_va_space(void) >>> while (start_addr + hop < end_addr) { >>> if (write(fd, (void *)(start_addr + hop), 1) != 1) >>> return 1; >>> - lseek(fd, 0, SEEK_SET); >>> hop += MAP_CHUNK_SIZE; >>> } >>> >> >> The reason I had not used /dev/null was that write() was succeeding >> to /dev/null >> even from an address not in my VA space. I was puzzled about this >> behaviour of >> /dev/null and I chose to ignore it and just use a real file. >> >> To test this behaviour, run the following program: >> >> #include <stdio.h> >> #include <stdlib.h> >> #include <unistd.h> >> #include <fcntl.h> >> #include <sys/mman.h> >> intmain() >> { >> intfd; >> fd = open("va_dump", O_CREAT| O_WRONLY, 0600); >> unlink("va_dump"); >> // fd = open("/dev/null", O_WRONLY); >> intret = munmap((void*)(1UL<< 30), 100); >> if(!ret) >> printf("munmap succeeded\n"); >> intres = write(fd, (void*)(1UL<< 30), 1); >> if(res == 1) >> printf("write succeeded\n"); >> return0; >> } >> The write will fail as expected, but if you comment out the va_dump >> lines and use /dev/null, the write will succeed. > > What exactly do we want to achieve with the write? Verify that the > output of /proc/self/map is reasonable and we can actually resolve a > fault / map a page? > > Why not access the memory directly+signal handler or using > /proc/self/mem, so you can avoid the temp file completely? > We want to determine whether an address belongs to our address space. The proper way to do that is to access the memory, get a segfault and jump to signal handler. I wanted to avoid this code churn, so chose to use write() so that I can validate the address without getting a segfault.
diff --git a/tools/testing/selftests/mm/virtual_address_range.c b/tools/testing/selftests/mm/virtual_address_range.c index 484f82c7b7c871f82a7d9ec6d6c649f2ab1eb0cd..4042fd878acd702d23da2c3293292de33bd48143 100644 --- a/tools/testing/selftests/mm/virtual_address_range.c +++ b/tools/testing/selftests/mm/virtual_address_range.c @@ -103,10 +103,9 @@ static int validate_complete_va_space(void) FILE *file; int fd; - fd = open("va_dump", O_CREAT | O_WRONLY, 0600); - unlink("va_dump"); + fd = open("/dev/null", O_WRONLY); if (fd < 0) { - ksft_test_result_skip("cannot create or open dump file\n"); + ksft_test_result_skip("cannot create or open /dev/null\n"); ksft_finished(); } @@ -152,7 +151,6 @@ static int validate_complete_va_space(void) while (start_addr + hop < end_addr) { if (write(fd, (void *)(start_addr + hop), 1) != 1) return 1; - lseek(fd, 0, SEEK_SET); hop += MAP_CHUNK_SIZE; }
During the execution of validate_complete_va_space() a lot of memory is on the VM subsystem. When running on a low memory subsystem an OOM may be triggered, when writing to the dump file as the filesystem may also require memory. On my test system with 1100MiB physical memory: Tasks state (memory values in pages): [ pid ] uid tgid total_vm rss rss_anon rss_file rss_shmem pgtables_bytes swapents oom_score_adj name [ 57] 0 57 34359215953 695 256 0 439 1064390656 0 0 virtual_address Out of memory: Killed process 57 (virtual_address) total-vm:137436863812kB, anon-rss:1024kB, file-rss:0kB, shmem-rss:1756kB, UID:0 pgtables:1039444kB oom_score_adj:0 <snip> fault_in_iov_iter_readable+0x4a/0xd0 generic_perform_write+0x9c/0x280 shmem_file_write_iter+0x86/0x90 vfs_write+0x29c/0x480 ksys_write+0x6c/0xe0 do_syscall_64+0x9e/0x1a0 entry_SYSCALL_64_after_hwframe+0x77/0x7f Write the dumped data into /dev/null instead which does not require additional memory during write(), making the code simpler as a side-effect. Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de> --- tools/testing/selftests/mm/virtual_address_range.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)