| Message ID | 20220305091205.4188398-1-yukuai3@huawei.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | support concurrent sync io for bfq on a specail occasion | expand |
friendly ping ... 在 2022/03/05 17:11, Yu Kuai 写道: > Currently, bfq can't handle sync io concurrently as long as they > are not issued from root group. This is because > 'bfqd->num_groups_with_pending_reqs > 0' is always true in > bfq_asymmetric_scenario(). > > This patchset tries to support concurrent sync io if all the sync ios > are issued from the same cgroup: > > 1) Count root_group into 'num_groups_with_pending_reqs', patch 1-5; > > 2) Don't idle if 'num_groups_with_pending_reqs' is 1, patch 6; > > 3) Don't count the group if the group doesn't have pending requests, > while it's child groups may have pending requests, patch 7; > > This is because, for example: > if sync ios are issued from cgroup /root/c1/c2, root, c1 and c2 > will all be counted into 'num_groups_with_pending_reqs', > which makes it impossible to handle sync ios concurrently. > > 4) Decrease 'num_groups_with_pending_reqs' when the last queue completes > all the requests, while child groups may still have pending > requests, patch 8-10; > > This is because, for example: > t1 issue sync io on root group, t2 and t3 issue sync io on the same > child group. num_groups_with_pending_reqs is 2 now. > After t1 stopped, num_groups_with_pending_reqs is still 2. sync io from > t2 and t3 still can't be handled concurrently. > > fio test script: startdelay is used to avoid queue merging > [global] > filename=/dev/nvme0n1 > allow_mounted_write=0 > ioengine=psync > direct=1 > ioscheduler=bfq > offset_increment=10g > group_reporting > rw=randwrite > bs=4k > > [test1] > numjobs=1 > > [test2] > startdelay=1 > numjobs=1 > > [test3] > startdelay=2 > numjobs=1 > > [test4] > startdelay=3 > numjobs=1 > > [test5] > startdelay=4 > numjobs=1 > > [test6] > startdelay=5 > numjobs=1 > > [test7] > startdelay=6 > numjobs=1 > > [test8] > startdelay=7 > numjobs=1 > > test result: > running fio on root cgroup > v5.17-rc6: 550 Mib/s > v5.17-rc6-patched: 550 Mib/s > > running fio on non-root cgroup > v5.17-rc6: 349 Mib/s > v5.17-rc6-patched: 550 Mib/s > > Yu Kuai (11): > block, bfq: add new apis to iterate bfq entities > block, bfq: apply news apis where root group is not expected > block, bfq: cleanup for __bfq_activate_requeue_entity() > block, bfq: move the increasement of 'num_groups_with_pending_reqs' to > it's caller > block, bfq: count root group into 'num_groups_with_pending_reqs' > block, bfq: do not idle if only one cgroup is activated > block, bfq: only count parent bfqg when bfqq is activated > block, bfq: record how many queues have pending requests in bfq_group > block, bfq: move forward __bfq_weights_tree_remove() > block, bfq: decrease 'num_groups_with_pending_reqs' earlier > block, bfq: cleanup bfqq_group() > > block/bfq-cgroup.c | 13 +++---- > block/bfq-iosched.c | 87 +++++++++++++++++++++++---------------------- > block/bfq-iosched.h | 41 +++++++++++++-------- > block/bfq-wf2q.c | 56 +++++++++++++++-------------- > 4 files changed, 106 insertions(+), 91 deletions(-) >
friendly ping ... 在 2022/03/11 14:31, yukuai (C) 写道: > friendly ping ... > > 在 2022/03/05 17:11, Yu Kuai 写道: >> Currently, bfq can't handle sync io concurrently as long as they >> are not issued from root group. This is because >> 'bfqd->num_groups_with_pending_reqs > 0' is always true in >> bfq_asymmetric_scenario(). >> >> This patchset tries to support concurrent sync io if all the sync ios >> are issued from the same cgroup: >> >> 1) Count root_group into 'num_groups_with_pending_reqs', patch 1-5; >> >> 2) Don't idle if 'num_groups_with_pending_reqs' is 1, patch 6; >> >> 3) Don't count the group if the group doesn't have pending requests, >> while it's child groups may have pending requests, patch 7; >> >> This is because, for example: >> if sync ios are issued from cgroup /root/c1/c2, root, c1 and c2 >> will all be counted into 'num_groups_with_pending_reqs', >> which makes it impossible to handle sync ios concurrently. >> >> 4) Decrease 'num_groups_with_pending_reqs' when the last queue completes >> all the requests, while child groups may still have pending >> requests, patch 8-10; >> >> This is because, for example: >> t1 issue sync io on root group, t2 and t3 issue sync io on the same >> child group. num_groups_with_pending_reqs is 2 now. >> After t1 stopped, num_groups_with_pending_reqs is still 2. sync io from >> t2 and t3 still can't be handled concurrently. >> >> fio test script: startdelay is used to avoid queue merging >> [global] >> filename=/dev/nvme0n1 >> allow_mounted_write=0 >> ioengine=psync >> direct=1 >> ioscheduler=bfq >> offset_increment=10g >> group_reporting >> rw=randwrite >> bs=4k >> >> [test1] >> numjobs=1 >> >> [test2] >> startdelay=1 >> numjobs=1 >> >> [test3] >> startdelay=2 >> numjobs=1 >> >> [test4] >> startdelay=3 >> numjobs=1 >> >> [test5] >> startdelay=4 >> numjobs=1 >> >> [test6] >> startdelay=5 >> numjobs=1 >> >> [test7] >> startdelay=6 >> numjobs=1 >> >> [test8] >> startdelay=7 >> numjobs=1 >> >> test result: >> running fio on root cgroup >> v5.17-rc6: 550 Mib/s >> v5.17-rc6-patched: 550 Mib/s >> >> running fio on non-root cgroup >> v5.17-rc6: 349 Mib/s >> v5.17-rc6-patched: 550 Mib/s >> >> Yu Kuai (11): >> block, bfq: add new apis to iterate bfq entities >> block, bfq: apply news apis where root group is not expected >> block, bfq: cleanup for __bfq_activate_requeue_entity() >> block, bfq: move the increasement of 'num_groups_with_pending_reqs' to >> it's caller >> block, bfq: count root group into 'num_groups_with_pending_reqs' >> block, bfq: do not idle if only one cgroup is activated >> block, bfq: only count parent bfqg when bfqq is activated >> block, bfq: record how many queues have pending requests in bfq_group >> block, bfq: move forward __bfq_weights_tree_remove() >> block, bfq: decrease 'num_groups_with_pending_reqs' earlier >> block, bfq: cleanup bfqq_group() >> >> block/bfq-cgroup.c | 13 +++---- >> block/bfq-iosched.c | 87 +++++++++++++++++++++++---------------------- >> block/bfq-iosched.h | 41 +++++++++++++-------- >> block/bfq-wf2q.c | 56 +++++++++++++++-------------- >> 4 files changed, 106 insertions(+), 91 deletions(-) >>
Hi, could you please add pointers to the thread(s) where we have already revised this series (if we have). I don't see any reference to that in this cover letter. Paolo > Il giorno 17 mar 2022, alle ore 02:49, yukuai (C) <yukuai3@huawei.com> ha scritto: > > friendly ping ... > > 在 2022/03/11 14:31, yukuai (C) 写道: >> friendly ping ... >> 在 2022/03/05 17:11, Yu Kuai 写道: >>> Currently, bfq can't handle sync io concurrently as long as they >>> are not issued from root group. This is because >>> 'bfqd->num_groups_with_pending_reqs > 0' is always true in >>> bfq_asymmetric_scenario(). >>> >>> This patchset tries to support concurrent sync io if all the sync ios >>> are issued from the same cgroup: >>> >>> 1) Count root_group into 'num_groups_with_pending_reqs', patch 1-5; >>> >>> 2) Don't idle if 'num_groups_with_pending_reqs' is 1, patch 6; >>> >>> 3) Don't count the group if the group doesn't have pending requests, >>> while it's child groups may have pending requests, patch 7; >>> >>> This is because, for example: >>> if sync ios are issued from cgroup /root/c1/c2, root, c1 and c2 >>> will all be counted into 'num_groups_with_pending_reqs', >>> which makes it impossible to handle sync ios concurrently. >>> >>> 4) Decrease 'num_groups_with_pending_reqs' when the last queue completes >>> all the requests, while child groups may still have pending >>> requests, patch 8-10; >>> >>> This is because, for example: >>> t1 issue sync io on root group, t2 and t3 issue sync io on the same >>> child group. num_groups_with_pending_reqs is 2 now. >>> After t1 stopped, num_groups_with_pending_reqs is still 2. sync io from >>> t2 and t3 still can't be handled concurrently. >>> >>> fio test script: startdelay is used to avoid queue merging >>> [global] >>> filename=/dev/nvme0n1 >>> allow_mounted_write=0 >>> ioengine=psync >>> direct=1 >>> ioscheduler=bfq >>> offset_increment=10g >>> group_reporting >>> rw=randwrite >>> bs=4k >>> >>> [test1] >>> numjobs=1 >>> >>> [test2] >>> startdelay=1 >>> numjobs=1 >>> >>> [test3] >>> startdelay=2 >>> numjobs=1 >>> >>> [test4] >>> startdelay=3 >>> numjobs=1 >>> >>> [test5] >>> startdelay=4 >>> numjobs=1 >>> >>> [test6] >>> startdelay=5 >>> numjobs=1 >>> >>> [test7] >>> startdelay=6 >>> numjobs=1 >>> >>> [test8] >>> startdelay=7 >>> numjobs=1 >>> >>> test result: >>> running fio on root cgroup >>> v5.17-rc6: 550 Mib/s >>> v5.17-rc6-patched: 550 Mib/s >>> >>> running fio on non-root cgroup >>> v5.17-rc6: 349 Mib/s >>> v5.17-rc6-patched: 550 Mib/s >>> >>> Yu Kuai (11): >>> block, bfq: add new apis to iterate bfq entities >>> block, bfq: apply news apis where root group is not expected >>> block, bfq: cleanup for __bfq_activate_requeue_entity() >>> block, bfq: move the increasement of 'num_groups_with_pending_reqs' to >>> it's caller >>> block, bfq: count root group into 'num_groups_with_pending_reqs' >>> block, bfq: do not idle if only one cgroup is activated >>> block, bfq: only count parent bfqg when bfqq is activated >>> block, bfq: record how many queues have pending requests in bfq_group >>> block, bfq: move forward __bfq_weights_tree_remove() >>> block, bfq: decrease 'num_groups_with_pending_reqs' earlier >>> block, bfq: cleanup bfqq_group() >>> >>> block/bfq-cgroup.c | 13 +++---- >>> block/bfq-iosched.c | 87 +++++++++++++++++++++++---------------------- >>> block/bfq-iosched.h | 41 +++++++++++++-------- >>> block/bfq-wf2q.c | 56 +++++++++++++++-------------- >>> 4 files changed, 106 insertions(+), 91 deletions(-) >>>
在 2022/03/18 20:38, Paolo Valente 写道: > Hi, > could you please add pointers to the thread(s) where we have already revised this series (if we have). I don't see any reference to that in this cover letter. Hi, Ok, sorry for that, following is the previours threads. This is a new patchset after RFC - Fix some term in commit messages and comments - Add some cleanup patches New RFC: use a new solution, and it has little relevance to previous versions. https://lore.kernel.org/lkml/20211127101132.486806-1-yukuai3@huawei.com/T/ - as suggested by Paolo, count root group into 'num_groups_with_pending_reqs' instead of handling root group separately. - Change the patchset title - New changes about when to modify 'num_groups_with_pending_reqs' Orignal v4: https://lore.kernel.org/lkml/20211014014556.3597008-2-yukuai3@huawei.com/t/ - fix a compile warning when CONFIG_BLK_CGROUP is not enabled. Orignal v3: https://www.spinics.net/lists/linux-block/msg74836.html - Instead of tracking each queue in root group, tracking root group directly just like non-root group does. - remove patch 3,4 from these series. Orignal v2: https://lore.kernel.org/lkml/20210806020826.1407257-1-yukuai3@huawei.com/ - as suggested by Paolo, add support to track if root_group have any pending requests, and use that to handle the situation when only one group is activated while root group doesn't have any pending requests. - modify commit message in patch 2
friendly ping ... 在 2022/03/17 9:49, yukuai (C) 写道: > friendly ping ... > > 在 2022/03/11 14:31, yukuai (C) 写道: >> friendly ping ... >> >> 在 2022/03/05 17:11, Yu Kuai 写道: >>> Currently, bfq can't handle sync io concurrently as long as they >>> are not issued from root group. This is because >>> 'bfqd->num_groups_with_pending_reqs > 0' is always true in >>> bfq_asymmetric_scenario(). >>> >>> This patchset tries to support concurrent sync io if all the sync ios >>> are issued from the same cgroup: >>> >>> 1) Count root_group into 'num_groups_with_pending_reqs', patch 1-5; >>> >>> 2) Don't idle if 'num_groups_with_pending_reqs' is 1, patch 6; >>> >>> 3) Don't count the group if the group doesn't have pending requests, >>> while it's child groups may have pending requests, patch 7; >>> >>> This is because, for example: >>> if sync ios are issued from cgroup /root/c1/c2, root, c1 and c2 >>> will all be counted into 'num_groups_with_pending_reqs', >>> which makes it impossible to handle sync ios concurrently. >>> >>> 4) Decrease 'num_groups_with_pending_reqs' when the last queue completes >>> all the requests, while child groups may still have pending >>> requests, patch 8-10; >>> >>> This is because, for example: >>> t1 issue sync io on root group, t2 and t3 issue sync io on the same >>> child group. num_groups_with_pending_reqs is 2 now. >>> After t1 stopped, num_groups_with_pending_reqs is still 2. sync io from >>> t2 and t3 still can't be handled concurrently. >>> >>> fio test script: startdelay is used to avoid queue merging >>> [global] >>> filename=/dev/nvme0n1 >>> allow_mounted_write=0 >>> ioengine=psync >>> direct=1 >>> ioscheduler=bfq >>> offset_increment=10g >>> group_reporting >>> rw=randwrite >>> bs=4k >>> >>> [test1] >>> numjobs=1 >>> >>> [test2] >>> startdelay=1 >>> numjobs=1 >>> >>> [test3] >>> startdelay=2 >>> numjobs=1 >>> >>> [test4] >>> startdelay=3 >>> numjobs=1 >>> >>> [test5] >>> startdelay=4 >>> numjobs=1 >>> >>> [test6] >>> startdelay=5 >>> numjobs=1 >>> >>> [test7] >>> startdelay=6 >>> numjobs=1 >>> >>> [test8] >>> startdelay=7 >>> numjobs=1 >>> >>> test result: >>> running fio on root cgroup >>> v5.17-rc6: 550 Mib/s >>> v5.17-rc6-patched: 550 Mib/s >>> >>> running fio on non-root cgroup >>> v5.17-rc6: 349 Mib/s >>> v5.17-rc6-patched: 550 Mib/s >>> >>> Yu Kuai (11): >>> block, bfq: add new apis to iterate bfq entities >>> block, bfq: apply news apis where root group is not expected >>> block, bfq: cleanup for __bfq_activate_requeue_entity() >>> block, bfq: move the increasement of >>> 'num_groups_with_pending_reqs' to >>> it's caller >>> block, bfq: count root group into 'num_groups_with_pending_reqs' >>> block, bfq: do not idle if only one cgroup is activated >>> block, bfq: only count parent bfqg when bfqq is activated >>> block, bfq: record how many queues have pending requests in bfq_group >>> block, bfq: move forward __bfq_weights_tree_remove() >>> block, bfq: decrease 'num_groups_with_pending_reqs' earlier >>> block, bfq: cleanup bfqq_group() >>> >>> block/bfq-cgroup.c | 13 +++---- >>> block/bfq-iosched.c | 87 +++++++++++++++++++++++---------------------- >>> block/bfq-iosched.h | 41 +++++++++++++-------- >>> block/bfq-wf2q.c | 56 +++++++++++++++-------------- >>> 4 files changed, 106 insertions(+), 91 deletions(-) >>>
friendly ping ... 在 2022/03/25 15:30, yukuai (C) 写道: > friendly ping ... > > 在 2022/03/17 9:49, yukuai (C) 写道: >> friendly ping ... >> >> 在 2022/03/11 14:31, yukuai (C) 写道: >>> friendly ping ... >>> >>> 在 2022/03/05 17:11, Yu Kuai 写道: >>>> Currently, bfq can't handle sync io concurrently as long as they >>>> are not issued from root group. This is because >>>> 'bfqd->num_groups_with_pending_reqs > 0' is always true in >>>> bfq_asymmetric_scenario(). >>>> >>>> This patchset tries to support concurrent sync io if all the sync ios >>>> are issued from the same cgroup: >>>> >>>> 1) Count root_group into 'num_groups_with_pending_reqs', patch 1-5; >>>> >>>> 2) Don't idle if 'num_groups_with_pending_reqs' is 1, patch 6; >>>> >>>> 3) Don't count the group if the group doesn't have pending requests, >>>> while it's child groups may have pending requests, patch 7; >>>> >>>> This is because, for example: >>>> if sync ios are issued from cgroup /root/c1/c2, root, c1 and c2 >>>> will all be counted into 'num_groups_with_pending_reqs', >>>> which makes it impossible to handle sync ios concurrently. >>>> >>>> 4) Decrease 'num_groups_with_pending_reqs' when the last queue >>>> completes >>>> all the requests, while child groups may still have pending >>>> requests, patch 8-10; >>>> >>>> This is because, for example: >>>> t1 issue sync io on root group, t2 and t3 issue sync io on the same >>>> child group. num_groups_with_pending_reqs is 2 now. >>>> After t1 stopped, num_groups_with_pending_reqs is still 2. sync io from >>>> t2 and t3 still can't be handled concurrently. >>>> >>>> fio test script: startdelay is used to avoid queue merging >>>> [global] >>>> filename=/dev/nvme0n1 >>>> allow_mounted_write=0 >>>> ioengine=psync >>>> direct=1 >>>> ioscheduler=bfq >>>> offset_increment=10g >>>> group_reporting >>>> rw=randwrite >>>> bs=4k >>>> >>>> [test1] >>>> numjobs=1 >>>> >>>> [test2] >>>> startdelay=1 >>>> numjobs=1 >>>> >>>> [test3] >>>> startdelay=2 >>>> numjobs=1 >>>> >>>> [test4] >>>> startdelay=3 >>>> numjobs=1 >>>> >>>> [test5] >>>> startdelay=4 >>>> numjobs=1 >>>> >>>> [test6] >>>> startdelay=5 >>>> numjobs=1 >>>> >>>> [test7] >>>> startdelay=6 >>>> numjobs=1 >>>> >>>> [test8] >>>> startdelay=7 >>>> numjobs=1 >>>> >>>> test result: >>>> running fio on root cgroup >>>> v5.17-rc6: 550 Mib/s >>>> v5.17-rc6-patched: 550 Mib/s >>>> >>>> running fio on non-root cgroup >>>> v5.17-rc6: 349 Mib/s >>>> v5.17-rc6-patched: 550 Mib/s >>>> >>>> Yu Kuai (11): >>>> block, bfq: add new apis to iterate bfq entities >>>> block, bfq: apply news apis where root group is not expected >>>> block, bfq: cleanup for __bfq_activate_requeue_entity() >>>> block, bfq: move the increasement of >>>> 'num_groups_with_pending_reqs' to >>>> it's caller >>>> block, bfq: count root group into 'num_groups_with_pending_reqs' >>>> block, bfq: do not idle if only one cgroup is activated >>>> block, bfq: only count parent bfqg when bfqq is activated >>>> block, bfq: record how many queues have pending requests in >>>> bfq_group >>>> block, bfq: move forward __bfq_weights_tree_remove() >>>> block, bfq: decrease 'num_groups_with_pending_reqs' earlier >>>> block, bfq: cleanup bfqq_group() >>>> >>>> block/bfq-cgroup.c | 13 +++---- >>>> block/bfq-iosched.c | 87 >>>> +++++++++++++++++++++++---------------------- >>>> block/bfq-iosched.h | 41 +++++++++++++-------- >>>> block/bfq-wf2q.c | 56 +++++++++++++++-------------- >>>> 4 files changed, 106 insertions(+), 91 deletions(-) >>>>
friendly ping ... 在 2022/04/01 11:43, yukuai (C) 写道: > friendly ping ... > > 在 2022/03/25 15:30, yukuai (C) 写道: >> friendly ping ... >> >> 在 2022/03/17 9:49, yukuai (C) 写道: >>> friendly ping ... >>> >>> 在 2022/03/11 14:31, yukuai (C) 写道: >>>> friendly ping ... >>>> >>>> 在 2022/03/05 17:11, Yu Kuai 写道: >>>>> Currently, bfq can't handle sync io concurrently as long as they >>>>> are not issued from root group. This is because >>>>> 'bfqd->num_groups_with_pending_reqs > 0' is always true in >>>>> bfq_asymmetric_scenario(). >>>>> >>>>> This patchset tries to support concurrent sync io if all the sync ios >>>>> are issued from the same cgroup: >>>>> >>>>> 1) Count root_group into 'num_groups_with_pending_reqs', patch 1-5; >>>>> >>>>> 2) Don't idle if 'num_groups_with_pending_reqs' is 1, patch 6; >>>>> >>>>> 3) Don't count the group if the group doesn't have pending requests, >>>>> while it's child groups may have pending requests, patch 7; >>>>> >>>>> This is because, for example: >>>>> if sync ios are issued from cgroup /root/c1/c2, root, c1 and c2 >>>>> will all be counted into 'num_groups_with_pending_reqs', >>>>> which makes it impossible to handle sync ios concurrently. >>>>> >>>>> 4) Decrease 'num_groups_with_pending_reqs' when the last queue >>>>> completes >>>>> all the requests, while child groups may still have pending >>>>> requests, patch 8-10; >>>>> >>>>> This is because, for example: >>>>> t1 issue sync io on root group, t2 and t3 issue sync io on the same >>>>> child group. num_groups_with_pending_reqs is 2 now. >>>>> After t1 stopped, num_groups_with_pending_reqs is still 2. sync io >>>>> from >>>>> t2 and t3 still can't be handled concurrently. >>>>> >>>>> fio test script: startdelay is used to avoid queue merging >>>>> [global] >>>>> filename=/dev/nvme0n1 >>>>> allow_mounted_write=0 >>>>> ioengine=psync >>>>> direct=1 >>>>> ioscheduler=bfq >>>>> offset_increment=10g >>>>> group_reporting >>>>> rw=randwrite >>>>> bs=4k >>>>> >>>>> [test1] >>>>> numjobs=1 >>>>> >>>>> [test2] >>>>> startdelay=1 >>>>> numjobs=1 >>>>> >>>>> [test3] >>>>> startdelay=2 >>>>> numjobs=1 >>>>> >>>>> [test4] >>>>> startdelay=3 >>>>> numjobs=1 >>>>> >>>>> [test5] >>>>> startdelay=4 >>>>> numjobs=1 >>>>> >>>>> [test6] >>>>> startdelay=5 >>>>> numjobs=1 >>>>> >>>>> [test7] >>>>> startdelay=6 >>>>> numjobs=1 >>>>> >>>>> [test8] >>>>> startdelay=7 >>>>> numjobs=1 >>>>> >>>>> test result: >>>>> running fio on root cgroup >>>>> v5.17-rc6: 550 Mib/s >>>>> v5.17-rc6-patched: 550 Mib/s >>>>> >>>>> running fio on non-root cgroup >>>>> v5.17-rc6: 349 Mib/s >>>>> v5.17-rc6-patched: 550 Mib/s >>>>> >>>>> Yu Kuai (11): >>>>> block, bfq: add new apis to iterate bfq entities >>>>> block, bfq: apply news apis where root group is not expected >>>>> block, bfq: cleanup for __bfq_activate_requeue_entity() >>>>> block, bfq: move the increasement of >>>>> 'num_groups_with_pending_reqs' to >>>>> it's caller >>>>> block, bfq: count root group into 'num_groups_with_pending_reqs' >>>>> block, bfq: do not idle if only one cgroup is activated >>>>> block, bfq: only count parent bfqg when bfqq is activated >>>>> block, bfq: record how many queues have pending requests in >>>>> bfq_group >>>>> block, bfq: move forward __bfq_weights_tree_remove() >>>>> block, bfq: decrease 'num_groups_with_pending_reqs' earlier >>>>> block, bfq: cleanup bfqq_group() >>>>> >>>>> block/bfq-cgroup.c | 13 +++---- >>>>> block/bfq-iosched.c | 87 >>>>> +++++++++++++++++++++++---------------------- >>>>> block/bfq-iosched.h | 41 +++++++++++++-------- >>>>> block/bfq-wf2q.c | 56 +++++++++++++++-------------- >>>>> 4 files changed, 106 insertions(+), 91 deletions(-) >>>>>
On Sat 05-03-22 17:11:54, Yu Kuai wrote: > Currently, bfq can't handle sync io concurrently as long as they > are not issued from root group. This is because > 'bfqd->num_groups_with_pending_reqs > 0' is always true in > bfq_asymmetric_scenario(). > > This patchset tries to support concurrent sync io if all the sync ios > are issued from the same cgroup: > > 1) Count root_group into 'num_groups_with_pending_reqs', patch 1-5; Seeing the complications and special casing for root_group I wonder: Won't we be better off to create fake bfq_sched_data in bfq_data and point root_group->sched_data there? AFAICS it would simplify the code considerably as root_group would be just another bfq_group, no need to special case it in various places, no games with bfqg->my_entity, etc. Paolo, do you see any problem with that? Honza
在 2022/04/13 19:12, Jan Kara 写道: > On Sat 05-03-22 17:11:54, Yu Kuai wrote: >> Currently, bfq can't handle sync io concurrently as long as they >> are not issued from root group. This is because >> 'bfqd->num_groups_with_pending_reqs > 0' is always true in >> bfq_asymmetric_scenario(). >> >> This patchset tries to support concurrent sync io if all the sync ios >> are issued from the same cgroup: >> >> 1) Count root_group into 'num_groups_with_pending_reqs', patch 1-5; > > Seeing the complications and special casing for root_group I wonder: Won't > we be better off to create fake bfq_sched_data in bfq_data and point > root_group->sched_data there? AFAICS it would simplify the code Hi, That sounds an good idel, in this case we only need to make sure the fake service tree will always be empty, which means we only need to special casing bfq_active/idle_insert to the fake service tree. Thanks, Kuai > considerably as root_group would be just another bfq_group, no need to > special case it in various places, no games with bfqg->my_entity, etc. > Paolo, do you see any problem with that? > > Honza >
> Il giorno 13 apr 2022, alle ore 13:12, Jan Kara <jack@suse.cz> ha scritto: > > On Sat 05-03-22 17:11:54, Yu Kuai wrote: >> Currently, bfq can't handle sync io concurrently as long as they >> are not issued from root group. This is because >> 'bfqd->num_groups_with_pending_reqs > 0' is always true in >> bfq_asymmetric_scenario(). >> >> This patchset tries to support concurrent sync io if all the sync ios >> are issued from the same cgroup: >> >> 1) Count root_group into 'num_groups_with_pending_reqs', patch 1-5; > > Seeing the complications and special casing for root_group I wonder: Won't > we be better off to create fake bfq_sched_data in bfq_data and point > root_group->sched_data there? AFAICS it would simplify the code > considerably as root_group would be just another bfq_group, no need to > special case it in various places, no games with bfqg->my_entity, etc. > Paolo, do you see any problem with that? > I do see the benefits. My only concern is that then we also need to check/change the places that rely on the assumption that we would change. Thanks, Paolo > Honza > -- > Jan Kara <jack@suse.com> > SUSE Labs, CR
Currently, bfq can't handle sync io concurrently as long as they are not issued from root group. This is because 'bfqd->num_groups_with_pending_reqs > 0' is always true in bfq_asymmetric_scenario(). This patchset tries to support concurrent sync io if all the sync ios are issued from the same cgroup: 1) Count root_group into 'num_groups_with_pending_reqs', patch 1-5; 2) Don't idle if 'num_groups_with_pending_reqs' is 1, patch 6; 3) Don't count the group if the group doesn't have pending requests, while it's child groups may have pending requests, patch 7; This is because, for example: if sync ios are issued from cgroup /root/c1/c2, root, c1 and c2 will all be counted into 'num_groups_with_pending_reqs', which makes it impossible to handle sync ios concurrently. 4) Decrease 'num_groups_with_pending_reqs' when the last queue completes all the requests, while child groups may still have pending requests, patch 8-10; This is because, for example: t1 issue sync io on root group, t2 and t3 issue sync io on the same child group. num_groups_with_pending_reqs is 2 now. After t1 stopped, num_groups_with_pending_reqs is still 2. sync io from t2 and t3 still can't be handled concurrently. fio test script: startdelay is used to avoid queue merging [global] filename=/dev/nvme0n1 allow_mounted_write=0 ioengine=psync direct=1 ioscheduler=bfq offset_increment=10g group_reporting rw=randwrite bs=4k [test1] numjobs=1 [test2] startdelay=1 numjobs=1 [test3] startdelay=2 numjobs=1 [test4] startdelay=3 numjobs=1 [test5] startdelay=4 numjobs=1 [test6] startdelay=5 numjobs=1 [test7] startdelay=6 numjobs=1 [test8] startdelay=7 numjobs=1 test result: running fio on root cgroup v5.17-rc6: 550 Mib/s v5.17-rc6-patched: 550 Mib/s running fio on non-root cgroup v5.17-rc6: 349 Mib/s v5.17-rc6-patched: 550 Mib/s Yu Kuai (11): block, bfq: add new apis to iterate bfq entities block, bfq: apply news apis where root group is not expected block, bfq: cleanup for __bfq_activate_requeue_entity() block, bfq: move the increasement of 'num_groups_with_pending_reqs' to it's caller block, bfq: count root group into 'num_groups_with_pending_reqs' block, bfq: do not idle if only one cgroup is activated block, bfq: only count parent bfqg when bfqq is activated block, bfq: record how many queues have pending requests in bfq_group block, bfq: move forward __bfq_weights_tree_remove() block, bfq: decrease 'num_groups_with_pending_reqs' earlier block, bfq: cleanup bfqq_group() block/bfq-cgroup.c | 13 +++---- block/bfq-iosched.c | 87 +++++++++++++++++++++++---------------------- block/bfq-iosched.h | 41 +++++++++++++-------- block/bfq-wf2q.c | 56 +++++++++++++++-------------- 4 files changed, 106 insertions(+), 91 deletions(-)