| Message ID | 20220329094048.2107094-1-yukuai3@huawei.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | improve large random io for HDD | expand |
On 3/29/22 3:40 AM, Yu Kuai wrote: > There is a defect for blk-mq compare to blk-sq, specifically split io > will end up discontinuous if the device is under high io pressure, while > split io will still be continuous in sq, this is because: > > 1) split bio is issued one by one, if one bio can't get tag, it will go > to wail. - patch 2 > 2) each time 8(or wake batch) requests is done, 8 waiters will be woken up. > Thus if a thread is woken up, it will unlikey to get multiple tags. > - patch 3,4 > 3) new io can preempt tag even if there are lots of threads waiting for > tags. - patch 5 > > Test environment: > x86 vm, nr_requests is set to 64, queue_depth is set to 32 and > max_sectors_kb is set to 128. > > I haven't tested this patchset on physical machine yet, I'll try later > if anyone thinks this approch is meaningful. A real machine test would definitely be a requirement. What real world uses cases is this solving? These days most devices have plenty of tags, and I would not really expect tag starvation to be much of a concern. However, I do think there's merrit in fixing the unfairness we have here. But not at the cost of all of this. Why not just simply enforce more strict ordering of tag allocations? If someone is waiting, you get to wait too. And I don't see much utility at all in tracking how many splits (and hence tags) would be required. Is this really a common issue, tons of splits and needing many tags? Why not just enforce the strict ordering as mentioned above, not allowing new allocators to get a tag if others are waiting, but perhaps allow someone submitting a string of splits to indeed keep allocating. Yes, it'll be less efficient to still wake one-by-one, but honestly do we really care about that? If you're stalled on waiting for other IO to finish and release a tag, that isn't very efficient to begin with and doesn't seem like a case worth optimizing for me.
在 2022/03/29 20:53, Jens Axboe 写道: > On 3/29/22 3:40 AM, Yu Kuai wrote: >> There is a defect for blk-mq compare to blk-sq, specifically split io >> will end up discontinuous if the device is under high io pressure, while >> split io will still be continuous in sq, this is because: >> >> 1) split bio is issued one by one, if one bio can't get tag, it will go >> to wail. - patch 2 >> 2) each time 8(or wake batch) requests is done, 8 waiters will be woken up. >> Thus if a thread is woken up, it will unlikey to get multiple tags. >> - patch 3,4 >> 3) new io can preempt tag even if there are lots of threads waiting for >> tags. - patch 5 >> >> Test environment: >> x86 vm, nr_requests is set to 64, queue_depth is set to 32 and >> max_sectors_kb is set to 128. >> >> I haven't tested this patchset on physical machine yet, I'll try later >> if anyone thinks this approch is meaningful. > > A real machine test would definitely be a requirement. What real world > uses cases is this solving? These days most devices have plenty of tags, > and I would not really expect tag starvation to be much of a concern. > > However, I do think there's merrit in fixing the unfairness we have > here. But not at the cost of all of this. Why not just simply enforce > more strict ordering of tag allocations? If someone is waiting, you get > to wait too. > > And I don't see much utility at all in tracking how many splits (and > hence tags) would be required. Is this really a common issue, tons of > splits and needing many tags? Why not just enforce the strict ordering > as mentioned above, not allowing new allocators to get a tag if others > are waiting, but perhaps allow someone submitting a string of splits to > indeed keep allocating. > > Yes, it'll be less efficient to still wake one-by-one, but honestly do > we really care about that? If you're stalled on waiting for other IO to > finish and release a tag, that isn't very efficient to begin with and > doesn't seem like a case worth optimizing for me. > Hi, Thanks for your adivce, I'll do more work based on your suggestions. Kuai