| Message ID | 20221121162132.00007540@huawei.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Null dereference in bdrv_unregister_buf() probably memory-backend-file related? | expand |
On Mon, 21 Nov 2022 at 11:22, Jonathan Cameron via <qemu-devel@nongnu.org> wrote: > > First CC list is a guess as I haven't managed to root cause where things are > going wrong yet. > > Originally hit this whilst rebasing some CXL patches on v7.2.0-rc1. > CXL makes extensive use of memory-backends and most my tests happen > to use memory-backend-file > > Issue seen on arm64 and x86 though helpfully on x86 the crash appears in an entirely > unrelated location (though the 'fix' works). > > Fairly minimal test command line. > > qemu-system-aarch64 \ > -M virt \ > -drive if=none,file=full.qcow2,format=qcow2,id=hd \ > -device virtio-blk,drive=hd \ > -object memory-backend-file,id=cxl-mem1,mem-path=/tmp/cxltest.raw,size=256M,align=256M \ > > Powerdown the machine or ctrl-c during boot gives a segfault. > On arm64 it was in a stable location that made at least some sense in that > bs in the below snippet is NULL. > > I added the follow work around and the segfault goes away... > > [PATCH] temp > > --- > block/io.c | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/block/io.c b/block/io.c > index b9424024f9..750e1366aa 100644 > --- a/block/io.c > +++ b/block/io.c > @@ -3324,6 +3324,9 @@ void bdrv_unregister_buf(BlockDriverState *bs, void *host, size_t size) > { > BdrvChild *child; > > + if (!bs) { > + return; > + } > GLOBAL_STATE_CODE(); > if (bs->drv && bs->drv->bdrv_unregister_buf) { > bs->drv->bdrv_unregister_buf(bs, host, size); bdrv_*() APIs generally don't accept NULL bs arguments. I think blk_unregister_buf() needs to handle the blk_bs() NULL return value. Can you confirm that the parent function is blk_unregister_buf()? This bug may have been introduced by commit baf422684d73 ("virtio-blk: use BDRV_REQ_REGISTERED_BUF optimization hint"). Stefan
On Mon, 21 Nov 2022 11:47:48 -0500 Stefan Hajnoczi <stefanha@gmail.com> wrote: > On Mon, 21 Nov 2022 at 11:22, Jonathan Cameron via > <qemu-devel@nongnu.org> wrote: > > > > First CC list is a guess as I haven't managed to root cause where things are > > going wrong yet. > > > > Originally hit this whilst rebasing some CXL patches on v7.2.0-rc1. > > CXL makes extensive use of memory-backends and most my tests happen > > to use memory-backend-file > > > > Issue seen on arm64 and x86 though helpfully on x86 the crash appears in an entirely > > unrelated location (though the 'fix' works). > > > > Fairly minimal test command line. > > > > qemu-system-aarch64 \ > > -M virt \ > > -drive if=none,file=full.qcow2,format=qcow2,id=hd \ > > -device virtio-blk,drive=hd \ > > -object memory-backend-file,id=cxl-mem1,mem-path=/tmp/cxltest.raw,size=256M,align=256M \ > > > > Powerdown the machine or ctrl-c during boot gives a segfault. > > On arm64 it was in a stable location that made at least some sense in that > > bs in the below snippet is NULL. > > > > I added the follow work around and the segfault goes away... > > > > [PATCH] temp > > > > --- > > block/io.c | 3 +++ > > 1 file changed, 3 insertions(+) > > > > diff --git a/block/io.c b/block/io.c > > index b9424024f9..750e1366aa 100644 > > --- a/block/io.c > > +++ b/block/io.c > > @@ -3324,6 +3324,9 @@ void bdrv_unregister_buf(BlockDriverState *bs, void *host, size_t size) > > { > > BdrvChild *child; > > > > + if (!bs) { > > + return; > > + } > > GLOBAL_STATE_CODE(); > > if (bs->drv && bs->drv->bdrv_unregister_buf) { > > bs->drv->bdrv_unregister_buf(bs, host, size); > > bdrv_*() APIs generally don't accept NULL bs arguments. > > I think blk_unregister_buf() needs to handle the blk_bs() NULL return > value. Can you confirm that the parent function is > blk_unregister_buf()? > > This bug may have been introduced by commit baf422684d73 ("virtio-blk: > use BDRV_REQ_REGISTERED_BUF optimization hint"). Got it in one. I just bisected to exactly that patch + using the below change indeed works just as well as the above. Now I'd send this as a patch, but I don't yet sufficiently understand what that change you referenced did to break things Seems it registered a notifier that is getting called for all ram blocks, not just the one virtio-blk ones? Perhaps better if you send a fix with an explanation :) Thanks for the quick response and correct identification of the problem. Jonathan diff --git a/block/block-backend.c b/block/block-backend.c index b48c91f4e1..e281569137 100644 --- a/block/block-backend.c +++ b/block/block-backend.c @@ -2582,8 +2582,13 @@ bool blk_register_buf(BlockBackend *blk, void *host, size_t size, Error **errp) void blk_unregister_buf(BlockBackend *blk, void *host, size_t size) { + BlockDriverState *bs; GLOBAL_STATE_CODE(); - bdrv_unregister_buf(blk_bs(blk), host, size); + + bs = blk_bs(blk); + if (bs) { + bdrv_unregister_buf(blk_bs(blk), host, size); + } } int coroutine_fn blk_co_copy_range(BlockBackend *blk_in, int64_t off_in, > > Stefan
On Mon, 21 Nov 2022 at 12:29, Jonathan Cameron <Jonathan.Cameron@huawei.com> wrote: > > On Mon, 21 Nov 2022 11:47:48 -0500 > Stefan Hajnoczi <stefanha@gmail.com> wrote: > > > On Mon, 21 Nov 2022 at 11:22, Jonathan Cameron via > > <qemu-devel@nongnu.org> wrote: > > > > > > First CC list is a guess as I haven't managed to root cause where things are > > > going wrong yet. > > > > > > Originally hit this whilst rebasing some CXL patches on v7.2.0-rc1. > > > CXL makes extensive use of memory-backends and most my tests happen > > > to use memory-backend-file > > > > > > Issue seen on arm64 and x86 though helpfully on x86 the crash appears in an entirely > > > unrelated location (though the 'fix' works). > > > > > > Fairly minimal test command line. > > > > > > qemu-system-aarch64 \ > > > -M virt \ > > > -drive if=none,file=full.qcow2,format=qcow2,id=hd \ > > > -device virtio-blk,drive=hd \ > > > -object memory-backend-file,id=cxl-mem1,mem-path=/tmp/cxltest.raw,size=256M,align=256M \ > > > > > > Powerdown the machine or ctrl-c during boot gives a segfault. > > > On arm64 it was in a stable location that made at least some sense in that > > > bs in the below snippet is NULL. > > > > > > I added the follow work around and the segfault goes away... > > > > > > [PATCH] temp > > > > > > --- > > > block/io.c | 3 +++ > > > 1 file changed, 3 insertions(+) > > > > > > diff --git a/block/io.c b/block/io.c > > > index b9424024f9..750e1366aa 100644 > > > --- a/block/io.c > > > +++ b/block/io.c > > > @@ -3324,6 +3324,9 @@ void bdrv_unregister_buf(BlockDriverState *bs, void *host, size_t size) > > > { > > > BdrvChild *child; > > > > > > + if (!bs) { > > > + return; > > > + } > > > GLOBAL_STATE_CODE(); > > > if (bs->drv && bs->drv->bdrv_unregister_buf) { > > > bs->drv->bdrv_unregister_buf(bs, host, size); > > > > bdrv_*() APIs generally don't accept NULL bs arguments. > > > > I think blk_unregister_buf() needs to handle the blk_bs() NULL return > > value. Can you confirm that the parent function is > > blk_unregister_buf()? > > > > This bug may have been introduced by commit baf422684d73 ("virtio-blk: > > use BDRV_REQ_REGISTERED_BUF optimization hint"). > Got it in one. I just bisected to exactly that patch > > + using the below change indeed works just as well as the above. > Now I'd send this as a patch, but I don't yet sufficiently understand what that change you > referenced did to break things Seems it registered a notifier that is getting > called for all ram blocks, not just the one virtio-blk ones? > > Perhaps better if you send a fix with an explanation :) Sure, no problem. I have reproduced the bug and will send a patch. Thanks, Stefan
diff --git a/block/io.c b/block/io.c index b9424024f9..750e1366aa 100644 --- a/block/io.c +++ b/block/io.c @@ -3324,6 +3324,9 @@ void bdrv_unregister_buf(BlockDriverState *bs, void *host, size_t size) { BdrvChild *child; + if (!bs) { + return; + } GLOBAL_STATE_CODE(); if (bs->drv && bs->drv->bdrv_unregister_buf) { bs->drv->bdrv_unregister_buf(bs, host, size);
First CC list is a guess as I haven't managed to root cause where things are going wrong yet. Originally hit this whilst rebasing some CXL patches on v7.2.0-rc1. CXL makes extensive use of memory-backends and most my tests happen to use memory-backend-file Issue seen on arm64 and x86 though helpfully on x86 the crash appears in an entirely unrelated location (though the 'fix' works). Fairly minimal test command line. qemu-system-aarch64 \ -M virt \ -drive if=none,file=full.qcow2,format=qcow2,id=hd \ -device virtio-blk,drive=hd \ -object memory-backend-file,id=cxl-mem1,mem-path=/tmp/cxltest.raw,size=256M,align=256M \ Powerdown the machine or ctrl-c during boot gives a segfault. On arm64 it was in a stable location that made at least some sense in that bs in the below snippet is NULL. I added the follow work around and the segfault goes away... [PATCH] temp --- block/io.c | 3 +++ 1 file changed, 3 insertions(+) -- 2.37.2 Not present on v7.1.0 . I'll start a bisection shortly but may take a while.