| Message ID | alpine.LRH.2.02.2011191517360.10231@file01.intranet.prod.int.rdu2.redhat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | blk-settings: make sure that max_sectors is aligned on "logical_block_size" boundary. | expand |
Greetings; Might I suggest using SECTOR_SIZE instead of 512? Or, perhaps, >> SECTOR_SHIFT instead of / 512. I don't understand the three conditionals. I believe max_sectors is supposed to be <= min(max_dev_sectors, max_hw_sectors), but I don't understand why max_sectors being small should adjust max_hw_sectors and max_dev_sectors. Are the conditions perhaps supposed to be different, adjusting each max_*sectors up to at least PAGE_SIZE / SECTOR_SIZE? Perhaps, like e.g. blk_queue_max_hw_sectors(), the conditionals should log if they are adjusting max_*sectors up to the minimum. Thanks! John Dorminy On Thu, Nov 19, 2020 at 3:37 PM Mikulas Patocka <mpatocka@redhat.com> wrote: > > We get these I/O errors when we run md-raid1 on the top of dm-integrity on > the top of ramdisk: > device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff > device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff > device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1 > device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1 > device-mapper: integrity: Bio not aligned on 8 sectors: 0x8048, 0xff > device-mapper: integrity: Bio not aligned on 8 sectors: 0x8147, 0xff > device-mapper: integrity: Bio not aligned on 8 sectors: 0x8246, 0xff > device-mapper: integrity: Bio not aligned on 8 sectors: 0x8345, 0xbb > > The ramdisk device has logical_block_size 512 and max_sectors 255. The > dm-integrity device uses logical_block_size 4096 and it doesn't affect the > "max_sectors" value - thus, it inherits 255 from the ramdisk. So, we have > a device with max_sectors not aligned on logical_block_size. > > The md-raid device sees that the underlying leg has max_sectors 255 and it > will split the bios on 255-sector boundary, making the bios unaligned on > logical_block_size. > > In order to fix the bug, we round down max_sectors to logical_block_size. > > Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> > Cc: stable@vger.kernel.org > > --- > block/blk-settings.c | 10 ++++++++++ > 1 file changed, 10 insertions(+) > > Index: linux-2.6/block/blk-settings.c > =================================================================== > --- linux-2.6.orig/block/blk-settings.c 2020-10-29 12:20:46.000000000 +0100 > +++ linux-2.6/block/blk-settings.c 2020-11-19 21:20:18.000000000 +0100 > @@ -591,6 +591,16 @@ int blk_stack_limits(struct queue_limits > ret = -1; > } > > + t->max_sectors = round_down(t->max_sectors, t->logical_block_size / 512); > + if (t->max_sectors < PAGE_SIZE / 512) > + t->max_sectors = PAGE_SIZE / 512; > + t->max_hw_sectors = round_down(t->max_hw_sectors, t->logical_block_size / 512); > + if (t->max_sectors < PAGE_SIZE / 512) > + t->max_hw_sectors = PAGE_SIZE / 512; > + t->max_dev_sectors = round_down(t->max_dev_sectors, t->logical_block_size / 512); > + if (t->max_sectors < PAGE_SIZE / 512) > + t->max_dev_sectors = PAGE_SIZE / 512; > + > /* Discard alignment and granularity */ > if (b->discard_granularity) { > alignment = queue_limit_discard_alignment(b, start); > > -- > dm-devel mailing list > dm-devel@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel >
On Thu, 19 Nov 2020, John Dorminy wrote: > Greetings; > > Might I suggest using SECTOR_SIZE instead of 512? Or, perhaps, >> > SECTOR_SHIFT instead of / 512. Yes, that's a good point. > I don't understand the three conditionals. I believe max_sectors is > supposed to be <= min(max_dev_sectors, max_hw_sectors), but I don't > understand why max_sectors being small should adjust max_hw_sectors > and max_dev_sectors. Are the conditions perhaps supposed to be > different, adjusting each max_*sectors up to at least PAGE_SIZE / > SECTOR_SIZE? I copied this pattern from blk_queue_max_hw_sectors. Perhaps, we could use: t->max_sectors = round_down(t->max_sectors, t->logical_block_size / 512); if (!t->max_sectors) t->max_sectors = t->logical_block_size / 512; instead. Jens, what do you think is better? Mikulas > Perhaps, like e.g. blk_queue_max_hw_sectors(), the > conditionals should log if they are adjusting max_*sectors up to the > minimum. > > Thanks! > > John Dorminy > > On Thu, Nov 19, 2020 at 3:37 PM Mikulas Patocka <mpatocka@redhat.com> wrote: > > > > We get these I/O errors when we run md-raid1 on the top of dm-integrity on > > the top of ramdisk: > > device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff > > device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff > > device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1 > > device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1 > > device-mapper: integrity: Bio not aligned on 8 sectors: 0x8048, 0xff > > device-mapper: integrity: Bio not aligned on 8 sectors: 0x8147, 0xff > > device-mapper: integrity: Bio not aligned on 8 sectors: 0x8246, 0xff > > device-mapper: integrity: Bio not aligned on 8 sectors: 0x8345, 0xbb > > > > The ramdisk device has logical_block_size 512 and max_sectors 255. The > > dm-integrity device uses logical_block_size 4096 and it doesn't affect the > > "max_sectors" value - thus, it inherits 255 from the ramdisk. So, we have > > a device with max_sectors not aligned on logical_block_size. > > > > The md-raid device sees that the underlying leg has max_sectors 255 and it > > will split the bios on 255-sector boundary, making the bios unaligned on > > logical_block_size. > > > > In order to fix the bug, we round down max_sectors to logical_block_size. > > > > Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> > > Cc: stable@vger.kernel.org > > > > --- > > block/blk-settings.c | 10 ++++++++++ > > 1 file changed, 10 insertions(+) > > > > Index: linux-2.6/block/blk-settings.c > > =================================================================== > > --- linux-2.6.orig/block/blk-settings.c 2020-10-29 12:20:46.000000000 +0100 > > +++ linux-2.6/block/blk-settings.c 2020-11-19 21:20:18.000000000 +0100 > > @@ -591,6 +591,16 @@ int blk_stack_limits(struct queue_limits > > ret = -1; > > } > > > > + t->max_sectors = round_down(t->max_sectors, t->logical_block_size / 512); > > + if (t->max_sectors < PAGE_SIZE / 512) > > + t->max_sectors = PAGE_SIZE / 512; > > + t->max_hw_sectors = round_down(t->max_hw_sectors, t->logical_block_size / 512); > > + if (t->max_sectors < PAGE_SIZE / 512) > > + t->max_hw_sectors = PAGE_SIZE / 512; > > + t->max_dev_sectors = round_down(t->max_dev_sectors, t->logical_block_size / 512); > > + if (t->max_sectors < PAGE_SIZE / 512) > > + t->max_dev_sectors = PAGE_SIZE / 512; > > + > > /* Discard alignment and granularity */ > > if (b->discard_granularity) { > > alignment = queue_limit_discard_alignment(b, start); > > > > -- > > dm-devel mailing list > > dm-devel@redhat.com > > https://www.redhat.com/mailman/listinfo/dm-devel > > >
Index: linux-2.6/block/blk-settings.c =================================================================== --- linux-2.6.orig/block/blk-settings.c 2020-10-29 12:20:46.000000000 +0100 +++ linux-2.6/block/blk-settings.c 2020-11-19 21:20:18.000000000 +0100 @@ -591,6 +591,16 @@ int blk_stack_limits(struct queue_limits ret = -1; } + t->max_sectors = round_down(t->max_sectors, t->logical_block_size / 512); + if (t->max_sectors < PAGE_SIZE / 512) + t->max_sectors = PAGE_SIZE / 512; + t->max_hw_sectors = round_down(t->max_hw_sectors, t->logical_block_size / 512); + if (t->max_sectors < PAGE_SIZE / 512) + t->max_hw_sectors = PAGE_SIZE / 512; + t->max_dev_sectors = round_down(t->max_dev_sectors, t->logical_block_size / 512); + if (t->max_sectors < PAGE_SIZE / 512) + t->max_dev_sectors = PAGE_SIZE / 512; + /* Discard alignment and granularity */ if (b->discard_granularity) { alignment = queue_limit_discard_alignment(b, start);
We get these I/O errors when we run md-raid1 on the top of dm-integrity on the top of ramdisk: device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff device-mapper: integrity: Bio not aligned on 8 sectors: 0xff00, 0xff device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1 device-mapper: integrity: Bio not aligned on 8 sectors: 0xffff, 0x1 device-mapper: integrity: Bio not aligned on 8 sectors: 0x8048, 0xff device-mapper: integrity: Bio not aligned on 8 sectors: 0x8147, 0xff device-mapper: integrity: Bio not aligned on 8 sectors: 0x8246, 0xff device-mapper: integrity: Bio not aligned on 8 sectors: 0x8345, 0xbb The ramdisk device has logical_block_size 512 and max_sectors 255. The dm-integrity device uses logical_block_size 4096 and it doesn't affect the "max_sectors" value - thus, it inherits 255 from the ramdisk. So, we have a device with max_sectors not aligned on logical_block_size. The md-raid device sees that the underlying leg has max_sectors 255 and it will split the bios on 255-sector boundary, making the bios unaligned on logical_block_size. In order to fix the bug, we round down max_sectors to logical_block_size. Signed-off-by: Mikulas Patocka <mpatocka@redhat.com> Cc: stable@vger.kernel.org --- block/blk-settings.c | 10 ++++++++++ 1 file changed, 10 insertions(+)