| Message ID | 20200225062351.21267-1-dgilbert@interlog.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | scsi_debug: host managed ZBC + doublestore | expand |
Evening Doug! > The major addition is support for host-managed ZBC devices. The bulk > of the work in this area was done by Damien Le Moal. It allows ZBC > devices with a mix of conventional and "sequential write required" > zones to be specified. [...] > The lower numbered patches in this set contain various measures to > improve the speed and usefulness of this driver. It is being used to > test the rewrite of the SCSI generic (sg) driver which is still > underway. These really should be separate series. One for the ZBC stuff (which generally looks OK), one for the backing store enablement, and maybe one for the general improvements that do not have other dependencies. It's much more manageable for reviewers when things come in smaller batches. Once a posted series has been merged, you can rebase your working tree and submit the next batch of 5-10 patches. Thanks!
On 2020-04-13 7:19 p.m., Martin K. Petersen wrote: in response to the cover letter of a patchset sent: 2020-02-25, 1:23 a.m. which is 7 weeks ago! > Evening Doug! > >> The major addition is support for host-managed ZBC devices. The bulk >> of the work in this area was done by Damien Le Moal. It allows ZBC >> devices with a mix of conventional and "sequential write required" >> zones to be specified. > > [...] > >> The lower numbered patches in this set contain various measures to >> improve the speed and usefulness of this driver. It is being used to >> test the rewrite of the SCSI generic (sg) driver which is still >> underway. > > These really should be separate series. One for the ZBC stuff (which > generally looks OK), one for the backing store enablement, and maybe one > for the general improvements that do not have other dependencies. But other things such as the "ZBC stuff" have dependencies on those general improvements in obvious (i.e. seen by git) and subtle ways. > It's much more manageable for reviewers when things come in smaller > batches. Once a posted series has been merged, you can rebase your > working tree and submit the next batch of 5-10 patches. I believe you overvalue the review process, it is mainly window dressing, unevenly applied. Doug Gilbert
The major addition is support for host-managed ZBC devices. The bulk of the work in this area was done by Damien Le Moal. It allows ZBC devices with a mix of conventional and "sequential write required" zones to be specified. These follow the same model as direct access devices in this driver. Namely, each device has its own metadata (including its write pointer(s)) but all (scsi_debug) devices share the same user data store (see doublestore below). Significantly, this simulation passes the test/zbc_tests.sh script in the https://github.com/hgst/libzbc repository. The lower numbered patches in this set contain various measures to improve the speed and usefulness of this driver. It is being used to test the rewrite of the SCSI generic (sg) driver which is still underway. Disk to disk copies are the test of choice by the author [DG]. Comparing the results between (simulated) disks is useless since all scsi_debug devices share the same user data store. This limitation was circumvented by adding the doublestore parameter. When set, doublestore doubles the data store and allocates them in an alternating pattern to each scsi_debug device. To enhance the comparisons, simulations of VERIFY(10 and 16) commands have been added. A further enhancement is to simulate the PRE-FETCH command (which does nothing as the data is already cached). doublestore can also help with ZBC testing. Single threaded copy commands like dd (and ddpt) can be used to copy one non-empty zone into another empty zone. Multiple-threaded copies (e.g. sgp_dd) can do out-of-order WRITEs which become a WRITE VIOLATION error in a "sequential write required" zone. The author [DG] found that precise command duration timing gave a false impression of how "bulletproof" the sg driver state machines and locking were. The first patch involves randomizing the command durations and it did expose various issues in the driver under test (sg). Since all scsi_debug memory store accesses are done in the context of queuecommand() call, the *_irqsave() and *_irqrestore() variants of the associated locks have been removed. That could be a problem if queuecommand() can ever be called form an interrupt or related context. Finally to address the discrepancy between command duration times seen by the sg driver compared to what was set with this driver's ndelay option, this driver's timekeeping for short durations was made more accurate. If and when this patchset is accepted, this page will be updated: http://sg.danny.cz/sg/sdebug26.html This patchset is against Martin Petersen's git repository and its 5.7/scsi-queue branch. Changes since v3: - make enumeration constants of sdebug_z_cond upper case - move stray alphabetical re-order into correct patch - add some reviewed-by lines - meld 'zbc module parameter' and 'zbc parameter can be string' into single 'add zbc parameter' patch - make zbc= parameter read-only Changes since v2 (RFC): - add support for host-managed zbc devices with conventional and "sequential write required" zones [DLM] Changes since v1 (RFC): - testing with version 1 caused several strange crashes that turned out to be caused by a code trick to read in the data-out buffer but _not_ place it in the big fake_storep array. This approach failed badly when multiple threads were doing verifies at the same time. - replace the code trick with a new do_dout_fetch() function - since the code trick was borrowed from the COMPARE AND WRITE implementation [resp_comp_write()] using do_dout_fetch() fixes the same bug in the existing driver which hasn't been reported (yet). Damien Le Moal (3): scsi_debug: zone_max_open module parameter scsi_debug: zone_nr_conv module parameter scsi_debug: zone_size_mb module parameter Douglas Gilbert (11): scsi_debug: randomize command completion time scsi_debug: add doublestore option scsi_debug: implement verify(10), add verify(16) scsi_debug: weaken rwlock around ramdisk access scsi_debug: improve command duration calculation scsi_debug: implement pre-fetch commands scsi_debug: expand zbc support scsi_debug: add zone commands scsi_debug: add zbc parameter scsi_debug: re-arrange parameters alphabetically scsi_debug: bump to version 1.89 drivers/scsi/scsi_debug.c | 1555 ++++++++++++++++++++++++++++++++----- 1 file changed, 1359 insertions(+), 196 deletions(-)