[libnbd,v2,00/23] libnbd 64-bit NBD extensions

Message

Eric Blake Nov. 14, 2022, 10:51 p.m. UTC

This series is posted alongside a spec change to NBD, and
interoperable with changes posted to qemu-nbd/qemu-storage-daemon.
The RFC patch at the end is optional; ineroperability with qemu works
only when either both projects omit the RFC patch, or when both
projects include it (if only one of the two RFC projects include it,
the protocol is incompatible between the two, but at least client and
server gracefully detect the bug rather than SEGV'ing).

Eric Blake (23):
  block_status: Refactor array storage
  internal: Refactor layout of replies in sbuf
  protocol: Add definitions for extended headers
  states: Prepare to send 64-bit requests
  states: Prepare to receive 64-bit replies
  states: Break deadlock if server goofs on extended replies
  generator: Add struct nbd_extent in prep for 64-bit extents
  block_status: Track 64-bit extents internally
  block_status: Accept 64-bit extents during block status
  api: Add [aio_]nbd_block_status_64
  api: Add several functions for controlling extended headers
  copy: Update nbdcopy to use 64-bit block status
  dump: Update nbddump to use 64-bit block status
  info: Expose extended-headers support through nbdinfo
  info: Update nbdinfo --map to use 64-bit block status
  examples: Update copy-libev to use 64-bit block status
  ocaml: Add example for 64-bit extents
  generator: Actually request extended headers
  api: Add nbd_[aio_]opt_extended_headers()
  interop: Add test of 64-bit block status
  api: Add nbd_can_block_status_payload()
  api: Add nbd_[aio_]block_status_filter()
  RFC: pread: Accept 64-bit holes

 docs/libnbd.pod                               |  18 +-
 info/nbdinfo.pod                              |  21 +-
 sh/nbdsh.pod                                  |   2 +-
 lib/internal.h                                |  42 +-
 lib/nbd-protocol.h                            | 120 ++--
 generator/API.ml                              | 532 +++++++++++++++---
 generator/API.mli                             |   1 +
 generator/C.ml                                |  24 +-
 generator/GoLang.ml                           |  24 +
 generator/Makefile.am                         |   3 +-
 generator/OCaml.ml                            |  18 +-
 generator/Python.ml                           |  20 +-
 generator/state_machine.ml                    |  50 +-
 generator/states-issue-command.c              |  33 +-
 .../states-newstyle-opt-extended-headers.c    | 110 ++++
 generator/states-newstyle-opt-starttls.c      |   7 +-
 .../states-newstyle-opt-structured-reply.c    |   3 +-
 generator/states-newstyle.c                   |   3 +
 generator/states-reply-simple.c               |   4 +-
 generator/states-reply-structured.c           | 279 ++++++---
 generator/states-reply.c                      |  57 +-
 lib/aio.c                                     |   7 +-
 lib/flags.c                                   |  11 +
 lib/handle.c                                  |  25 +-
 lib/opt.c                                     |  44 ++
 lib/rw.c                                      | 250 +++++++-
 python/t/110-defaults.py                      |   1 +
 python/t/120-set-non-defaults.py              |   2 +
 python/t/465-block-status-64.py               |  56 ++
 ocaml/examples/Makefile.am                    |   3 +-
 ocaml/examples/extents64.ml                   |  42 ++
 ocaml/helpers.c                               |  20 +
 ocaml/nbd-c.h                                 |   3 +-
 ocaml/tests/Makefile.am                       |   1 +
 ocaml/tests/test_110_defaults.ml              |   2 +
 ocaml/tests/test_120_set_non_defaults.ml      |   3 +
 ocaml/tests/test_465_block_status_64.ml       |  58 ++
 tests/Makefile.am                             |   4 +
 tests/meta-base-allocation.c                  | 111 +++-
 tests/pwrite-extended.c                       | 112 ++++
 examples/copy-libev.c                         |  21 +-
 examples/server-flags.c                       |   7 +-
 interop/Makefile.am                           |  18 +
 interop/block-status-payload.c                | 241 ++++++++
 interop/block-status-payload.sh               |  80 +++
 interop/large-status.c                        | 186 ++++++
 interop/large-status.sh                       |  49 ++
 interop/opt-extended-headers.c                | 153 +++++
 interop/opt-extended-headers.sh               |  29 +
 .gitignore                                    |   4 +
 copy/nbd-ops.c                                |  22 +-
 dump/dump.c                                   |  27 +-
 fuzzing/libnbd-fuzz-wrapper.c                 |  22 +-
 golang/Makefile.am                            |   1 +
 golang/handle.go                              |   8 +-
 golang/libnbd_110_defaults_test.go            |   8 +
 golang/libnbd_120_set_non_defaults_test.go    |  12 +
 golang/libnbd_465_block_status_64_test.go     | 119 ++++
 info/can.c                                    |  14 +
 info/info-can.sh                              |  30 +
 info/info-packets.sh                          |  17 +-
 info/main.c                                   |   7 +-
 info/map.c                                    |  67 +--
 info/show.c                                   |   9 +-
 64 files changed, 2920 insertions(+), 357 deletions(-)
 create mode 100644 generator/states-newstyle-opt-extended-headers.c
 create mode 100644 python/t/465-block-status-64.py
 create mode 100644 ocaml/examples/extents64.ml
 create mode 100644 ocaml/tests/test_465_block_status_64.ml
 create mode 100644 tests/pwrite-extended.c
 create mode 100644 interop/block-status-payload.c
 create mode 100755 interop/block-status-payload.sh
 create mode 100644 interop/large-status.c
 create mode 100755 interop/large-status.sh
 create mode 100644 interop/opt-extended-headers.c
 create mode 100755 interop/opt-extended-headers.sh
 create mode 100644 golang/libnbd_465_block_status_64_test.go