[v5] block/vxhs.c Add support for a new block device type called "vxhs"

Message ID	1477982098-15462-1-git-send-email-ashish.mittal@veritas.com (mailing list archive)
State	New, archived
Headers	show Return-Path: <qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org> From: Ashish Mittal <ashmit602@gmail.com> To: qemu-devel@nongnu.org, pbonzini@redhat.com, kwolf@redhat.com, armbru@redhat.com, berrange@redhat.com, jcody@redhat.com, famz@redhat.com, ashish.mittal@veritas.com, stefanha@gmail.com, Rakesh.Ranjan@veritas.com, Buddhi.Madhav@veritas.com Date: Mon, 31 Oct 2016 23:34:58 -0700 Message-Id: <1477982098-15462-1-git-send-email-ashish.mittal@veritas.com> Subject: [Qemu-devel] [PATCH v5] block/vxhs.c Add support for a new block device type called "vxhs" Precedence: list Cc: Ketan.Nilangekar@veritas.com, Abhijit.Dey@veritas.com, Venkatesha.Mg@veritas.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" <qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org>

Done! All builds are fine, but I have no way of knowing if that fixed the issue! Please let me know if you still see a problem. [root@rhel72ga-build-upstream qemu] 2016-11-01 09:46:06$ make CC block/vxhs.o LINK qemu-nbd LINK qemu-img LINK qemu-io LINK aarch64-softmmu/qemu-system-aarch64 LINK alpha-softmmu/qemu-system-alpha LINK arm-softmmu/qemu-system-arm LINK cris-softmmu/qemu-system-cris LINK i386-softmmu/qemu-system-i386 LINK lm32-softmmu/qemu-system-lm32 LINK m68k-softmmu/qemu-system-m68k LINK microblazeel-softmmu/qemu-system-microblazeel LINK microblaze-softmmu/qemu-system-microblaze LINK mips64el-softmmu/qemu-system-mips64el LINK mips64-softmmu/qemu-system-mips64 LINK mipsel-softmmu/qemu-system-mipsel LINK mips-softmmu/qemu-system-mips LINK moxie-softmmu/qemu-system-moxie LINK or32-softmmu/qemu-system-or32 LINK ppc64-softmmu/qemu-system-ppc64 LINK ppcemb-softmmu/qemu-system-ppcemb LINK ppc-softmmu/qemu-system-ppc LINK s390x-softmmu/qemu-system-s390x LINK sh4eb-softmmu/qemu-system-sh4eb LINK sh4-softmmu/qemu-system-sh4 LINK sparc64-softmmu/qemu-system-sparc64 LINK sparc-softmmu/qemu-system-sparc LINK tricore-softmmu/qemu-system-tricore LINK unicore32-softmmu/qemu-system-unicore32 LINK x86_64-softmmu/qemu-system-x86_64 LINK xtensaeb-softmmu/qemu-system-xtensaeb LINK xtensa-softmmu/qemu-system-xtensa [root@rhel72ga-build-upstream qemu] 2016-11-01 09:48:13$ Thanks! On Tue, Nov 1, 2016 at 9:27 AM, Daniel P. Berrange <berrange@redhat.com> wrote: > On Tue, Nov 01, 2016 at 09:09:31AM -0700, ashish mittal wrote: >> Hi Daniel, >> >> Thanks for pointing that out. I had done a fresh configure and make >> before submitting the patch. I am somehow not able to reproduce that >> error! >> >> Checked out fresh source just now and tried again, still no luck. >> >> [root@rhel72ga-build-upstream qemu] 2016-11-01 08:51:15$ git apply >> ~/qemu/31Oct2016/0001-block-vxhs.c-Add-support-for-a-new-block-device-type.patch >> [root@rhel72ga-build-upstream qemu] 2016-11-01 08:51:31$ ./configure >> --enable-vxhs >> Install prefix /usr/local >> BIOS directory /usr/local/share/qemu >> binary directory /usr/local/bin >> library directory /usr/local/lib >> module directory /usr/local/lib/qemu >> libexec directory /usr/local/libexec >> include directory /usr/local/include >> config directory /usr/local/etc >> local state directory /usr/local/var >> Manual directory /usr/local/share/man >> ELF interp prefix /usr/gnemul/qemu-%M >> Source path /root/qemu_second_buildarea/qemu >> C compiler cc >> Host C compiler cc >> C++ compiler c++ >> Objective-C compiler cc >> ARFLAGS rv >> CFLAGS -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -g >> QEMU_CFLAGS -I/usr/include/pixman-1 -Werror >> -DHAS_LIBSSH2_SFTP_FSYNC -pthread -I/usr/include/glib-2.0 >> -I/usr/lib64/glib-2.0/include -fPIE -DPIE -m64 -mcx16 >> -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE >> -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings >> -Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv >> -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs >> -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers >> -Wold-style-declaration -Wold-style-definition -Wtype-limits >> -fstack-protector-strong -I/usr/include/p11-kit-1 >> -I/usr/include/libpng15 -I/usr/include/spice-server >> -I/usr/include/cacard -I/usr/include/glib-2.0 >> -I/usr/lib64/glib-2.0/include -I/usr/include/pixman-1 >> -I/usr/include/nss3 -I/usr/include/nspr4 -I/usr/include/spice-1 >> -I/usr/include/cacard -I/usr/include/nss3 -I/usr/include/nspr4 >> -I/usr/include/libusb-1.0 >> LDFLAGS -Wl,--warn-common -Wl,-z,relro -Wl,-z,now -pie -m64 -g >> make make >> install install >> python python -B >> smbd /usr/sbin/smbd >> module support no >> host CPU x86_64 >> host big endian no >> target list aarch64-softmmu alpha-softmmu arm-softmmu >> cris-softmmu i386-softmmu lm32-softmmu m68k-softmmu >> microblazeel-softmmu microblaze-softmmu mips64el-softmmu >> mips64-softmmu mipsel-softmmu mips-softmmu moxie-softmmu or32-softmmu >> ppc64-softmmu ppcemb-softmmu ppc-softmmu s390x-softmmu sh4eb-softmmu >> sh4-softmmu sparc64-softmmu sparc-softmmu tricore-softmmu >> unicore32-softmmu x86_64-softmmu xtensaeb-softmmu xtensa-softmmu >> aarch64-linux-user alpha-linux-user armeb-linux-user arm-linux-user >> cris-linux-user i386-linux-user m68k-linux-user >> microblazeel-linux-user microblaze-linux-user mips64el-linux-user >> mips64-linux-user mipsel-linux-user mips-linux-user >> mipsn32el-linux-user mipsn32-linux-user or32-linux-user >> ppc64abi32-linux-user ppc64le-linux-user ppc64-linux-user >> ppc-linux-user s390x-linux-user sh4eb-linux-user sh4-linux-user >> sparc32plus-linux-user sparc64-linux-user sparc-linux-user >> tilegx-linux-user x86_64-linux-user >> tcg debug enabled no >> gprof enabled no >> sparse enabled no >> strip binaries yes >> profiler no >> static build no >> pixman system >> SDL support yes (1.2.15) >> GTK support no >> GTK GL support no >> VTE support no >> TLS priority NORMAL >> GNUTLS support yes >> GNUTLS rnd yes >> libgcrypt no >> libgcrypt kdf no >> nettle yes (2.7.1) >> nettle kdf yes >> libtasn1 yes >> curses support yes >> virgl support no >> curl support yes >> mingw32 support no >> Audio drivers oss >> Block whitelist (rw) >> Block whitelist (ro) >> VirtFS support yes >> VNC support yes >> VNC SASL support yes >> VNC JPEG support yes >> VNC PNG support yes >> xen support no >> brlapi support yes >> bluez support yes >> Documentation yes >> PIE yes >> vde support no >> netmap support no >> Linux AIO support yes >> ATTR/XATTR support yes >> Install blobs yes >> KVM support yes >> COLO support yes >> RDMA support yes >> TCG interpreter no >> fdt support yes >> preadv support yes >> fdatasync yes >> madvise yes >> posix_madvise yes >> libcap-ng support yes >> vhost-net support yes >> vhost-scsi support yes >> vhost-vsock support yes >> Trace backends log >> spice support yes (0.12.6/0.12.4) >> rbd support yes >> xfsctl support no >> smartcard support yes >> libusb yes >> usb net redir yes >> OpenGL support no >> OpenGL dmabufs no >> libiscsi support yes >> libnfs support no >> build guest agent yes >> QGA VSS support no >> QGA w32 disk info no >> QGA MSI support no >> seccomp support yes >> coroutine backend ucontext >> coroutine pool yes >> debug stack usage no >> GlusterFS support yes >> Archipelago support no >> gcov gcov >> gcov enabled no >> TPM support yes >> libssh2 support yes >> TPM passthrough yes >> QOM debugging yes >> lzo support yes >> snappy support yes >> bzip2 support no >> NUMA host support yes >> tcmalloc support no >> jemalloc support no >> avx2 optimization yes >> replication support yes >> VxHS block device yes >> [root@rhel72ga-build-upstream qemu] 2016-11-01 08:52:22$ echo $? >> 0 >> >> Am I missing some step here? >> Should I just add #include <sys/uio.h> to qnio_api.h ? >> Doing a build on all platforms right now. > > Yes, sys/uio.h seems to be the recommended header file to get > the struct iovec definition pulled in. > > Regards, > Daniel > -- > |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :| > |: http://libvirt.org -o- http://virt-manager.org :| > |: http://entangle-photo.org -o- http://search.cpan.org/~danberr/ :|

diff --git a/block/Makefile.objs b/block/Makefile.objs index 67a036a..58313a2 100644 --- a/block/Makefile.objs +++ b/block/Makefile.objs @@ -18,6 +18,7 @@ block-obj-$(CONFIG_LIBNFS) += nfs.o block-obj-$(CONFIG_CURL) += curl.o block-obj-$(CONFIG_RBD) += rbd.o block-obj-$(CONFIG_GLUSTERFS) += gluster.o +block-obj-$(CONFIG_VXHS) += vxhs.o block-obj-$(CONFIG_ARCHIPELAGO) += archipelago.o block-obj-$(CONFIG_LIBSSH2) += ssh.o block-obj-y += accounting.o dirty-bitmap.o @@ -38,6 +39,7 @@ rbd.o-cflags := $(RBD_CFLAGS) rbd.o-libs := $(RBD_LIBS) gluster.o-cflags := $(GLUSTERFS_CFLAGS) gluster.o-libs := $(GLUSTERFS_LIBS) +vxhs.o-libs := $(VXHS_LIBS) ssh.o-cflags := $(LIBSSH2_CFLAGS) ssh.o-libs := $(LIBSSH2_LIBS) archipelago.o-libs := $(ARCHIPELAGO_LIBS) diff --git a/block/trace-events b/block/trace-events index aff8a96..5cb8089 100644 --- a/block/trace-events +++ b/block/trace-events @@ -113,3 +113,24 @@ qed_aio_write_data(void *s, void *acb, int ret, uint64_t offset, size_t len) "s qed_aio_write_prefill(void *s, void *acb, uint64_t start, size_t len, uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64 qed_aio_write_postfill(void *s, void *acb, uint64_t start, size_t len, uint64_t offset) "s %p acb %p start %"PRIu64" len %zu offset %"PRIu64 qed_aio_write_main(void *s, void *acb, int ret, uint64_t offset, size_t len) "s %p acb %p ret %d offset %"PRIu64" len %zu" + +# block/vxhs.c +vxhs_iio_callback(int error, int reason) "ctx is NULL: error %d, reason %d" +vxhs_setup_qnio(void *s) "Context to HyperScale IO manager = %p" +vxhs_iio_callback_chnfail(int err, int error) "QNIO channel failed, no i/o %d, %d" +vxhs_iio_callback_unknwn(int opcode, int err) "unexpected opcode %d, errno %d" +vxhs_open_fail(int ret) "Could not open the device. Error = %d" +vxhs_open_epipe(int ret) "Could not create a pipe for device. Bailing out. Error=%d" +vxhs_aio_rw_invalid(int req) "Invalid I/O request iodir %d" +vxhs_aio_rw_ioerr(char *guid, int iodir, uint64_t size, uint64_t off, void *acb, int ret, int err) "IO ERROR (vDisk %s) FOR : Read/Write = %d size = %lu offset = %lu ACB = %p. Error = %d, errno = %d" +vxhs_get_vdisk_stat_err(char *guid, int ret, int err) "vDisk (%s) stat ioctl failed, ret = %d, errno = %d" +vxhs_get_vdisk_stat(char *vdisk_guid, uint64_t vdisk_size) "vDisk %s stat ioctl returned size %lu" +vxhs_qnio_iio_open(const char *ip) "Failed to connect to storage agent on host-ip %s" +vxhs_qnio_iio_devopen(const char *fname) "Failed to open vdisk device: %s" +vxhs_complete_aio(void *acb, uint64_t ret) "aio failed acb %p ret %ld" +vxhs_parse_uri_filename(const char *filename) "URI passed via bdrv_parse_filename %s" +vxhs_qemu_init_vdisk(const char *vdisk_id) "vdisk-id from json %s" +vxhs_parse_uri_hostinfo(int num, char *host, int port) "Host %d: IP %s, Port %d" +vxhs_qemu_init(char *of_vsa_addr, int port) "Adding host %s:%d to BDRVVXHSState" +vxhs_qemu_init_filename(const char *filename) "Filename passed as %s" +vxhs_close(char *vdisk_guid) "Closing vdisk %s" diff --git a/block/vxhs.c b/block/vxhs.c new file mode 100644 index 0000000..8913e8f --- /dev/null +++ b/block/vxhs.c @@ -0,0 +1,689 @@ +/* + * QEMU Block driver for Veritas HyperScale (VxHS) + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + * + */ + +#include "qemu/osdep.h" +#include "block/block_int.h" +#include <qnio/qnio_api.h> +#include "qapi/qmp/qerror.h" +#include "qapi/qmp/qdict.h" +#include "qapi/qmp/qstring.h" +#include "trace.h" +#include "qemu/uri.h" +#include "qapi/error.h" +#include "qemu/error-report.h" + +#define VDISK_FD_READ 0 +#define VDISK_FD_WRITE 1 + +#define VXHS_OPT_FILENAME "filename" +#define VXHS_OPT_VDISK_ID "vdisk-id" +#define VXHS_OPT_SERVER "server" +#define VXHS_OPT_HOST "host" +#define VXHS_OPT_PORT "port" + +typedef struct QNIOLibState { + int refcnt; + void *context; +} QNIOLibState; + +typedef enum { + VDISK_AIO_READ, + VDISK_AIO_WRITE, + VDISK_STAT +} VDISKAIOCmd; + +/* + * HyperScale AIO callbacks structure + */ +typedef struct VXHSAIOCB { + BlockAIOCB common; + int err; + int direction; /* IO direction (r/w) */ + size_t io_offset; + size_t size; + QEMUIOVector *qiov; +} VXHSAIOCB; + +typedef struct VXHSvDiskHostsInfo { + int qnio_cfd; /* Channel FD */ + int vdisk_rfd; /* vDisk remote FD */ + char *hostip; /* Host's IP addresses */ + int port; /* Host's port number */ +} VXHSvDiskHostsInfo; + +/* + * Structure per vDisk maintained for state + */ +typedef struct BDRVVXHSState { + int fds[2]; + int event_reader_pos; + VXHSAIOCB *qnio_event_acb; + VXHSvDiskHostsInfo vdisk_hostinfo; /* Per host info */ + char *vdisk_guid; +} BDRVVXHSState; + +/* QNIO Library State */ +static QNIOLibState qniolib; + +/* vdisk prefix to pass to qnio */ +static const char vdisk_prefix[] = "/dev/of/vdisk"; + +static void vxhs_iio_callback(int32_t rfd, uint32_t reason, void *ctx, + uint32_t error, uint32_t opcode) +{ + VXHSAIOCB *acb = NULL; + BDRVVXHSState *s = NULL; + ssize_t ret; + + switch (opcode) { + case IRP_READ_REQUEST: + case IRP_WRITE_REQUEST: + + /* + * ctx is VXHSAIOCB* + * ctx is NULL if error is QNIOERROR_CHANNEL_HUP or + * reason is IIO_REASON_HUP + */ + if (ctx) { + acb = ctx; + s = acb->common.bs->opaque; + } else { + trace_vxhs_iio_callback(error, reason); + goto out; + } + + if (error) { + if (!acb->err) { + acb->err = error; + } + trace_vxhs_iio_callback(error, reason); + } + + ret = qemu_write_full(s->fds[VDISK_FD_WRITE], &acb, sizeof(acb)); + g_assert(ret == sizeof(acb)); + break; + + default: + if (error == QNIOERROR_CHANNEL_HUP) { + /* + * Channel failed, spontaneous notification, + * not in response to I/O + */ + trace_vxhs_iio_callback_chnfail(error, errno); + } else { + trace_vxhs_iio_callback_unknwn(opcode, error); + } + break; + } +out: + return; +} + +/* + * Initialize QNIO library on first open. + */ +static int vxhs_qnio_open(void) +{ + int ret = 0; + + if (qniolib.refcnt != 0) { + g_assert(qniolib.context != NULL); + qniolib.refcnt++; + return 0; + } + qniolib.context = iio_init(QNIO_VERSION, vxhs_iio_callback); + if (qniolib.context == NULL) { + ret = -ENODEV; + } else { + qniolib.refcnt = 1; + } + return ret; +} + +/* + * Cleanup QNIO library on last close. + */ +static void vxhs_qnio_close(void) +{ + qniolib.refcnt--; + if (qniolib.refcnt == 0) { + iio_fini(qniolib.context); + qniolib.context = NULL; + } +} + +static int vxhs_qnio_iio_open(int *cfd, const char *of_vsa_addr, + int *rfd, const char *file_name) +{ + int ret = 0; + bool qnio_open = false; + + ret = vxhs_qnio_open(); + if (ret) { + return ret; + } + qnio_open = true; + + /* + * Open qnio channel to storage agent if not opened before. + */ + *cfd = iio_open(qniolib.context, of_vsa_addr, 0); + if (*cfd < 0) { + trace_vxhs_qnio_iio_open(of_vsa_addr); + ret = -ENODEV; + goto err_out; + } + + /* + * Open vdisk device + */ + *rfd = iio_devopen(qniolib.context, *cfd, file_name, 0); + if (*rfd < 0) { + trace_vxhs_qnio_iio_devopen(file_name); + ret = -ENODEV; + goto err_out; + } + return 0; + +err_out: + if (*cfd >= 0) { + iio_close(qniolib.context, *cfd); + } + if (qnio_open) { + vxhs_qnio_close(); + } + *cfd = -1; + *rfd = -1; + return ret; +} + +static void vxhs_qnio_iio_close(BDRVVXHSState *s) +{ + /* + * Close vDisk device + */ + if (s->vdisk_hostinfo.vdisk_rfd >= 0) { + iio_devclose(qniolib.context, 0, s->vdisk_hostinfo.vdisk_rfd); + s->vdisk_hostinfo.vdisk_rfd = -1; + } + + /* + * Close QNIO channel against cached channel-fd + */ + if (s->vdisk_hostinfo.qnio_cfd >= 0) { + iio_close(qniolib.context, s->vdisk_hostinfo.qnio_cfd); + s->vdisk_hostinfo.qnio_cfd = -1; + } + + vxhs_qnio_close(); +} + +static void vxhs_complete_aio(VXHSAIOCB *acb, BDRVVXHSState *s) +{ + BlockCompletionFunc *cb = acb->common.cb; + void *opaque = acb->common.opaque; + int ret = 0; + + if (acb->err != 0) { + trace_vxhs_complete_aio(acb, acb->err); + /* + * We mask all the IO errors generically as EIO for upper layers + * Right now our IO Manager uses non standard error codes. Instead + * of confusing upper layers with incorrect interpretation we are + * doing this workaround. + */ + ret = (-EIO); + } + + qemu_aio_unref(acb); + cb(opaque, ret); +} + +/* + * This is the HyperScale event handler registered to QEMU. + * It is invoked when any IO gets completed and written on pipe + * by callback called from QNIO thread context. Then it marks + * the AIO as completed, and releases HyperScale AIO callbacks. + */ +static void vxhs_aio_event_reader(void *opaque) +{ + BDRVVXHSState *s = opaque; + char *p; + ssize_t ret; + + do { + p = (char *)&s->qnio_event_acb; + ret = read(s->fds[VDISK_FD_READ], p + s->event_reader_pos, + sizeof(s->qnio_event_acb) - s->event_reader_pos); + if (ret > 0) { + s->event_reader_pos += ret; + if (s->event_reader_pos == sizeof(s->qnio_event_acb)) { + s->event_reader_pos = 0; + vxhs_complete_aio(s->qnio_event_acb, s); + } + } + } while (ret < 0 && errno == EINTR); +} + +static QemuOptsList runtime_opts = { + .name = "vxhs", + .head = QTAILQ_HEAD_INITIALIZER(runtime_opts.head), + .desc = { + { + .name = VXHS_OPT_FILENAME, + .type = QEMU_OPT_STRING, + .help = "URI to the Veritas HyperScale image", + }, + { + .name = VXHS_OPT_VDISK_ID, + .type = QEMU_OPT_STRING, + .help = "UUID of the VxHS vdisk", + }, + { /* end of list */ } + }, +}; + +static QemuOptsList runtime_tcp_opts = { + .name = "vxhs_tcp", + .head = QTAILQ_HEAD_INITIALIZER(runtime_tcp_opts.head), + .desc = { + { + .name = VXHS_OPT_HOST, + .type = QEMU_OPT_STRING, + .help = "host address (ipv4 addresses)", + }, + { + .name = VXHS_OPT_PORT, + .type = QEMU_OPT_NUMBER, + .help = "port number on which VxHSD is listening (default 9999)", + .def_value_str = "9999" + }, + { /* end of list */ } + }, +}; + +/* + * Parse the incoming URI and populate *options with the host information. + * URI syntax has the limitation of supporting only one host info. + * To pass multiple host information, use the JSON syntax. + */ +static int vxhs_parse_uri(const char *filename, QDict *options) +{ + URI *uri = NULL; + char *hoststr, *portstr; + char *port; + int ret = 0; + + trace_vxhs_parse_uri_filename(filename); + uri = uri_parse(filename); + if (!uri || !uri->server || !uri->path) { + uri_free(uri); + return -EINVAL; + } + + hoststr = g_strdup(VXHS_OPT_SERVER".host"); + qdict_put(options, hoststr, qstring_from_str(uri->server)); + g_free(hoststr); + + portstr = g_strdup(VXHS_OPT_SERVER".port"); + if (uri->port) { + port = g_strdup_printf("%d", uri->port); + qdict_put(options, portstr, qstring_from_str(port)); + g_free(port); + } + g_free(portstr); + + if (strstr(uri->path, "vxhs") == NULL) { + qdict_put(options, "vdisk-id", qstring_from_str(uri->path)); + } + + trace_vxhs_parse_uri_hostinfo(1, uri->server, uri->port); + uri_free(uri); + + return ret; +} + +static void vxhs_parse_filename(const char *filename, QDict *options, + Error **errp) +{ + if (qdict_haskey(options, "vdisk-id") || qdict_haskey(options, "server")) { + error_setg(errp, "vdisk-id/server and a file name may not be specified " + "at the same time"); + return; + } + + if (strstr(filename, "://")) { + int ret = vxhs_parse_uri(filename, options); + if (ret < 0) { + error_setg(errp, "Invalid URI. URI should be of the form " + " vxhs://<host_ip>:<port>/{<vdisk-id>}"); + } + } +} + +static int vxhs_qemu_init(QDict *options, BDRVVXHSState *s, + int *cfd, int *rfd, Error **errp) +{ + QDict *backing_options = NULL; + QemuOpts *opts, *tcp_opts; + const char *vxhs_filename; + char *of_vsa_addr = NULL; + Error *local_err = NULL; + const char *vdisk_id_opt; + const char *server_host_opt; + char *file_name = NULL; + char *str = NULL; + int ret = 0; + + opts = qemu_opts_create(&runtime_opts, NULL, 0, &error_abort); + qemu_opts_absorb_qdict(opts, options, &local_err); + if (local_err) { + ret = -EINVAL; + goto out; + } + + vxhs_filename = qemu_opt_get(opts, VXHS_OPT_FILENAME); + if (vxhs_filename) { + trace_vxhs_qemu_init_filename(vxhs_filename); + } + + vdisk_id_opt = qemu_opt_get(opts, VXHS_OPT_VDISK_ID); + if (!vdisk_id_opt) { + error_setg(&local_err, QERR_MISSING_PARAMETER, VXHS_OPT_VDISK_ID); + ret = -EINVAL; + goto out; + } + s->vdisk_guid = g_strdup(vdisk_id_opt); + trace_vxhs_qemu_init_vdisk(vdisk_id_opt); + + str = g_strdup_printf(VXHS_OPT_SERVER"."); + qdict_extract_subqdict(options, &backing_options, str); + + /* Create opts info from runtime_tcp_opts list */ + tcp_opts = qemu_opts_create(&runtime_tcp_opts, NULL, 0, &error_abort); + qemu_opts_absorb_qdict(tcp_opts, backing_options, &local_err); + if (local_err) { + qdict_del(backing_options, str); + qemu_opts_del(tcp_opts); + ret = -EINVAL; + goto out; + } + + server_host_opt = qemu_opt_get(tcp_opts, VXHS_OPT_HOST); + if (!server_host_opt) { + error_setg(&local_err, QERR_MISSING_PARAMETER, + VXHS_OPT_SERVER"."VXHS_OPT_HOST); + ret = -EINVAL; + goto out; + } + + s->vdisk_hostinfo.hostip = g_strdup(server_host_opt); + + s->vdisk_hostinfo.port = g_ascii_strtoll(qemu_opt_get(tcp_opts, + VXHS_OPT_PORT), + NULL, 0); + + s->vdisk_hostinfo.qnio_cfd = -1; + s->vdisk_hostinfo.vdisk_rfd = -1; + trace_vxhs_qemu_init(s->vdisk_hostinfo.hostip, + s->vdisk_hostinfo.port); + + qdict_del(backing_options, str); + qemu_opts_del(tcp_opts); + + file_name = g_strdup_printf("%s%s", vdisk_prefix, s->vdisk_guid); + of_vsa_addr = g_strdup_printf("of://%s:%d", + s->vdisk_hostinfo.hostip, + s->vdisk_hostinfo.port); + + ret = vxhs_qnio_iio_open(cfd, of_vsa_addr, rfd, file_name); + if (ret) { + error_setg(&local_err, "Failed qnio_iio_open"); + ret = -EIO; + } + +out: + g_free(str); + g_free(file_name); + g_free(of_vsa_addr); + qemu_opts_del(opts); + + if (ret < 0) { + error_propagate(errp, local_err); + g_free(s->vdisk_hostinfo.hostip); + g_free(s->vdisk_guid); + s->vdisk_guid = NULL; + errno = -ret; + } + + return ret; +} + +static int vxhs_open(BlockDriverState *bs, QDict *options, + int bdrv_flags, Error **errp) +{ + BDRVVXHSState *s = bs->opaque; + AioContext *aio_context; + int qemu_qnio_cfd = -1; + int qemu_rfd = -1; + int ret = 0; + + ret = vxhs_qemu_init(options, s, &qemu_qnio_cfd, &qemu_rfd, errp); + if (ret < 0) { + trace_vxhs_open_fail(ret); + return ret; + } + + s->vdisk_hostinfo.qnio_cfd = qemu_qnio_cfd; + s->vdisk_hostinfo.vdisk_rfd = qemu_rfd; + + /* + * Create a pipe for communicating between two threads in different + * context. Set handler for read event, which gets triggered when + * IO completion is done by non-QEMU context. + */ + ret = qemu_pipe(s->fds); + if (ret < 0) { + trace_vxhs_open_epipe(ret); + ret = -errno; + goto errout; + } + fcntl(s->fds[VDISK_FD_READ], F_SETFL, O_NONBLOCK); + + aio_context = bdrv_get_aio_context(bs); + aio_set_fd_handler(aio_context, s->fds[VDISK_FD_READ], + false, vxhs_aio_event_reader, NULL, s); + return 0; + +errout: + /* + * Close remote vDisk device if it was opened earlier + */ + vxhs_qnio_iio_close(s); + trace_vxhs_open_fail(ret); + return ret; +} + +static const AIOCBInfo vxhs_aiocb_info = { + .aiocb_size = sizeof(VXHSAIOCB) +}; + +/* + * This allocates QEMU-VXHS callback for each IO + * and is passed to QNIO. When QNIO completes the work, + * it will be passed back through the callback. + */ +static BlockAIOCB *vxhs_aio_rw(BlockDriverState *bs, int64_t sector_num, + QEMUIOVector *qiov, int nb_sectors, + BlockCompletionFunc *cb, void *opaque, int iodir) +{ + VXHSAIOCB *acb = NULL; + BDRVVXHSState *s = bs->opaque; + size_t size; + uint64_t offset; + int iio_flags = 0; + int ret = 0; + uint32_t rfd = s->vdisk_hostinfo.vdisk_rfd; + + offset = sector_num * BDRV_SECTOR_SIZE; + size = nb_sectors * BDRV_SECTOR_SIZE; + acb = qemu_aio_get(&vxhs_aiocb_info, bs, cb, opaque); + /* + * Setup or initialize VXHSAIOCB. + * Every single field should be initialized since + * acb will be picked up from the slab without + * initializing with zero. + */ + acb->io_offset = offset; + acb->size = size; + acb->err = 0; + acb->qiov = qiov; + acb->direction = iodir; + + iio_flags = (IIO_FLAG_DONE | IIO_FLAG_ASYNC); + + switch (iodir) { + case VDISK_AIO_WRITE: + ret = iio_writev(qniolib.context, rfd, qiov->iov, qiov->niov, + offset, (uint64_t)size, (void *)acb, iio_flags); + break; + case VDISK_AIO_READ: + ret = iio_readv(qniolib.context, rfd, qiov->iov, qiov->niov, + offset, (uint64_t)size, (void *)acb, iio_flags); + break; + default: + trace_vxhs_aio_rw_invalid(iodir); + goto errout; + } + + if (ret != 0) { + trace_vxhs_aio_rw_ioerr(s->vdisk_guid, iodir, size, offset, + acb, ret, errno); + goto errout; + } + return &acb->common; + +errout: + qemu_aio_unref(acb); + return NULL; +} + +static BlockAIOCB *vxhs_aio_readv(BlockDriverState *bs, + int64_t sector_num, QEMUIOVector *qiov, + int nb_sectors, + BlockCompletionFunc *cb, void *opaque) +{ + return vxhs_aio_rw(bs, sector_num, qiov, nb_sectors, cb, + opaque, VDISK_AIO_READ); +} + +static BlockAIOCB *vxhs_aio_writev(BlockDriverState *bs, + int64_t sector_num, QEMUIOVector *qiov, + int nb_sectors, + BlockCompletionFunc *cb, void *opaque) +{ + return vxhs_aio_rw(bs, sector_num, qiov, nb_sectors, + cb, opaque, VDISK_AIO_WRITE); +} + +static void vxhs_close(BlockDriverState *bs) +{ + BDRVVXHSState *s = bs->opaque; + + trace_vxhs_close(s->vdisk_guid); + close(s->fds[VDISK_FD_READ]); + close(s->fds[VDISK_FD_WRITE]); + + /* + * Clearing all the event handlers for oflame registered to QEMU + */ + aio_set_fd_handler(bdrv_get_aio_context(bs), s->fds[VDISK_FD_READ], + false, NULL, NULL, NULL); + g_free(s->vdisk_guid); + s->vdisk_guid = NULL; + vxhs_qnio_iio_close(s); + + /* + * Free the dynamically allocated hostip string + */ + g_free(s->vdisk_hostinfo.hostip); + s->vdisk_hostinfo.hostip = NULL; + s->vdisk_hostinfo.port = 0; +} + +static int64_t vxhs_get_vdisk_stat(BDRVVXHSState *s) +{ + int64_t vdisk_size = -1; + int ret = 0; + uint32_t rfd = s->vdisk_hostinfo.vdisk_rfd; + + ret = iio_ioctl(qniolib.context, rfd, IOR_VDISK_STAT, &vdisk_size, NULL, 0); + if (ret < 0) { + trace_vxhs_get_vdisk_stat_err(s->vdisk_guid, ret, errno); + return -EIO; + } + + trace_vxhs_get_vdisk_stat(s->vdisk_guid, vdisk_size); + return vdisk_size; +} + +/* + * Returns the size of vDisk in bytes. This is required + * by QEMU block upper block layer so that it is visible + * to guest. + */ +static int64_t vxhs_getlength(BlockDriverState *bs) +{ + BDRVVXHSState *s = bs->opaque; + int64_t vdisk_size; + + vdisk_size = vxhs_get_vdisk_stat(s); + if (vdisk_size < 0) { + return -EIO; + } + + return vdisk_size; +} + +static void vxhs_detach_aio_context(BlockDriverState *bs) +{ + BDRVVXHSState *s = bs->opaque; + + aio_set_fd_handler(bdrv_get_aio_context(bs), s->fds[VDISK_FD_READ], + false, NULL, NULL, NULL); +} + +static void vxhs_attach_aio_context(BlockDriverState *bs, + AioContext *new_context) +{ + BDRVVXHSState *s = bs->opaque; + + aio_set_fd_handler(new_context, s->fds[VDISK_FD_READ], + false, vxhs_aio_event_reader, NULL, s); +} + +static BlockDriver bdrv_vxhs = { + .format_name = "vxhs", + .protocol_name = "vxhs", + .instance_size = sizeof(BDRVVXHSState), + .bdrv_file_open = vxhs_open, + .bdrv_parse_filename = vxhs_parse_filename, + .bdrv_close = vxhs_close, + .bdrv_getlength = vxhs_getlength, + .bdrv_aio_readv = vxhs_aio_readv, + .bdrv_aio_writev = vxhs_aio_writev, + .bdrv_detach_aio_context = vxhs_detach_aio_context, + .bdrv_attach_aio_context = vxhs_attach_aio_context, +}; + +static void bdrv_vxhs_init(void) +{ + bdrv_register(&bdrv_vxhs); +} + +block_init(bdrv_vxhs_init); diff --git a/configure b/configure index 6b7acb1..a50d1f8 100755 --- a/configure +++ b/configure @@ -322,6 +322,7 @@ numa="" tcmalloc="no" jemalloc="no" replication="yes" +vxhs="" # parse CC options first for opt do @@ -1167,6 +1168,11 @@ for opt do ;; --enable-replication) replication="yes" ;; + --disable-vxhs) vxhs="no" + ;; + --enable-vxhs) vxhs="yes" + ;; + *) echo "ERROR: unknown option $opt" echo "Try '$0 --help' for more information" @@ -1400,6 +1406,7 @@ disabled with --disable-FEATURE, default is enabled if available: tcmalloc tcmalloc support jemalloc jemalloc support replication replication support + vxhs Veritas HyperScale vDisk backend support NOTE: The object files are built at the place where configure is launched EOF @@ -4694,6 +4701,33 @@ if do_cc -nostdlib -Wl,-r -Wl,--no-relax -o $TMPMO $TMPO; then fi ########################################## +# Veritas HyperScale block driver VxHS +# Check if libqnio is installed + +if test "$vxhs" != "no" ; then + cat > $TMPC <<EOF +#include <stdint.h> +#include <qnio/qnio_api.h> + +void *vxhs_callback; + +int main(void) { + iio_init(QNIO_VERSION, vxhs_callback); + return 0; +} +EOF + vxhs_libs="-lqnio" + if compile_prog "" "$vxhs_libs" ; then + vxhs=yes + else + if test "$vxhs" = "yes" ; then + feature_not_found "vxhs block device" "Install libqnio. See github" + fi + vxhs=no + fi +fi + +########################################## # End of CC checks # After here, no more $cc or $ld runs @@ -5060,6 +5094,7 @@ echo "tcmalloc support $tcmalloc" echo "jemalloc support $jemalloc" echo "avx2 optimization $avx2_opt" echo "replication support $replication" +echo "VxHS block device $vxhs" if test "$sdl_too_old" = "yes"; then echo "-> Your SDL version is too old - please upgrade to have SDL support" @@ -5672,6 +5707,12 @@ if test "$pthread_setname_np" = "yes" ; then echo "CONFIG_PTHREAD_SETNAME_NP=y" >> $config_host_mak fi +if test "$vxhs" = "yes" ; then + echo "CONFIG_VXHS=y" >> $config_host_mak + echo "VXHS_CFLAGS=$vxhs_cflags" >> $config_host_mak + echo "VXHS_LIBS=$vxhs_libs" >> $config_host_mak +fi + if test "$tcg_interpreter" = "yes"; then QEMU_INCLUDES="-I\$(SRC_PATH)/tcg/tci $QEMU_INCLUDES" elif test "$ARCH" = "sparc64" ; then diff --git a/qapi/block-core.json b/qapi/block-core.json index 5af040b..f1c650f 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -1714,7 +1714,7 @@ # # @host_device, @host_cdrom: Since 2.1 # @gluster: Since 2.7 -# @nbd, @nfs, @replication, @ssh: Since 2.8 +# @nbd, @nfs, @replication, @ssh, @vxhs: Since 2.8 # # Since: 2.0 ## @@ -1724,7 +1724,7 @@ 'host_device', 'http', 'https', 'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'parallels', 'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'replication', 'ssh', 'tftp', 'vdi', 'vhdx', 'vmdk', 'vpc', - 'vvfat' ] } + 'vvfat','vxhs' ] } ## # @BlockdevOptionsFile @@ -2352,6 +2352,21 @@ 'data': { '*offset': 'int', '*size': 'int' } } ## +# @BlockdevOptionsVxHS +# +# Driver specific block device options for VxHS +# +# @vdisk-id: UUID of VxHS volume +# +# @server: vxhs server IP, port +# +# Since: 2.8 +## +{ 'struct': 'BlockdevOptionsVxHS', + 'data': { 'vdisk-id': 'str', + 'server': 'InetSocketAddress' } } + +## # @BlockdevOptions # # Options for creating a block device. Many options are available for all @@ -2415,7 +2430,8 @@ 'vhdx': 'BlockdevOptionsGenericFormat', 'vmdk': 'BlockdevOptionsGenericCOWFormat', 'vpc': 'BlockdevOptionsGenericFormat', - 'vvfat': 'BlockdevOptionsVVFAT' + 'vvfat': 'BlockdevOptionsVVFAT', + 'vxhs': 'BlockdevOptionsVxHS' } } ##

[v5] block/vxhs.c Add support for a new block device type called "vxhs"

Commit Message

Comments

Patch