From patchwork Thu Mar 24 08:29:54 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Cornelia Huck X-Patchwork-Id: 8658911 Return-Path: X-Original-To: patchwork-qemu-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 8221B9F44D for ; Thu, 24 Mar 2016 08:32:27 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 865E520395 for ; Thu, 24 Mar 2016 08:32:26 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 25FFD20351 for ; Thu, 24 Mar 2016 08:32:25 +0000 (UTC) Received: from localhost ([::1]:48487 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aj0gq-0004VK-Gx for patchwork-qemu-devel@patchwork.kernel.org; Thu, 24 Mar 2016 04:32:24 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51968) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aj0em-00018P-8g for qemu-devel@nongnu.org; Thu, 24 Mar 2016 04:30:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aj0ei-0001gY-TL for qemu-devel@nongnu.org; Thu, 24 Mar 2016 04:30:16 -0400 Received: from e06smtp13.uk.ibm.com ([195.75.94.109]:51559) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aj0ei-0001gD-Jt for qemu-devel@nongnu.org; Thu, 24 Mar 2016 04:30:12 -0400 Received: from localhost by e06smtp13.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 24 Mar 2016 08:30:11 -0000 Received: from d06dlp03.portsmouth.uk.ibm.com (9.149.20.15) by e06smtp13.uk.ibm.com (192.168.101.143) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Thu, 24 Mar 2016 08:30:09 -0000 X-IBM-Helo: d06dlp03.portsmouth.uk.ibm.com X-IBM-MailFrom: cornelia.huck@de.ibm.com X-IBM-RcptTo: qemu-devel@nongnu.org Received: from b06cxnps4074.portsmouth.uk.ibm.com (d06relay11.portsmouth.uk.ibm.com [9.149.109.196]) by d06dlp03.portsmouth.uk.ibm.com (Postfix) with ESMTP id C2ABD1B08070 for ; Thu, 24 Mar 2016 08:30:41 +0000 (GMT) Received: from d06av06.portsmouth.uk.ibm.com (d06av06.portsmouth.uk.ibm.com [9.149.37.217]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id u2O8U9wt61341852 for ; Thu, 24 Mar 2016 08:30:09 GMT Received: from d06av06.portsmouth.uk.ibm.com (localhost [127.0.0.1]) by d06av06.portsmouth.uk.ibm.com (8.14.4/8.14.4/NCO v10.0 AVout) with ESMTP id u2O8U8Xc005599 for ; Thu, 24 Mar 2016 04:30:09 -0400 Received: from gondolin.boeblingen.de.ibm.com (dyn-9-152-224-197.boeblingen.de.ibm.com [9.152.224.197]) by d06av06.portsmouth.uk.ibm.com (8.14.4/8.14.4/NCO v10.0 AVin) with ESMTP id u2O8U5Sb005417 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 24 Mar 2016 04:30:08 -0400 From: Cornelia Huck To: peter.maydell@linaro.org Date: Thu, 24 Mar 2016 09:29:54 +0100 Message-Id: <1458808203-21339-6-git-send-email-cornelia.huck@de.ibm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1458808203-21339-1-git-send-email-cornelia.huck@de.ibm.com> References: <1458808203-21339-1-git-send-email-cornelia.huck@de.ibm.com> X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 16032408-0013-0000-0000-00000BD21988 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 195.75.94.109 Cc: qemu-devel@nongnu.org, agraf@suse.de, borntraeger@de.ibm.com, jfrei@linux.vnet.ibm.com, Cornelia Huck , "Eugene \(jno\) Dvurechenski" Subject: [Qemu-devel] [PULL for-2.6 05/14] pc-bios/s390-ccw: update virtio implementation to allow up to 3 vrings X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: "Eugene (jno) Dvurechenski" Add ability to work with up to 3 vrings, which is required for virtio-scsi implementation. Implement the optional cookie to speed up processing of virtio notifications. Reviewed-by: Cornelia Huck Signed-off-by: Eugene (jno) Dvurechenski Signed-off-by: Cornelia Huck --- pc-bios/s390-ccw/main.c | 1 - pc-bios/s390-ccw/s390-ccw.h | 1 - pc-bios/s390-ccw/virtio.c | 140 +++++++++++++++++++++++++++----------------- pc-bios/s390-ccw/virtio.h | 4 ++ 4 files changed, 90 insertions(+), 56 deletions(-) diff --git a/pc-bios/s390-ccw/main.c b/pc-bios/s390-ccw/main.c index 7f192f3..6bf44a7 100644 --- a/pc-bios/s390-ccw/main.c +++ b/pc-bios/s390-ccw/main.c @@ -12,7 +12,6 @@ #include "virtio.h" char stack[PAGE_SIZE * 8] __attribute__((__aligned__(PAGE_SIZE))); -char ring_area[PAGE_SIZE * 8] __attribute__((__aligned__(PAGE_SIZE))); uint64_t boot_value; static SubChannelId blk_schid = { .one = 1 }; diff --git a/pc-bios/s390-ccw/s390-ccw.h b/pc-bios/s390-ccw/s390-ccw.h index a5c0684..3e00d42 100644 --- a/pc-bios/s390-ccw/s390-ccw.h +++ b/pc-bios/s390-ccw/s390-ccw.h @@ -61,7 +61,6 @@ void consume_sclp_int(void); void panic(const char *string); void write_subsystem_identification(void); extern char stack[PAGE_SIZE * 8] __attribute__((__aligned__(PAGE_SIZE))); -extern char ring_area[PAGE_SIZE * 8] __attribute__((__aligned__(PAGE_SIZE))); extern uint64_t boot_value; /* sclp-ascii.c */ diff --git a/pc-bios/s390-ccw/virtio.c b/pc-bios/s390-ccw/virtio.c index 64c6e07..d366aa3 100644 --- a/pc-bios/s390-ccw/virtio.c +++ b/pc-bios/s390-ccw/virtio.c @@ -11,29 +11,35 @@ #include "s390-ccw.h" #include "virtio.h" -static VRing block; +static VRing block[VIRTIO_MAX_VQS]; +static char ring_area[VIRTIO_RING_SIZE * VIRTIO_MAX_VQS] + __attribute__((__aligned__(PAGE_SIZE))); +static int nr_vqs = 1; static char chsc_page[PAGE_SIZE] __attribute__((__aligned__(PAGE_SIZE))); +/* virtio spec v1.0 para 4.3.3.2 */ static long kvm_hypercall(unsigned long nr, unsigned long param1, - unsigned long param2) + unsigned long param2, unsigned long param3) { register ulong r_nr asm("1") = nr; register ulong r_param1 asm("2") = param1; register ulong r_param2 asm("3") = param2; + register ulong r_param3 asm("4") = param3; register long retval asm("2"); asm volatile ("diag 2,4,0x500" : "=d" (retval) - : "d" (r_nr), "0" (r_param1), "r"(r_param2) + : "d" (r_nr), "0" (r_param1), "r"(r_param2), "d"(r_param3) : "memory", "cc"); return retval; } -static void virtio_notify(SubChannelId schid) +static long virtio_notify(SubChannelId schid, int vq_idx, long cookie) { - kvm_hypercall(KVM_S390_VIRTIO_CCW_NOTIFY, *(u32 *)&schid, 0); + return kvm_hypercall(KVM_S390_VIRTIO_CCW_NOTIFY, *(u32 *)&schid, + vq_idx, cookie); } /*********************************************** @@ -106,15 +112,17 @@ static void virtio_reset(SubChannelId schid) run_ccw(schid, CCW_CMD_VDEV_RESET, NULL, 0); } -static void vring_init(VRing *vr, unsigned int num, void *p, - unsigned long align) +static void vring_init(VRing *vr, VqInfo *info) { + void *p = (void *) info->queue; + debug_print_addr("init p", p); - vr->num = num; + vr->id = info->index; + vr->num = info->num; vr->desc = p; - vr->avail = p + num * sizeof(VRingDesc); - vr->used = (void *)(((unsigned long)&vr->avail->ring[num] + align-1) - & ~(align - 1)); + vr->avail = p + info->num * sizeof(VRingDesc); + vr->used = (void *)(((unsigned long)&vr->avail->ring[info->num] + + info->align - 1) & ~(info->align - 1)); /* Zero out all relevant field */ vr->avail->flags = 0; @@ -125,13 +133,15 @@ static void vring_init(VRing *vr, unsigned int num, void *p, vr->used->idx = 0; vr->used_idx = 0; vr->next_idx = 0; + vr->cookie = 0; debug_print_addr("init vr", vr); } -static void vring_notify(SubChannelId schid) +static bool vring_notify(VRing *vr) { - virtio_notify(schid); + vr->cookie = virtio_notify(vr->schid, vr->id, vr->cookie); + return vr->cookie >= 0; } static void vring_send_buf(VRing *vr, void *p, int len, int flags) @@ -167,6 +177,21 @@ ulong get_second(void) return (get_clock() >> 12) / 1000000; } +static int vr_poll(VRing *vr) +{ + if (vr->used->idx == vr->used_idx) { + vring_notify(vr); + yield(); + return 0; + } + + vr->used_idx = vr->used->idx; + vr->next_idx = 0; + vr->desc[0].len = 0; + vr->desc[0].flags = 0; + return 1; /* vr has been updated */ +} + /* * Wait for the host to reply. * @@ -174,28 +199,24 @@ ulong get_second(void) * * Returns 0 on success, 1 on timeout. */ -static int vring_wait_reply(VRing *vr, int timeout) +static int vring_wait_reply(int timeout) { ulong target_second = get_second() + timeout; - SubChannelId schid = vr->schid; - int r = 0; - /* Wait until the used index has moved. */ - while (vr->used->idx == vr->used_idx) { - vring_notify(schid); - if (timeout && (get_second() >= target_second)) { - r = 1; - break; + /* Wait for any queue to be updated by the host */ + do { + int i, r = 0; + + for (i = 0; i < nr_vqs; i++) { + r += vr_poll(&block[i]); } yield(); - } - - vr->used_idx = vr->used->idx; - vr->next_idx = 0; - vr->desc[0].len = 0; - vr->desc[0].flags = 0; + if (r) { + return 0; + } + } while (!timeout || (get_second() < target_second)); - return r; + return 1; } /*********************************************** @@ -213,21 +234,21 @@ int virtio_read_many(ulong sector, void *load_addr, int sec_num) out_hdr.ioprio = 99; out_hdr.sector = virtio_sector_adjust(sector); - vring_send_buf(&block, &out_hdr, sizeof(out_hdr), VRING_DESC_F_NEXT); + vring_send_buf(&block[0], &out_hdr, sizeof(out_hdr), VRING_DESC_F_NEXT); /* This is where we want to receive data */ - vring_send_buf(&block, load_addr, virtio_get_block_size() * sec_num, + vring_send_buf(&block[0], load_addr, virtio_get_block_size() * sec_num, VRING_DESC_F_WRITE | VRING_HIDDEN_IS_CHAIN | VRING_DESC_F_NEXT); /* status field */ - vring_send_buf(&block, &status, sizeof(u8), VRING_DESC_F_WRITE | + vring_send_buf(&block[0], &status, sizeof(u8), VRING_DESC_F_WRITE | VRING_HIDDEN_IS_CHAIN); /* Now we can tell the host to read */ - vring_wait_reply(&block, 0); + vring_wait_reply(0); - r = drain_irqs(block.schid); + r = drain_irqs(block[0].schid); if (r) { /* Well, whatever status is supposed to contain... */ status = 1; @@ -363,15 +384,18 @@ uint64_t virtio_get_blocks(void) (virtio_get_block_size() / VIRTIO_SECTOR_SIZE); } -void virtio_setup_block(SubChannelId schid) +static void virtio_setup_ccw(SubChannelId schid, + int nvr, void *cfg, int cfg_size) { - VqInfo info; - VqConfig config = {}; + int i; blk_cfg.blk_size = 0; /* mark "illegal" - setup started... */ + nr_vqs = nvr; guessed_disk_nature = false; virtio_reset(schid); + IPL_assert(run_ccw(schid, CCW_CMD_READ_CONF, cfg, cfg_size) == 0, + "Could not get block device configuration"); /* * Skipping CCW_CMD_READ_FEAT. We're not doing anything fancy, and @@ -379,25 +403,33 @@ void virtio_setup_block(SubChannelId schid) * expect it. */ - config.index = 0; - if (run_ccw(schid, CCW_CMD_READ_VQ_CONF, &config, sizeof(config))) { - panic("Could not get block device VQ configuration\n"); - } - if (run_ccw(schid, CCW_CMD_READ_CONF, &blk_cfg, sizeof(blk_cfg))) { - panic("Could not get block device configuration\n"); + for (i = 0; i < nr_vqs; i++) { + VqInfo info = { + .queue = (unsigned long long) ring_area + (i * VIRTIO_RING_SIZE), + .align = KVM_S390_VIRTIO_RING_ALIGN, + .index = i, + .num = 0, + }; + VqConfig config = { + .index = i, + .num = 0, + }; + + IPL_assert( + run_ccw(schid, CCW_CMD_READ_VQ_CONF, &config, sizeof(config)) == 0, + "Could not get block device VQ configuration"); + info.num = config.num; + vring_init(&block[i], &info); + block[i].schid = schid; + IPL_assert(run_ccw(schid, CCW_CMD_SET_VQ, &info, sizeof(info)) == 0, + "Cannot set VQ info"); } - vring_init(&block, config.num, ring_area, - KVM_S390_VIRTIO_RING_ALIGN); - - info.queue = (unsigned long long) ring_area; - info.align = KVM_S390_VIRTIO_RING_ALIGN; - info.index = 0; - info.num = config.num; - block.schid = schid; + virtio_set_status(schid, VIRTIO_CONFIG_S_DRIVER_OK); +} - if (!run_ccw(schid, CCW_CMD_SET_VQ, &info, sizeof(info))) { - virtio_set_status(schid, VIRTIO_CONFIG_S_DRIVER_OK); - } +void virtio_setup_block(SubChannelId schid) +{ + virtio_setup_ccw(schid, 1, &blk_cfg, sizeof(blk_cfg)); if (!virtio_ipl_disk_is_valid()) { /* make sure all getters but blocksize return 0 for invalid IPL disk */ diff --git a/pc-bios/s390-ccw/virtio.h b/pc-bios/s390-ccw/virtio.h index af6e142..d17b135 100644 --- a/pc-bios/s390-ccw/virtio.h +++ b/pc-bios/s390-ccw/virtio.h @@ -72,6 +72,8 @@ struct VirtioDev { }; typedef struct VirtioDev VirtioDev; +#define VIRTIO_RING_SIZE (PAGE_SIZE * 8) +#define VIRTIO_MAX_VQS 3 #define KVM_S390_VIRTIO_RING_ALIGN 4096 #define VRING_USED_F_NO_NOTIFY 1 @@ -130,6 +132,8 @@ struct VRing { VRingAvail *avail; VRingUsed *used; SubChannelId schid; + long cookie; + int id; }; typedef struct VRing VRing;