From patchwork Tue Sep 5 11:16:40 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Halil Pasic X-Patchwork-Id: 9938465 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 66F8D602B6 for ; Tue, 5 Sep 2017 11:18:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 55B892893F for ; Tue, 5 Sep 2017 11:18:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4A25228945; Tue, 5 Sep 2017 11:18:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4DD802893F for ; Tue, 5 Sep 2017 11:18:53 +0000 (UTC) Received: from localhost ([::1]:58154 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dpBsa-0005DJ-F8 for patchwork-qemu-devel@patchwork.kernel.org; Tue, 05 Sep 2017 07:18:52 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46412) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dpBr2-0005Cj-4G for qemu-devel@nongnu.org; Tue, 05 Sep 2017 07:17:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dpBqv-0007yW-VI for qemu-devel@nongnu.org; Tue, 05 Sep 2017 07:17:16 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:35655 helo=mx0a-001b2d01.pphosted.com) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1dpBqv-0007wu-PX for qemu-devel@nongnu.org; Tue, 05 Sep 2017 07:17:09 -0400 Received: from pps.filterd (m0098421.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.21/8.16.0.21) with SMTP id v85BDqJF044609 for ; Tue, 5 Sep 2017 07:17:08 -0400 Received: from e06smtp15.uk.ibm.com (e06smtp15.uk.ibm.com [195.75.94.111]) by mx0a-001b2d01.pphosted.com with ESMTP id 2cspu36g19-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Tue, 05 Sep 2017 07:17:07 -0400 Received: from localhost by e06smtp15.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 5 Sep 2017 12:17:06 +0100 Received: from b06cxnps4074.portsmouth.uk.ibm.com (9.149.109.196) by e06smtp15.uk.ibm.com (192.168.101.145) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 5 Sep 2017 12:17:04 +0100 Received: from d06av26.portsmouth.uk.ibm.com (d06av26.portsmouth.uk.ibm.com [9.149.105.62]) by b06cxnps4074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v85BH3vO21889138; Tue, 5 Sep 2017 11:17:03 GMT Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A4D19AE04D; Tue, 5 Sep 2017 12:12:20 +0100 (BST) Received: from d06av26.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 64936AE045; Tue, 5 Sep 2017 12:12:20 +0100 (BST) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av26.portsmouth.uk.ibm.com (Postfix) with ESMTPS; Tue, 5 Sep 2017 12:12:20 +0100 (BST) From: Halil Pasic To: Cornelia Huck Date: Tue, 5 Sep 2017 13:16:40 +0200 X-Mailer: git-send-email 2.13.5 X-TM-AS-GCONF: 00 x-cbid: 17090511-0020-0000-0000-000003B3AFAC X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17090511-0021-0000-0000-000042441837 Message-Id: <20170905111645.18068-1-pasic@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-09-05_06:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=2 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1709050172 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [generic] [fuzzy] X-Received-From: 148.163.158.5 Subject: [Qemu-devel] [PATCH 0/5] add CCW indirect data access support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dong Jia Shi , Halil Pasic , Pierre Morel , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Abstract -------- The objective of this series is introducing CCW IDA (indirect data access) support to our virtual channel subsystem implementation. Briefly CCW IDA can be thought of as a kind of a scatter gather support for a single CCW. If certain flags are set, the cda is to be interpreted as an address to a list which in turn holds further addresses designating the actual data. Thus the scheme which we are currently using for accessing CCW payload does not work in general case. Currently there is no immediate need for proper IDA handling (no use case), but since it IDA is a non-optional part of the architecture, the only way towards AR compliance is actually implementing IDA. Testing ------- Because there is not matching (emulated) QEMU device for the Linux drivers using IDA testing requires extra effort. The last patch in this series introduces an emulated CCW device (just for testing) which does support IDA (using the infrastructure introduced in the previous patches). The last patch isn't strictly meant for merging. To exercise this QEMU device an out-of-tree Linux kernel module is provided at the end of this cover letter (apply to an empty repo using git am -c ). Halil Pasic (5): s390x/css: introduce css data stream s390x/css: use ccw data stream virtio-ccw: use ccw data stream s390x/css: support ccw IDA s390x/ccs: add ccw-tester emulated device hw/s390x/Makefile.objs | 1 + hw/s390x/ccw-tester.c | 179 +++++++++++++++++++++++++++++++++++++++++++++++++ hw/s390x/css.c | 168 +++++++++++++++++++++++++++++++++++++++++++++- hw/s390x/virtio-ccw.c | 158 +++++++++++++------------------------------ include/hw/s390x/css.h | 67 ++++++++++++++++++ 5 files changed, 461 insertions(+), 112 deletions(-) create mode 100644 hw/s390x/ccw-tester.c diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..1b9eac9 --- /dev/null +++ b/.gitignore @@ -0,0 +1,8 @@ +#ignore these +*.o +*.cmd +*.ko +*.mod.c +Module.symvers +modules.order +.tmp_versions/ diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..0583456 --- /dev/null +++ b/Makefile @@ -0,0 +1,10 @@ +ifneq ($(KERNELRELEASE),) +obj-m := ccw_tester.o +else +# normal makefile +KDIR ?= /lib/modules/`uname -r`/build + +default: + $(MAKE) -C $(KDIR) M=$$PWD + +endif diff --git a/ccw_tester.c b/ccw_tester.c new file mode 100644 index 0000000..320486a --- /dev/null +++ b/ccw_tester.c @@ -0,0 +1,420 @@ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +inline bool _ccw_test_assert(bool expr, const char *loc, int ln, + const char *expl) +{ + if (expr) + printk(KERN_NOTICE "ok -- %s:%d\n", loc, ln); + else + printk(KERN_WARNING "not ok -- %s:%d (%s)\n", loc, ln, expl); + return expr; +} + + +#define ccw_test_assert(_expr, _expl) ({_ccw_test_assert((_expr), \ + __func__, __LINE__, (_expl)); }) + +struct workqueue_struct *work_q; + +static __u16 cu_type = 0x3831; +module_param(cu_type, ushort, 0000); +MODULE_PARM_DESC(cu_type, "Use this cu type for matching (default 0x3831)"); + + +static struct ccw_device_id ccw_tester_ids[] = { + { CCW_DEVICE(0, 0) }, /* placeholder */ + {}, +}; + +struct ccw_test_work { + struct work_struct work; + struct ccw1 *ccw; + __u32 intparm; + void *private; + void (*setup)(struct ccw_test_work *w); + void (*do_test)(struct ccw_test_work *w); + void (*teardown)(struct ccw_test_work *w); + struct irb irb; + int ret; + bool doing_io; +}; + +struct ccw_tester_device { + spinlock_t lock; + wait_queue_head_t wait_q; + struct ccw_device *cdev; + struct ccw_test_work work; + bool work_pending; +}; + +static struct ccw_tester_device *to_mydev(struct ccw_device *cdev) +{ + return dev_get_drvdata(&(cdev->dev)); +} + + +static void ccw_tester_auto_online(void *data, async_cookie_t cookie) +{ + struct ccw_device *cdev = data; + int ret; + + ret = ccw_device_set_online(cdev); + if (ret) + dev_warn(&cdev->dev, "Failed to set online: %d\n", ret); +} + +static void do_io_work(struct ccw_tester_device *tdev) +{ + struct ccw_test_work *w = &tdev->work; + unsigned long flags; + int retry = 124; + + do { + spin_lock_irqsave(get_ccwdev_lock(tdev->cdev), flags); + tdev->work.doing_io = true; + w->ret = ccw_device_start(tdev->cdev, w->ccw, w->intparm, 0, 0); + spin_unlock_irqrestore(get_ccwdev_lock(tdev->cdev), flags); + cpu_relax(); + } while (w->ret == -EBUSY && --retry > 0); + wait_event(tdev->wait_q, w->doing_io == false); +} + + +static void w_fib_w_setup(struct ccw_test_work *w) +{ + const int test_fib_length = 32; + u32 *test_fib; + int i; + + test_fib = kcalloc(test_fib_length, sizeof(u32), + GFP_DMA | GFP_KERNEL); + if (!test_fib) + w->ret = -ENOMEM; + w->private = test_fib; + + test_fib[0] = 1; + test_fib[1] = 2; + for (i = 2; i < test_fib_length; ++i) + test_fib[i] = test_fib[i - 1] + test_fib[i - 2]; + + w->ccw->cmd_code = 0x02; + w->ccw->count = sizeof(*test_fib) * test_fib_length; + w->ccw->cda = (__u32)(unsigned long) test_fib; +} + +static void do_test_do_io(struct ccw_test_work *w) +{ + struct ccw_tester_device *tdev; + + tdev = container_of(w, struct ccw_tester_device, work); + do_io_work(tdev); +} + +static void basic_teardown(struct ccw_test_work *w) +{ + kfree(w->private); + w->private = NULL; + if (w->ret) + printk(KERN_WARNING "w_fib_w_teardown ret = %d\n", w->ret); +} + +static int irb_is_error(struct irb *irb) +{ + if (scsw_cstat(&irb->scsw) != 0) + return 1; + if (scsw_dstat(&irb->scsw) & ~(DEV_STAT_CHN_END | DEV_STAT_DEV_END)) + return 1; + if (scsw_cc(&irb->scsw) != 0) + return 1; + return 0; +} + +static void ccw_tester_int_handler(struct ccw_device *cdev, + unsigned long intparm, + struct irb *irb) +{ + struct ccw_tester_device *tdev = to_mydev(cdev); + + memcpy(&tdev->work.irb, irb, sizeof(*irb)); + tdev->work.doing_io = false; + wake_up(&tdev->wait_q); +} + +static bool expect_is_not_fib(struct irb *irb, int count_expected) +{ + if (!(irb_is_error(irb) + && (scsw_dstat(&irb->scsw) & DEV_STAT_UNIT_EXCEP) + && scsw_stctl(&irb->scsw) & SCSW_STCTL_ALERT_STATUS)) + return false; + if (irb->scsw.cmd.count == count_expected) + return true; + printk(KERN_NOTICE + "expected residual count of %d got %d (fib at wrong place)\n", + count_expected, irb->scsw.cmd.count); + return false; +} + + +static void w_fib_do_test(struct ccw_test_work *w) +{ + u32 *test_fib = w->private; + + do_test_do_io(w); + ccw_test_assert(!irb_is_error(&w->irb), "completion expected"); + test_fib[25] = 0; + do_test_do_io(w); + ccw_test_assert(expect_is_not_fib(&w->irb, + (31-25)*sizeof(u32)), "expected non fib"); +} + + +static int queue_ccw_test_work(struct ccw_tester_device *tdev, + void (*setup)(struct ccw_test_work *), + void (*do_test)(struct ccw_test_work *), + void (*teardown)(struct ccw_test_work *)) +{ + if (!spin_trylock(&tdev->lock)) + return -EBUSY; + if (tdev->work_pending) { + spin_unlock(&tdev->lock); + return -EBUSY; + } + tdev->work_pending = true; + tdev->work.setup = setup; + tdev->work.do_test = do_test; + tdev->work.teardown = teardown; + queue_work(work_q, &tdev->work.work); + spin_unlock(&tdev->lock); + return 0; +} + + +static ssize_t w_fib_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct ccw_tester_device *tdev = to_mydev(to_ccwdev(dev)); + int ret; + + ret = queue_ccw_test_work(tdev, + w_fib_w_setup, w_fib_do_test, basic_teardown); + return ret ? ret : count; +} + +static u32 *u32_arr_in_idal_buf_at(struct idal_buffer const *ib, int i) +{ + u64 b = IDA_BLOCK_SIZE/sizeof(u32); + + return (u32 *)(ib->data[i/b]) + i % b; +} + +#define IDAL_TEST_BYTES (IDA_BLOCK_SIZE * 3 + IDA_BLOCK_SIZE/2) +#define IDAL_TEST_ELEMENTS (IDAL_TEST_BYTES/sizeof(u32)) + +static void w_fib_idal_setup(struct ccw_test_work *w) +{ + struct idal_buffer *ib = NULL; + u32 n, n_1 = 2, n_2 = 1; + int i = 0; + + ib = idal_buffer_alloc(IDAL_TEST_BYTES, 0); + if (IS_ERR(ib)) { + w->ret = PTR_ERR(ib); + return; + } + w->private = ib; + *u32_arr_in_idal_buf_at(ib, 0) = n_2; + *u32_arr_in_idal_buf_at(ib, 1) = n_1; + for (i = 2; i < IDAL_TEST_ELEMENTS; ++i) { + n = n_1 + n_2; + n_2 = n_1; + n_1 = n; + *u32_arr_in_idal_buf_at(ib, i) = n; + } + idal_buffer_set_cda(ib, w->ccw); + w->ccw->count = IDAL_TEST_BYTES; + w->ccw->cmd_code = 0x02; +} + +static void idal_teardown(struct ccw_test_work *w) +{ + if (w->private) { + idal_buffer_free(w->private); + w->private = NULL; + } + if (w->ret) + printk(KERN_WARNING "w_fib_w_teardown ret = %d\n", w->ret); +} + +static void w_fib_do_idal_test(struct ccw_test_work *w) +{ + struct idal_buffer *ib = w->private; + + /* we have one already set up, fire it */ + do_test_do_io(w); + ccw_test_assert(!irb_is_error(&w->irb), "completion expected"); + + /* let's break fib and check if the device detects it */ + ++(*u32_arr_in_idal_buf_at(ib, IDAL_TEST_ELEMENTS - 5)); + do_test_do_io(w); + ccw_test_assert(expect_is_not_fib(&w->irb, + 4 * sizeof(u32)), "expected non fib"); + /* shorten the seq so the broken element is not included */ + w->ccw->count = IDAL_TEST_BYTES - 5 * sizeof(u32); + do_test_do_io(w); + ccw_test_assert(!irb_is_error(&w->irb), "completion expected"); +} + +static ssize_t w_fib_idal_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + int ret; + struct ccw_tester_device *tdev = to_mydev(to_ccwdev(dev)); + + ret = queue_ccw_test_work(tdev, + w_fib_idal_setup, w_fib_do_idal_test, idal_teardown); + return ret ? ret : count; +} + + +static DEVICE_ATTR_WO(w_fib); +static DEVICE_ATTR_WO(w_fib_idal); + +static void do_ccw_test_work(struct work_struct *work) +{ + + struct ccw_test_work *w; + struct ccw_tester_device *tdev; + + w = container_of(work, struct ccw_test_work, work); + tdev = container_of(w, struct ccw_tester_device, work); + + w->ret = 0; + w->setup(w); + w->do_test(w); + w->teardown(w); + spin_lock(&tdev->lock); + tdev->work_pending = false; + spin_unlock(&tdev->lock); + memset(w->ccw, 0, sizeof(*(w->ccw))); + memset(&w->irb, 0, sizeof(w->irb)); +} + +static int ccw_tester_offline(struct ccw_device *cdev) +{ + struct ccw_tester_device *tdev = to_mydev(cdev); + + if (!tdev) + return 0; + device_remove_file(&(cdev->dev), &dev_attr_w_fib); + device_remove_file(&(cdev->dev), &dev_attr_w_fib_idal); + spin_lock(&tdev->lock); + tdev->work_pending = true; + spin_unlock(&tdev->lock); + kfree(tdev->work.ccw); + tdev->work.ccw = NULL; + kfree(tdev); + dev_set_drvdata(&cdev->dev, NULL); + return 0; +} + +static int ccw_tester_online(struct ccw_device *cdev) +{ + int ret; + struct ccw_tester_device *tdev; + + tdev = kzalloc(sizeof(*tdev), GFP_KERNEL); + if (!tdev) { + dev_warn(&cdev->dev, "Could not get memory\n"); + return -ENOMEM; + } + init_waitqueue_head(&tdev->wait_q); + INIT_WORK(&(tdev->work.work), do_ccw_test_work); + spin_lock_init(&tdev->lock); + tdev->work.ccw = kzalloc(sizeof(*tdev->work.ccw), GFP_DMA | GFP_KERNEL); + if (!tdev) { + dev_warn(&cdev->dev, "Could not get memory\n"); + ret = -ENOMEM; + goto out_free; + } + dev_set_drvdata(&cdev->dev, tdev); + tdev->cdev = cdev; + + ret = device_create_file(&(cdev->dev), &dev_attr_w_fib); + if (ret) + goto out_free; + ret = device_create_file(&(cdev->dev), &dev_attr_w_fib_idal); + if (ret) + goto out_free; + return ret; +out_free: + ccw_tester_offline(cdev); + return ret; +} + +static void ccw_tester_remove(struct ccw_device *cdev) +{ + ccw_device_set_offline(cdev); +} + +static int ccw_tester_probe(struct ccw_device *cdev) +{ + cdev->handler = ccw_tester_int_handler; + async_schedule(ccw_tester_auto_online, cdev); + return 0; +} + +static struct ccw_driver ccw_tester_driver = { + .driver = { + .owner = THIS_MODULE, + .name = "ccw_tester", + }, + .ids = ccw_tester_ids, + .probe = ccw_tester_probe, + .set_online = ccw_tester_online, + .set_offline = ccw_tester_offline, + .remove = ccw_tester_remove, + .int_class = IRQIO_VIR, +}; + + +static int __init ccw_tester_init(void) +{ + work_q = create_singlethread_workqueue("ccw-tester"); + ccw_tester_ids[0].cu_type = cu_type; + return ccw_driver_register(&ccw_tester_driver); +} +module_init(ccw_tester_init); + +static void __exit ccw_tester_exit(void) +{ + ccw_driver_unregister(&ccw_tester_driver); +} +module_exit(ccw_tester_exit); + +MODULE_DESCRIPTION("ccw test driver -- throw ccws at devices"); +MODULE_LICENSE("GPL v2"); +MODULE_AUTHOR("Halil Pasic ");