From patchwork Tue Jun 16 19:50:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Farman X-Patchwork-Id: 11608557 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3DF6D14E3 for ; Tue, 16 Jun 2020 19:51:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 305552082E for ; Tue, 16 Jun 2020 19:51:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731100AbgFPTvD (ORCPT ); Tue, 16 Jun 2020 15:51:03 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:41284 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730393AbgFPTvC (ORCPT ); Tue, 16 Jun 2020 15:51:02 -0400 Received: from pps.filterd (m0098404.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 05GJXbnk052800; Tue, 16 Jun 2020 15:51:01 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 31q0rcf1sn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 16 Jun 2020 15:51:01 -0400 Received: from m0098404.ppops.net (m0098404.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 05GJjgIj112890; Tue, 16 Jun 2020 15:51:01 -0400 Received: from ppma03ams.nl.ibm.com (62.31.33a9.ip4.static.sl-reverse.com [169.51.49.98]) by mx0a-001b2d01.pphosted.com with ESMTP id 31q0rcf1s0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 16 Jun 2020 15:51:00 -0400 Received: from pps.filterd (ppma03ams.nl.ibm.com [127.0.0.1]) by ppma03ams.nl.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 05GJo7ZY013895; Tue, 16 Jun 2020 19:50:58 GMT Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by ppma03ams.nl.ibm.com with ESMTP id 31mpe85vrg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 16 Jun 2020 19:50:58 +0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 05GJotTH64356560 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 16 Jun 2020 19:50:55 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6FEF44C040; Tue, 16 Jun 2020 19:50:55 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 513F54C04E; Tue, 16 Jun 2020 19:50:55 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTPS; Tue, 16 Jun 2020 19:50:55 +0000 (GMT) Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 4958) id EBD64E027E; Tue, 16 Jun 2020 21:50:54 +0200 (CEST) From: Eric Farman To: Cornelia Huck Cc: Jared Rossi , Halil Pasic , linux-s390@vger.kernel.org, kvm@vger.kernel.org, Eric Farman Subject: [RFC PATCH v3 2/3] vfio-ccw: Remove the CP_PENDING FSM state Date: Tue, 16 Jun 2020 21:50:52 +0200 Message-Id: <20200616195053.99253-3-farman@linux.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200616195053.99253-1-farman@linux.ibm.com> References: <20200616195053.99253-1-farman@linux.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216,18.0.687 definitions=2020-06-16_12:2020-06-16,2020-06-16 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 malwarescore=0 phishscore=0 priorityscore=1501 mlxlogscore=999 adultscore=0 impostorscore=0 bulkscore=0 spamscore=0 clxscore=1015 suspectscore=2 lowpriorityscore=0 cotscore=-2147483648 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2004280000 definitions=main-2006160138 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The FSM state is set to CP_PROCESSING when a START operation begins, is set to CP_PENDING when a START operation is done (the SSCH instruction gets a cc0), and is set back to IDLE when the final interrupt is received (or if the START fails somehow). However, it is categorically impossible to distinguish between interrupts when other instructions can generate "final" interrupts via the async region, so using this information for any decision-making at the completion of an I/O is fraught with peril. We could replace "CP_PENDING" with a generic "OPERATION_PENDING" state, but it doesn't appear to buy us anything as far as knowing if we get one interrupt for a START and another for a CLEAR, or if we just get one interrupt for the CLEAR and the START never got off the ground. So let's remove that entirely, and just move the FSM back to IDLE once the START process completes. Signed-off-by: Eric Farman --- drivers/s390/cio/vfio_ccw_drv.c | 3 --- drivers/s390/cio/vfio_ccw_fsm.c | 16 +++------------- drivers/s390/cio/vfio_ccw_ops.c | 3 +-- drivers/s390/cio/vfio_ccw_private.h | 1 - 4 files changed, 4 insertions(+), 19 deletions(-) diff --git a/drivers/s390/cio/vfio_ccw_drv.c b/drivers/s390/cio/vfio_ccw_drv.c index 7e2a790dc9a1..b76c9008871b 100644 --- a/drivers/s390/cio/vfio_ccw_drv.c +++ b/drivers/s390/cio/vfio_ccw_drv.c @@ -101,9 +101,6 @@ static void vfio_ccw_sch_io_todo(struct work_struct *work) memcpy(private->io_region->irb_area, irb, sizeof(*irb)); mutex_unlock(&private->io_mutex); - if (private->mdev && is_final) - private->state = VFIO_CCW_STATE_IDLE; - if (private->io_trigger) eventfd_signal(private->io_trigger, 1); } diff --git a/drivers/s390/cio/vfio_ccw_fsm.c b/drivers/s390/cio/vfio_ccw_fsm.c index d806f88eba72..f0952192480e 100644 --- a/drivers/s390/cio/vfio_ccw_fsm.c +++ b/drivers/s390/cio/vfio_ccw_fsm.c @@ -49,7 +49,6 @@ static int fsm_io_helper(struct vfio_ccw_private *private) */ sch->schib.scsw.cmd.actl |= SCSW_ACTL_START_PEND; ret = 0; - private->state = VFIO_CCW_STATE_CP_PENDING; private->cp.started = true; break; case 1: /* Status pending */ @@ -188,12 +187,6 @@ static void fsm_io_error(struct vfio_ccw_private *private, private->io_region->ret_code = -EIO; } -static void fsm_io_busy(struct vfio_ccw_private *private, - enum vfio_ccw_event event) -{ - private->io_region->ret_code = -EBUSY; -} - static void fsm_io_retry(struct vfio_ccw_private *private, enum vfio_ccw_event event) { @@ -309,6 +302,7 @@ static void fsm_io_request(struct vfio_ccw_private *private, cp_free(&private->cp); goto err_out; } + private->state = VFIO_CCW_STATE_IDLE; return; } else if (scsw->cmd.fctl & SCSW_FCTL_HALT_FUNC) { VFIO_CCW_MSG_EVENT(2, @@ -331,6 +325,8 @@ static void fsm_io_request(struct vfio_ccw_private *private, err_out: trace_vfio_ccw_fsm_io_request(scsw->cmd.fctl, schid, io_region->ret_code, errstr); + + private->state = VFIO_CCW_STATE_IDLE; } /* @@ -405,10 +401,4 @@ fsm_func_t *vfio_ccw_jumptable[NR_VFIO_CCW_STATES][NR_VFIO_CCW_EVENTS] = { [VFIO_CCW_EVENT_ASYNC_REQ] = fsm_async_retry, [VFIO_CCW_EVENT_INTERRUPT] = fsm_irq, }, - [VFIO_CCW_STATE_CP_PENDING] = { - [VFIO_CCW_EVENT_NOT_OPER] = fsm_notoper, - [VFIO_CCW_EVENT_IO_REQ] = fsm_io_busy, - [VFIO_CCW_EVENT_ASYNC_REQ] = fsm_async_request, - [VFIO_CCW_EVENT_INTERRUPT] = fsm_irq, - }, }; diff --git a/drivers/s390/cio/vfio_ccw_ops.c b/drivers/s390/cio/vfio_ccw_ops.c index 8b3ed5b45277..0b67c04decee 100644 --- a/drivers/s390/cio/vfio_ccw_ops.c +++ b/drivers/s390/cio/vfio_ccw_ops.c @@ -276,8 +276,7 @@ static ssize_t vfio_ccw_mdev_write_io_region(struct vfio_ccw_private *private, } vfio_ccw_fsm_event(private, VFIO_CCW_EVENT_IO_REQ); - if (region->ret_code != 0) - private->state = VFIO_CCW_STATE_IDLE; + ret = (region->ret_code != 0) ? region->ret_code : count; out_unlock: diff --git a/drivers/s390/cio/vfio_ccw_private.h b/drivers/s390/cio/vfio_ccw_private.h index 8723156b29ea..f592897f1930 100644 --- a/drivers/s390/cio/vfio_ccw_private.h +++ b/drivers/s390/cio/vfio_ccw_private.h @@ -125,7 +125,6 @@ enum vfio_ccw_state { VFIO_CCW_STATE_STANDBY, VFIO_CCW_STATE_IDLE, VFIO_CCW_STATE_CP_PROCESSING, - VFIO_CCW_STATE_CP_PENDING, /* last element! */ NR_VFIO_CCW_STATES }; From patchwork Tue Jun 16 19:50:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Farman X-Patchwork-Id: 11608555 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A012814DD for ; Tue, 16 Jun 2020 19:51:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 91592208B8 for ; Tue, 16 Jun 2020 19:51:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731064AbgFPTvC (ORCPT ); Tue, 16 Jun 2020 15:51:02 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:42192 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730183AbgFPTvB (ORCPT ); Tue, 16 Jun 2020 15:51:01 -0400 Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 05GJ7PqH145980; Tue, 16 Jun 2020 15:51:01 -0400 Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 31q23tc694-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 16 Jun 2020 15:51:01 -0400 Received: from m0098393.ppops.net (m0098393.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.36/8.16.0.36) with SMTP id 05GJmeTQ007358; Tue, 16 Jun 2020 15:51:01 -0400 Received: from ppma01fra.de.ibm.com (46.49.7a9f.ip4.static.sl-reverse.com [159.122.73.70]) by mx0a-001b2d01.pphosted.com with ESMTP id 31q23tc68g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 16 Jun 2020 15:51:00 -0400 Received: from pps.filterd (ppma01fra.de.ibm.com [127.0.0.1]) by ppma01fra.de.ibm.com (8.16.0.42/8.16.0.42) with SMTP id 05GJnqlZ016464; Tue, 16 Jun 2020 19:50:58 GMT Received: from b06cxnps3075.portsmouth.uk.ibm.com (d06relay10.portsmouth.uk.ibm.com [9.149.109.195]) by ppma01fra.de.ibm.com with ESMTP id 31mpe7jd8k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 16 Jun 2020 19:50:58 +0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps3075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 05GJoto942008594 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 16 Jun 2020 19:50:55 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5F64FA4057; Tue, 16 Jun 2020 19:50:55 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4CE08A4055; Tue, 16 Jun 2020 19:50:55 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTPS; Tue, 16 Jun 2020 19:50:55 +0000 (GMT) Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 4958) id EE7D7E02B1; Tue, 16 Jun 2020 21:50:54 +0200 (CEST) From: Eric Farman To: Cornelia Huck Cc: Jared Rossi , Halil Pasic , linux-s390@vger.kernel.org, kvm@vger.kernel.org, Eric Farman Subject: [RFC PATCH v3 3/3] vfio-ccw: Check workqueue before doing START Date: Tue, 16 Jun 2020 21:50:53 +0200 Message-Id: <20200616195053.99253-4-farman@linux.ibm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200616195053.99253-1-farman@linux.ibm.com> References: <20200616195053.99253-1-farman@linux.ibm.com> X-TM-AS-GCONF: 00 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216,18.0.687 definitions=2020-06-16_12:2020-06-16,2020-06-16 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 cotscore=-2147483648 impostorscore=0 mlxlogscore=476 adultscore=0 priorityscore=1501 clxscore=1015 lowpriorityscore=0 malwarescore=0 bulkscore=0 phishscore=0 spamscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2004280000 definitions=main-2006160134 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When an interrupt is received via the IRQ, the bulk of the work is stacked on a workqueue for later processing. Which means that concurrent START or a HALT/CLEAR operation (via the async_region) will race with this process and require some serialization. Once we have all our locks acquired, let's just look to see if we're in a window where the process has been started from the IRQ, but not yet picked up by vfio-ccw to clean up an I/O. If there is, mark the request as BUSY so it can be redriven after we have a chance to breathe. Signed-off-by: Eric Farman --- drivers/s390/cio/vfio_ccw_fsm.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/s390/cio/vfio_ccw_fsm.c b/drivers/s390/cio/vfio_ccw_fsm.c index f0952192480e..9dc5b4d549b3 100644 --- a/drivers/s390/cio/vfio_ccw_fsm.c +++ b/drivers/s390/cio/vfio_ccw_fsm.c @@ -28,6 +28,11 @@ static int fsm_io_helper(struct vfio_ccw_private *private) spin_lock_irqsave(sch->lock, flags); + if (work_pending(&private->io_work)) { + ret = -EBUSY; + goto out; + } + orb = cp_get_orb(&private->cp, (u32)(addr_t)sch, sch->lpm); if (!orb) { ret = -EIO;