From patchwork Wed Oct 7 20:54:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 11821837 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CD54217D5 for ; Wed, 7 Oct 2020 20:57:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A4E4020782 for ; Wed, 7 Oct 2020 20:57:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="0FrkmP3t" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728476AbgJGU5d (ORCPT ); Wed, 7 Oct 2020 16:57:33 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:49764 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726152AbgJGU5b (ORCPT ); Wed, 7 Oct 2020 16:57:31 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KoJjn029026; Wed, 7 Oct 2020 20:57:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=Tv8vJdJcHktH+HW6Vmwm6CHowylPwVzDpBg7/BJwSE0=; b=0FrkmP3taSh+2Ah9qtMeMtQRcyeYoi5Ds15ncs61GkRjGDWKU05aAlCerLQrA7NIezQV 43M8MyeJdZZbnyDYG7n1zdMIb+fO/SJiYL2nE9s/ypDMLhHBgA2ZByfBHOj04semiYRk p/3265qWyEy6hBB2edcG2Ss6V+qa4Bqapfbmry4HHfuUAJ/UJQg46PE0bmlyIwEy/S6n GvpuIAHgDgZQnf1+Q1dAnOrXvVN7ZRauMJMdrsF7TKI6SuUXlzzjD1xPCi0cYcZ7LH2N 4wc9yUvUmZRVekmzdLOra4x4kAnqp3BGIqloREl+OCZVgKg+6zAbBxldbrT4FuNLECPC Gg== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2130.oracle.com with ESMTP id 33xetb4eqf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 07 Oct 2020 20:57:19 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KpEsv029234; Wed, 7 Oct 2020 20:55:18 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userp3020.oracle.com with ESMTP id 33yyjhpubp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 07 Oct 2020 20:55:18 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 097KtHAt009045; Wed, 7 Oct 2020 20:55:17 GMT Received: from ol2.localdomain (/73.88.28.6) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Oct 2020 13:55:16 -0700 From: Mike Christie To: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, pbonzini@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Subject: [PATCH 01/16] vhost scsi: add lun parser helper Date: Wed, 7 Oct 2020 15:54:46 -0500 Message-Id: <1602104101-5592-2-git-send-email-michael.christie@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> References: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 bulkscore=0 phishscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 mlxscore=0 clxscore=1015 priorityscore=1501 adultscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 malwarescore=0 suspectscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org Move code to parse lun from req's lun_buf to helper, so tmf code can use it in the next patch. Signed-off-by: Mike Christie Reviewed-by: Paolo Bonzini --- drivers/vhost/scsi.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index b22adf0..0ea78d0 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -907,6 +907,11 @@ static void vhost_scsi_submission_work(struct work_struct *work) return ret; } +static u16 vhost_buf_to_lun(u8 *lun_buf) +{ + return ((lun_buf[2] << 8) | lun_buf[3]) & 0x3FFF; +} + static void vhost_scsi_handle_vq(struct vhost_scsi *vs, struct vhost_virtqueue *vq) { @@ -1045,12 +1050,12 @@ static void vhost_scsi_submission_work(struct work_struct *work) tag = vhost64_to_cpu(vq, v_req_pi.tag); task_attr = v_req_pi.task_attr; cdb = &v_req_pi.cdb[0]; - lun = ((v_req_pi.lun[2] << 8) | v_req_pi.lun[3]) & 0x3FFF; + lun = vhost_buf_to_lun(v_req_pi.lun); } else { tag = vhost64_to_cpu(vq, v_req.tag); task_attr = v_req.task_attr; cdb = &v_req.cdb[0]; - lun = ((v_req.lun[2] << 8) | v_req.lun[3]) & 0x3FFF; + lun = vhost_buf_to_lun(v_req.lun); } /* * Check that the received CDB size does not exceeded our From patchwork Wed Oct 7 20:54:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 11821825 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6D76A14D5 for ; Wed, 7 Oct 2020 20:57:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4455920782 for ; Wed, 7 Oct 2020 20:57:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="IZDJFJPC" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728300AbgJGU5X (ORCPT ); Wed, 7 Oct 2020 16:57:23 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:44652 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727863AbgJGU5X (ORCPT ); Wed, 7 Oct 2020 16:57:23 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KoHae038736; Wed, 7 Oct 2020 20:57:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=/KqkHB58Rj/XvdnkF/U+YxEU2g19hhmhXV5WSObxxMQ=; b=IZDJFJPC184HtWz93G2pIOi/+hEpDr/ld9HhwTLM7w3oK7AsofYtZF1qfhOVYP/6tClb +ufctZmHchTmiJu35/EUZpVeAcBYiVjdvGpN0vvB4kZy6zLowJixgjist/PGvb+dvCXw NjR+VKkEZoJAgtp3cWAi4T0TiCm98WGxO9X40Bar2gZRpWRbno103sIf9wk8wvwL2STf rzGuUkyw3AAfbJVDWBXiJLyeSOsL8yZdPmFYFRBfy2WjxiJge8H3emy1st2XSZ2/Y9VV 0sXxuL0oF07PBqrDaVM8b/Vx2zRRBBBF6j4siLCcm1p6MDEMvOUKN/J+DEg/mzpLCyUC FA== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2120.oracle.com with ESMTP id 33xhxn47d4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 07 Oct 2020 20:57:19 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KosuO125736; Wed, 7 Oct 2020 20:55:18 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserp3020.oracle.com with ESMTP id 3410k03um8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 07 Oct 2020 20:55:18 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 097KtHjD009052; Wed, 7 Oct 2020 20:55:17 GMT Received: from ol2.localdomain (/73.88.28.6) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Oct 2020 13:55:17 -0700 From: Mike Christie To: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, pbonzini@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Subject: [PATCH 02/16] vhost: remove work arg from vhost_work_flush Date: Wed, 7 Oct 2020 15:54:47 -0500 Message-Id: <1602104101-5592-3-git-send-email-michael.christie@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> References: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 mlxlogscore=999 spamscore=0 adultscore=0 bulkscore=0 malwarescore=0 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 bulkscore=0 impostorscore=0 lowpriorityscore=0 suspectscore=0 phishscore=0 mlxlogscore=999 adultscore=0 clxscore=1015 spamscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org vhost_work_flush doesn't do anything with the work arg. This patch drops it and then renames vhost_work_flush to vhost_work_dev_flush to reflect that the function flushes all the works in the dev and not just a specific queue or work item. Signed-off-by: Mike Christie Acked-by: Jason Wang --- drivers/vhost/scsi.c | 4 ++-- drivers/vhost/vhost.c | 8 ++++---- drivers/vhost/vhost.h | 2 +- drivers/vhost/vsock.c | 2 +- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 0ea78d0..86617bb 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -1370,8 +1370,8 @@ static void vhost_scsi_flush(struct vhost_scsi *vs) /* Flush both the vhost poll and vhost work */ for (i = 0; i < VHOST_SCSI_MAX_VQ; i++) vhost_scsi_flush_vq(vs, i); - vhost_work_flush(&vs->dev, &vs->vs_completion_work); - vhost_work_flush(&vs->dev, &vs->vs_event_work); + vhost_work_dev_flush(&vs->dev); + vhost_work_dev_flush(&vs->dev); /* Wait for all reqs issued before the flush to be finished */ for (i = 0; i < VHOST_SCSI_MAX_VQ; i++) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 9ad45e1..fbb66f6 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -231,7 +231,7 @@ void vhost_poll_stop(struct vhost_poll *poll) } EXPORT_SYMBOL_GPL(vhost_poll_stop); -void vhost_work_flush(struct vhost_dev *dev, struct vhost_work *work) +void vhost_work_dev_flush(struct vhost_dev *dev) { struct vhost_flush_struct flush; @@ -243,13 +243,13 @@ void vhost_work_flush(struct vhost_dev *dev, struct vhost_work *work) wait_for_completion(&flush.wait_event); } } -EXPORT_SYMBOL_GPL(vhost_work_flush); +EXPORT_SYMBOL_GPL(vhost_work_dev_flush); /* Flush any work that has been scheduled. When calling this, don't hold any * locks that are also used by the callback. */ void vhost_poll_flush(struct vhost_poll *poll) { - vhost_work_flush(poll->dev, &poll->work); + vhost_work_dev_flush(poll->dev); } EXPORT_SYMBOL_GPL(vhost_poll_flush); @@ -533,7 +533,7 @@ static int vhost_attach_cgroups(struct vhost_dev *dev) attach.owner = current; vhost_work_init(&attach.work, vhost_attach_cgroups_work); vhost_work_queue(dev, &attach.work); - vhost_work_flush(dev, &attach.work); + vhost_work_dev_flush(dev); return attach.ret; } diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h index 9032d3c..11db183 100644 --- a/drivers/vhost/vhost.h +++ b/drivers/vhost/vhost.h @@ -46,7 +46,7 @@ void vhost_poll_init(struct vhost_poll *poll, vhost_work_fn_t fn, void vhost_poll_stop(struct vhost_poll *poll); void vhost_poll_flush(struct vhost_poll *poll); void vhost_poll_queue(struct vhost_poll *poll); -void vhost_work_flush(struct vhost_dev *dev, struct vhost_work *work); +void vhost_work_dev_flush(struct vhost_dev *dev); long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *argp); struct vhost_log { diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c index a483cec..f40205f 100644 --- a/drivers/vhost/vsock.c +++ b/drivers/vhost/vsock.c @@ -652,7 +652,7 @@ static void vhost_vsock_flush(struct vhost_vsock *vsock) for (i = 0; i < ARRAY_SIZE(vsock->vqs); i++) if (vsock->vqs[i].handle_kick) vhost_poll_flush(&vsock->vqs[i].poll); - vhost_work_flush(&vsock->dev, &vsock->send_pkt_work); + vhost_work_dev_flush(&vsock->dev); } static void vhost_vsock_reset_orphans(struct sock *sk) From patchwork Wed Oct 7 20:54:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 11821785 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6026E17D1 for ; Wed, 7 Oct 2020 20:55:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3481C2087D for ; Wed, 7 Oct 2020 20:55:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="qUshph/T" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728372AbgJGUzZ (ORCPT ); Wed, 7 Oct 2020 16:55:25 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:48200 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728183AbgJGUzY (ORCPT ); Wed, 7 Oct 2020 16:55:24 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KoCLM028894; Wed, 7 Oct 2020 20:55:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=Wi7RtYH0fZg2S+uVbL6yjJO6oNhjvPLtXETQITgty8o=; b=qUshph/T6NsCSv0NOMt76/iWVx3vmU/Uvi2AuPlglowG9Q4rv7xib8EjDjVapLJ3iI6E pgrilmZGDX/7EJlMyziUn4fKl/8eWo7XUX67h/54Lc9gQhq863GRSZpdeEQe+0PGhuey 0cwSe6p1jtSoe74C1p9Y3jEgFUN/baWVomIKkssxaRn00K6rqi48MGrX3XwDKwImaZax PiyL9+BY84Vj4bO04k0Ws8Z6yVTzL4bNCv+Lh2pVA3mmthq33iuCLzMPk+oWjbPp/ix3 TzYtpFZr6rgcGmsxXiCxEh1XhO5V/Ngd6FA3vuTpJ9oq4fUvBGdEhZgcYwGvaeabErf5 Lg== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2130.oracle.com with ESMTP id 33xetb4eet-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 07 Oct 2020 20:55:20 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KpD0S029005; Wed, 7 Oct 2020 20:55:19 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userp3020.oracle.com with ESMTP id 33yyjhpudb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 07 Oct 2020 20:55:19 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 097KtILK005628; Wed, 7 Oct 2020 20:55:18 GMT Received: from ol2.localdomain (/73.88.28.6) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Oct 2020 13:55:18 -0700 From: Mike Christie To: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, pbonzini@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Subject: [PATCH 03/16] vhost net: use goto error handling in open Date: Wed, 7 Oct 2020 15:54:48 -0500 Message-Id: <1602104101-5592-4-git-send-email-michael.christie@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> References: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 bulkscore=0 phishscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 mlxscore=0 clxscore=1015 priorityscore=1501 adultscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 malwarescore=0 suspectscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org In the next patches vhost_dev_init will be able to fail. This patch has vhost_net_open use goto error handling like is done in the other vhost code to make handling vhost_dev_init failures easier to handle and extend in the future. Signed-off-by: Mike Christie --- drivers/vhost/net.c | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-) diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index 531a00d..831d824 100644 --- a/drivers/vhost/net.c +++ b/drivers/vhost/net.c @@ -1286,27 +1286,18 @@ static int vhost_net_open(struct inode *inode, struct file *f) if (!n) return -ENOMEM; vqs = kmalloc_array(VHOST_NET_VQ_MAX, sizeof(*vqs), GFP_KERNEL); - if (!vqs) { - kvfree(n); - return -ENOMEM; - } + if (!vqs) + goto err_vqs; queue = kmalloc_array(VHOST_NET_BATCH, sizeof(void *), GFP_KERNEL); - if (!queue) { - kfree(vqs); - kvfree(n); - return -ENOMEM; - } + if (!queue) + goto err_queue; n->vqs[VHOST_NET_VQ_RX].rxq.queue = queue; xdp = kmalloc_array(VHOST_NET_BATCH, sizeof(*xdp), GFP_KERNEL); - if (!xdp) { - kfree(vqs); - kvfree(n); - kfree(queue); - return -ENOMEM; - } + if (!xdp) + goto err_xdp; n->vqs[VHOST_NET_VQ_TX].xdp = xdp; dev = &n->dev; @@ -1338,6 +1329,14 @@ static int vhost_net_open(struct inode *inode, struct file *f) n->refcnt_bias = 0; return 0; + +err_xdp: + kfree(queue); +err_queue: + kfree(vqs); +err_vqs: + kvfree(n); + return -ENOMEM; } static struct socket *vhost_net_stop_vq(struct vhost_net *n, From patchwork Wed Oct 7 20:54:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 11821827 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 293B414D5 for ; Wed, 7 Oct 2020 20:57:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 08C3120782 for ; Wed, 7 Oct 2020 20:57:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="W6kdl5eY" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728312AbgJGU5Z (ORCPT ); Wed, 7 Oct 2020 16:57:25 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:49684 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727863AbgJGU5Z (ORCPT ); Wed, 7 Oct 2020 16:57:25 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KoCTX028867; Wed, 7 Oct 2020 20:57:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=jl06B6+Iimn6YgraBzPh9332NSVn761yJXji70dh550=; b=W6kdl5eYh/hHR3In82q/7qXLqYb3mslg+XPNV9CxN7IccA3h48OS79fzLHvjUW9Qwii8 YWnMJv4+ZcgRTZ1LQJXmKRYxboDO8hZpgPaMUV2RbF8DIBRDMfXdbgDzk+ef9SSyHzg5 hU5kDJ+LNWGqguAeShtw6KDLxTe/RgdI3k2HfefuXdVIQC/6oYuTCroetW115N9Snnzw uYg2UbTG+lghNyyydfsu3XWJ71VMYhUks32ETiAdzTva0hxWqI650RlyThj10EQ+zrXL UOS56/Oy4DudJPxmSpdx8VUysxgjdbIGzHTV+S82SOHLRRE/FfX5zO7od7NbttI2LGY5 7A== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by aserp2130.oracle.com with ESMTP id 33xetb4eqh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 07 Oct 2020 20:57:21 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KorIP125445; Wed, 7 Oct 2020 20:55:20 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserp3020.oracle.com with ESMTP id 3410k03up8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 07 Oct 2020 20:55:20 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 097KtIWU021057; Wed, 7 Oct 2020 20:55:19 GMT Received: from ol2.localdomain (/73.88.28.6) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Oct 2020 13:55:18 -0700 From: Mike Christie To: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, pbonzini@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Subject: [PATCH 04/16] vhost: prep vhost_dev_init users to handle failures Date: Wed, 7 Oct 2020 15:54:49 -0500 Message-Id: <1602104101-5592-5-git-send-email-michael.christie@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> References: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 mlxlogscore=999 spamscore=0 adultscore=0 bulkscore=0 malwarescore=0 suspectscore=2 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 mlxscore=0 clxscore=1015 priorityscore=1501 adultscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 malwarescore=0 suspectscore=2 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org This is just a prep patch to get vhost_dev_init callers ready to handle the next patch where the function can fail. In this patch vhost_dev_init just returns 0, but I think it's easier to check for goto/error handling errors separated from the next patch. Signed-off-by: Mike Christie Reported-by: kernel test robot Reported-by: kernel test robot Reported-by: Dan Carpenter --- drivers/vhost/net.c | 11 +++++++---- drivers/vhost/scsi.c | 7 +++++-- drivers/vhost/test.c | 9 +++++++-- drivers/vhost/vdpa.c | 6 ++++-- drivers/vhost/vhost.c | 14 ++++++++------ drivers/vhost/vhost.h | 10 +++++----- drivers/vhost/vsock.c | 9 ++++++--- 7 files changed, 42 insertions(+), 24 deletions(-) diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index 831d824..fd30b53 100644 --- a/drivers/vhost/net.c +++ b/drivers/vhost/net.c @@ -1316,10 +1316,11 @@ static int vhost_net_open(struct inode *inode, struct file *f) n->vqs[i].rx_ring = NULL; vhost_net_buf_init(&n->vqs[i].rxq); } - vhost_dev_init(dev, vqs, VHOST_NET_VQ_MAX, - UIO_MAXIOV + VHOST_NET_BATCH, - VHOST_NET_PKT_WEIGHT, VHOST_NET_WEIGHT, true, - NULL); + if (vhost_dev_init(dev, vqs, VHOST_NET_VQ_MAX, + UIO_MAXIOV + VHOST_NET_BATCH, + VHOST_NET_PKT_WEIGHT, VHOST_NET_WEIGHT, true, + NULL)) + goto err_dev_init; vhost_poll_init(n->poll + VHOST_NET_VQ_TX, handle_tx_net, EPOLLOUT, dev); vhost_poll_init(n->poll + VHOST_NET_VQ_RX, handle_rx_net, EPOLLIN, dev); @@ -1330,6 +1331,8 @@ static int vhost_net_open(struct inode *inode, struct file *f) return 0; +err_dev_init: + kfree(xdp); err_xdp: kfree(queue); err_queue: diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 86617bb..63ba363 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -1632,14 +1632,17 @@ static int vhost_scsi_open(struct inode *inode, struct file *f) vqs[i] = &vs->vqs[i].vq; vs->vqs[i].vq.handle_kick = vhost_scsi_handle_kick; } - vhost_dev_init(&vs->dev, vqs, VHOST_SCSI_MAX_VQ, UIO_MAXIOV, - VHOST_SCSI_WEIGHT, 0, true, NULL); + if (vhost_dev_init(&vs->dev, vqs, VHOST_SCSI_MAX_VQ, UIO_MAXIOV, + VHOST_SCSI_WEIGHT, 0, true, NULL)) + goto err_dev_init; vhost_scsi_init_inflight(vs, NULL); f->private_data = vs; return 0; +err_dev_init: + kfree(vqs); err_vqs: kvfree(vs); err_vs: diff --git a/drivers/vhost/test.c b/drivers/vhost/test.c index a09dedc..c255ae5 100644 --- a/drivers/vhost/test.c +++ b/drivers/vhost/test.c @@ -119,12 +119,17 @@ static int vhost_test_open(struct inode *inode, struct file *f) dev = &n->dev; vqs[VHOST_TEST_VQ] = &n->vqs[VHOST_TEST_VQ]; n->vqs[VHOST_TEST_VQ].handle_kick = handle_vq_kick; - vhost_dev_init(dev, vqs, VHOST_TEST_VQ_MAX, UIO_MAXIOV, - VHOST_TEST_PKT_WEIGHT, VHOST_TEST_WEIGHT, true, NULL); + if (vhost_dev_init(dev, vqs, VHOST_TEST_VQ_MAX, UIO_MAXIOV, + VHOST_TEST_PKT_WEIGHT, VHOST_TEST_WEIGHT, true, NULL) + goto err_dev_init; f->private_data = n; return 0; + +err_dev_init: + kfree(vqs); + return -ENOMEM; } static void *vhost_test_stop_vq(struct vhost_test *n, diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index 62a9bb0..d413ceb 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -817,8 +817,9 @@ static int vhost_vdpa_open(struct inode *inode, struct file *filep) vqs[i] = &v->vqs[i]; vqs[i]->handle_kick = handle_vq_kick; } - vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false, - vhost_vdpa_process_iotlb_msg); + if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false, + vhost_vdpa_process_iotlb_msg)) + goto err_dev_init; dev->iotlb = vhost_iotlb_alloc(0, 0); if (!dev->iotlb) { @@ -836,6 +837,7 @@ static int vhost_vdpa_open(struct inode *inode, struct file *filep) err_init_iotlb: vhost_dev_cleanup(&v->vdev); +err_dev_init: kfree(vqs); err: atomic_dec(&v->opened); diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index fbb66f6..b05e690 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -459,12 +459,12 @@ static size_t vhost_get_desc_size(struct vhost_virtqueue *vq, return sizeof(*vq->desc) * num; } -void vhost_dev_init(struct vhost_dev *dev, - struct vhost_virtqueue **vqs, int nvqs, - int iov_limit, int weight, int byte_weight, - bool use_worker, - int (*msg_handler)(struct vhost_dev *dev, - struct vhost_iotlb_msg *msg)) +int vhost_dev_init(struct vhost_dev *dev, + struct vhost_virtqueue **vqs, int nvqs, + int iov_limit, int weight, int byte_weight, + bool use_worker, + int (*msg_handler)(struct vhost_dev *dev, + struct vhost_iotlb_msg *msg)) { struct vhost_virtqueue *vq; int i; @@ -501,6 +501,8 @@ void vhost_dev_init(struct vhost_dev *dev, vhost_poll_init(&vq->poll, vq->handle_kick, EPOLLIN, dev); } + + return 0; } EXPORT_SYMBOL_GPL(vhost_dev_init); diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h index 11db183..a053318 100644 --- a/drivers/vhost/vhost.h +++ b/drivers/vhost/vhost.h @@ -167,11 +167,11 @@ struct vhost_dev { }; bool vhost_exceeds_weight(struct vhost_virtqueue *vq, int pkts, int total_len); -void vhost_dev_init(struct vhost_dev *, struct vhost_virtqueue **vqs, - int nvqs, int iov_limit, int weight, int byte_weight, - bool use_worker, - int (*msg_handler)(struct vhost_dev *dev, - struct vhost_iotlb_msg *msg)); +int vhost_dev_init(struct vhost_dev *dev, struct vhost_virtqueue **vqs, + int nvqs, int iov_limit, int weight, int byte_weight, + bool use_worker, + int (*msg_handler)(struct vhost_dev *dev, + struct vhost_iotlb_msg *msg)); long vhost_dev_set_owner(struct vhost_dev *dev); bool vhost_dev_has_owner(struct vhost_dev *dev); long vhost_dev_check_owner(struct vhost_dev *); diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c index f40205f..a1a35e1 100644 --- a/drivers/vhost/vsock.c +++ b/drivers/vhost/vsock.c @@ -630,9 +630,10 @@ static int vhost_vsock_dev_open(struct inode *inode, struct file *file) vsock->vqs[VSOCK_VQ_TX].handle_kick = vhost_vsock_handle_tx_kick; vsock->vqs[VSOCK_VQ_RX].handle_kick = vhost_vsock_handle_rx_kick; - vhost_dev_init(&vsock->dev, vqs, ARRAY_SIZE(vsock->vqs), - UIO_MAXIOV, VHOST_VSOCK_PKT_WEIGHT, - VHOST_VSOCK_WEIGHT, true, NULL); + if (vhost_dev_init(&vsock->dev, vqs, ARRAY_SIZE(vsock->vqs), + UIO_MAXIOV, VHOST_VSOCK_PKT_WEIGHT, + VHOST_VSOCK_WEIGHT, true, NULL)) + goto err_dev_init; file->private_data = vsock; spin_lock_init(&vsock->send_pkt_list_lock); @@ -640,6 +641,8 @@ static int vhost_vsock_dev_open(struct inode *inode, struct file *file) vhost_work_init(&vsock->send_pkt_work, vhost_transport_send_pkt_work); return 0; +err_dev_init: + kfree(vqs); out: vhost_vsock_free(vsock); return ret; From patchwork Wed Oct 7 20:54:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 11821779 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EFFC514D5 for ; Wed, 7 Oct 2020 20:55:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CB9A42083B for ; Wed, 7 Oct 2020 20:55:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Jruyjena" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728562AbgJGUzZ (ORCPT ); Wed, 7 Oct 2020 16:55:25 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:48204 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728227AbgJGUzY (ORCPT ); Wed, 7 Oct 2020 16:55:24 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KoWp5029087; Wed, 7 Oct 2020 20:55:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=poLcKZ6/BP8EW9T+WZC/daa9IF1Bf81Hb8L+IgoSLWU=; b=JruyjenaJlTeyTb4ihfxSFY2odN26N8JVQPf0gac9YjTJdtP/6/d/bTnMwIafL/DrFP4 vDPA+VXS9cPMtK0KD0aZWMPnRbEBx3FhCT3wpSrOyEO/ki9YGZ2UbJkqQuxCaEB9ky6g NXVL1vygMLW8E4mxtqc5lWeOJyOY3NzjA0VJZRjiv0iyQdrmYqFD5vRJuiEqOYl75Qqr +FvGlFOIpiDJXJTzMcodwh3OTslIAsKhEVQ8VuW+k/odSGm6S31u6XgJiewD1HRR67aW cZsYPw/Eelbs0+u0P9gavYPWBp0fjmHUGLmE5PKjJ9E9P43CZ0MJ0srhWwhoLmb1p06v Ow== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2130.oracle.com with ESMTP id 33xetb4eeu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 07 Oct 2020 20:55:20 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KpKud166290; Wed, 7 Oct 2020 20:55:20 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserp3030.oracle.com with ESMTP id 33y2vpyuwc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 07 Oct 2020 20:55:20 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 097KtJUJ009063; Wed, 7 Oct 2020 20:55:19 GMT Received: from ol2.localdomain (/73.88.28.6) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Oct 2020 13:55:19 -0700 From: Mike Christie To: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, pbonzini@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Subject: [PATCH 05/16] vhost: move vq iovec allocation to dev init time Date: Wed, 7 Oct 2020 15:54:50 -0500 Message-Id: <1602104101-5592-6-git-send-email-michael.christie@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> References: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 malwarescore=0 suspectscore=2 spamscore=0 phishscore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 mlxscore=0 clxscore=1015 priorityscore=1501 adultscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 malwarescore=0 suspectscore=2 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org The next patches allow us to create vqs on demand after vhost_dev_init and vhost_dev_set_owner have been called. For vhost-scsi we don't know the number of vqs we really want until the vring/vq setup operations have started up. For other devices we know the number of vqs at vhost_dev_init time, so for those devs we init the vq and allocate the needed iovecs. For vhost-scsi we will do it later when userspace has instructed us to setup a specific vq. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 71 +++++++++++++++++++++++++++------------------------ 1 file changed, 38 insertions(+), 33 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index b05e690..d6af5a4 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -384,29 +384,27 @@ static void vhost_vq_free_iovecs(struct vhost_virtqueue *vq) vq->heads = NULL; } -/* Helper to allocate iovec buffers for all vqs. */ -static long vhost_dev_alloc_iovecs(struct vhost_dev *dev) +static int vhost_vq_alloc_iovecs(struct vhost_dev *dev, + struct vhost_virtqueue *vq) { - struct vhost_virtqueue *vq; - int i; + vq->indirect = kmalloc_array(UIO_MAXIOV, sizeof(*vq->indirect), + GFP_KERNEL); + if (!vq->indirect) + return -ENOMEM; + + if (!dev->iov_limit) + return 0; + + vq->log = kmalloc_array(dev->iov_limit, sizeof(*vq->log), GFP_KERNEL); + vq->heads = kmalloc_array(dev->iov_limit, sizeof(*vq->heads), + GFP_KERNEL); + if (!vq->log || !vq->heads) + goto err_nomem; - for (i = 0; i < dev->nvqs; ++i) { - vq = dev->vqs[i]; - vq->indirect = kmalloc_array(UIO_MAXIOV, - sizeof(*vq->indirect), - GFP_KERNEL); - vq->log = kmalloc_array(dev->iov_limit, sizeof(*vq->log), - GFP_KERNEL); - vq->heads = kmalloc_array(dev->iov_limit, sizeof(*vq->heads), - GFP_KERNEL); - if (!vq->indirect || !vq->log || !vq->heads) - goto err_nomem; - } return 0; err_nomem: - for (; i >= 0; --i) - vhost_vq_free_iovecs(dev->vqs[i]); + vhost_vq_free_iovecs(vq); return -ENOMEM; } @@ -459,6 +457,21 @@ static size_t vhost_get_desc_size(struct vhost_virtqueue *vq, return sizeof(*vq->desc) * num; } +static int vhost_vq_init(struct vhost_dev *dev, struct vhost_virtqueue *vq) +{ + vq->log = NULL; + vq->indirect = NULL; + vq->heads = NULL; + vq->dev = dev; + mutex_init(&vq->mutex); + vhost_vq_reset(dev, vq); + + if (vq->handle_kick) + vhost_poll_init(&vq->poll, vq->handle_kick, EPOLLIN, dev); + + return vhost_vq_alloc_iovecs(dev, vq); +} + int vhost_dev_init(struct vhost_dev *dev, struct vhost_virtqueue **vqs, int nvqs, int iov_limit, int weight, int byte_weight, @@ -466,7 +479,6 @@ int vhost_dev_init(struct vhost_dev *dev, int (*msg_handler)(struct vhost_dev *dev, struct vhost_iotlb_msg *msg)) { - struct vhost_virtqueue *vq; int i; dev->vqs = vqs; @@ -490,19 +502,16 @@ int vhost_dev_init(struct vhost_dev *dev, for (i = 0; i < dev->nvqs; ++i) { - vq = dev->vqs[i]; - vq->log = NULL; - vq->indirect = NULL; - vq->heads = NULL; - vq->dev = dev; - mutex_init(&vq->mutex); - vhost_vq_reset(dev, vq); - if (vq->handle_kick) - vhost_poll_init(&vq->poll, vq->handle_kick, - EPOLLIN, dev); + if (vhost_vq_init(dev, dev->vqs[i])) + goto err_vq_init; } return 0; + +err_vq_init: + for (--i; i >= 0; --i) + vhost_vq_free_iovecs(dev->vqs[i]); + return -ENOMEM; } EXPORT_SYMBOL_GPL(vhost_dev_init); @@ -607,10 +616,6 @@ long vhost_dev_set_owner(struct vhost_dev *dev) goto err_cgroup; } - err = vhost_dev_alloc_iovecs(dev); - if (err) - goto err_cgroup; - return 0; err_cgroup: if (dev->worker) { From patchwork Wed Oct 7 20:54:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 11821791 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B9C736CA for ; Wed, 7 Oct 2020 20:55:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 91A012083B for ; Wed, 7 Oct 2020 20:55:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="hIMHGdof" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727828AbgJGUz3 (ORCPT ); Wed, 7 Oct 2020 16:55:29 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:43316 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727816AbgJGUz1 (ORCPT ); Wed, 7 Oct 2020 16:55:27 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097Knj6O038420; Wed, 7 Oct 2020 20:55:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=8f1iB9F7SEnv/Y0diwaXj6AGiOaAb9Dd4CzmiVH4x3c=; b=hIMHGdofsO5iSl23VVzTsN4yLZwCjGdB4pQLSR6+dQIr71DbYm4b6pNz1XsCjEAm+yar I4hjCK1803WW092B1gVQ1zNoz2jKnKieJlWtFvJxm1ijxqeSF2rz5yquwgW8rESujyEL k81P6DTodtRS87+ofbHcoLxr6U4nK2H35iv+wOMZaEjv5GhNHoV+bkGGcjAyf5Jd+Fnf GA0RaMaHjTuEoVAtOKtpjH7J+dRUCF4h54NjmVdV2KqlVcROlbdgi7eLYG3Nyn8qNzZb KN0MPiJHo9j97E5RLVGqraYJAsu4QfB+a+6EuqUQIacen6jAGlckp+bbtaNMmHX+/xk3 lQ== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by userp2120.oracle.com with ESMTP id 33xhxn475d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 07 Oct 2020 20:55:21 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KpL7c166351; Wed, 7 Oct 2020 20:55:20 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by aserp3030.oracle.com with ESMTP id 33y2vpyux3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 07 Oct 2020 20:55:20 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 097KtKe1009066; Wed, 7 Oct 2020 20:55:20 GMT Received: from ol2.localdomain (/73.88.28.6) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Oct 2020 13:55:19 -0700 From: Mike Christie To: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, pbonzini@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Subject: [PATCH 06/16] vhost: support delayed vq creation Date: Wed, 7 Oct 2020 15:54:51 -0500 Message-Id: <1602104101-5592-7-git-send-email-michael.christie@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> References: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 malwarescore=0 suspectscore=2 spamscore=0 phishscore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 bulkscore=0 impostorscore=0 lowpriorityscore=0 suspectscore=2 phishscore=0 mlxlogscore=999 adultscore=0 clxscore=1015 spamscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org This allows vq creation to be done when it's first accessed by userspace. vhost-scsi doesn't know how many queues the user requested until they are first setup, and we don't want to allocate resources like the iovecs for 128 vqs when we are only using 1 or 2 most of the time. In the next pathces, vhost-scsi will also switch to preallocating cmds per vq instead of per lio session and we don't want to allocate them for 127 extra vqs if they are not in use. With this patch when a driver calls vhost_dev_init they pass in the number of vqs that they know they need and the max they can support. This patch has all the drivers pass in the same value for both the initial number of vqs and the max. The next patch will convert scsi. The other drivers like net/vsock have their vqs hard coded in the kernel or setup/discovered via other methods like with vdpa. Signed-off-by: Mike Christie --- drivers/vhost/net.c | 2 +- drivers/vhost/scsi.c | 4 +-- drivers/vhost/test.c | 5 ++-- drivers/vhost/vdpa.c | 2 +- drivers/vhost/vhost.c | 71 ++++++++++++++++++++++++++++++++++----------------- drivers/vhost/vhost.h | 7 +++-- drivers/vhost/vsock.c | 11 ++++---- 7 files changed, 66 insertions(+), 36 deletions(-) diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index fd30b53..fce46f0 100644 --- a/drivers/vhost/net.c +++ b/drivers/vhost/net.c @@ -1316,7 +1316,7 @@ static int vhost_net_open(struct inode *inode, struct file *f) n->vqs[i].rx_ring = NULL; vhost_net_buf_init(&n->vqs[i].rxq); } - if (vhost_dev_init(dev, vqs, VHOST_NET_VQ_MAX, + if (vhost_dev_init(dev, vqs, VHOST_NET_VQ_MAX, VHOST_NET_VQ_MAX, UIO_MAXIOV + VHOST_NET_BATCH, VHOST_NET_PKT_WEIGHT, VHOST_NET_WEIGHT, true, NULL)) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 63ba363..5d412f1 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -1632,8 +1632,8 @@ static int vhost_scsi_open(struct inode *inode, struct file *f) vqs[i] = &vs->vqs[i].vq; vs->vqs[i].vq.handle_kick = vhost_scsi_handle_kick; } - if (vhost_dev_init(&vs->dev, vqs, VHOST_SCSI_MAX_VQ, UIO_MAXIOV, - VHOST_SCSI_WEIGHT, 0, true, NULL)) + if (vhost_dev_init(&vs->dev, vqs, VHOST_SCSI_MAX_VQ, VHOST_SCSI_MAX_VQ, + UIO_MAXIOV, VHOST_SCSI_WEIGHT, 0, true, NULL)) goto err_dev_init; vhost_scsi_init_inflight(vs, NULL); diff --git a/drivers/vhost/test.c b/drivers/vhost/test.c index c255ae5..9d2bfa3 100644 --- a/drivers/vhost/test.c +++ b/drivers/vhost/test.c @@ -119,8 +119,9 @@ static int vhost_test_open(struct inode *inode, struct file *f) dev = &n->dev; vqs[VHOST_TEST_VQ] = &n->vqs[VHOST_TEST_VQ]; n->vqs[VHOST_TEST_VQ].handle_kick = handle_vq_kick; - if (vhost_dev_init(dev, vqs, VHOST_TEST_VQ_MAX, UIO_MAXIOV, - VHOST_TEST_PKT_WEIGHT, VHOST_TEST_WEIGHT, true, NULL) + if (vhost_dev_init(dev, vqs, VHOST_TEST_VQ_MAX, VHOST_TEST_VQ_MAX, + UIO_MAXIOV, VHOST_TEST_PKT_WEIGHT, VHOST_TEST_WEIGHT, + true, NULL) goto err_dev_init; f->private_data = n; diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index d413ceb..9507ebb 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -817,7 +817,7 @@ static int vhost_vdpa_open(struct inode *inode, struct file *filep) vqs[i] = &v->vqs[i]; vqs[i]->handle_kick = handle_vq_kick; } - if (vhost_dev_init(dev, vqs, nvqs, 0, 0, 0, false, + if (vhost_dev_init(dev, vqs, nvqs, nvqs, 0, 0, 0, false, vhost_vdpa_process_iotlb_msg)) goto err_dev_init; diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index d6af5a4..1507400 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -294,7 +294,7 @@ static void vhost_vq_meta_reset(struct vhost_dev *d) { int i; - for (i = 0; i < d->nvqs; ++i) + for (i = 0; i < d->max_nvqs; ++i) __vhost_vq_meta_reset(d->vqs[i]); } @@ -332,6 +332,7 @@ static void vhost_vq_reset(struct vhost_dev *dev, vq->busyloop_timeout = 0; vq->umem = NULL; vq->iotlb = NULL; + vq->initialized = false; vhost_vring_call_reset(&vq->call_ctx); __vhost_vq_meta_reset(vq); } @@ -412,7 +413,7 @@ static void vhost_dev_free_iovecs(struct vhost_dev *dev) { int i; - for (i = 0; i < dev->nvqs; ++i) + for (i = 0; i < dev->max_nvqs; ++i) vhost_vq_free_iovecs(dev->vqs[i]); } @@ -457,7 +458,7 @@ static size_t vhost_get_desc_size(struct vhost_virtqueue *vq, return sizeof(*vq->desc) * num; } -static int vhost_vq_init(struct vhost_dev *dev, struct vhost_virtqueue *vq) +static void __vhost_vq_init(struct vhost_dev *dev, struct vhost_virtqueue *vq) { vq->log = NULL; vq->indirect = NULL; @@ -468,12 +469,29 @@ static int vhost_vq_init(struct vhost_dev *dev, struct vhost_virtqueue *vq) if (vq->handle_kick) vhost_poll_init(&vq->poll, vq->handle_kick, EPOLLIN, dev); +} + +static int vhost_vq_init(struct vhost_dev *dev, int vq_idx) +{ + struct vhost_virtqueue *vq; + int ret; + + if (vq_idx >= dev->max_nvqs) + return -ENOBUFS; + + vq = dev->vqs[vq_idx]; + __vhost_vq_init(dev, vq); + ret = vhost_vq_alloc_iovecs(dev, vq); + if (ret) + return ret; - return vhost_vq_alloc_iovecs(dev, vq); + vq->initialized = true; + dev->nvqs++; + return 0; } int vhost_dev_init(struct vhost_dev *dev, - struct vhost_virtqueue **vqs, int nvqs, + struct vhost_virtqueue **vqs, int nvqs, int max_nvqs, int iov_limit, int weight, int byte_weight, bool use_worker, int (*msg_handler)(struct vhost_dev *dev, @@ -482,7 +500,8 @@ int vhost_dev_init(struct vhost_dev *dev, int i; dev->vqs = vqs; - dev->nvqs = nvqs; + dev->nvqs = 0; + dev->max_nvqs = max_nvqs; mutex_init(&dev->mutex); dev->log_ctx = NULL; dev->umem = NULL; @@ -500,12 +519,15 @@ int vhost_dev_init(struct vhost_dev *dev, INIT_LIST_HEAD(&dev->pending_list); spin_lock_init(&dev->iotlb_lock); - - for (i = 0; i < dev->nvqs; ++i) { - if (vhost_vq_init(dev, dev->vqs[i])) + for (i = 0; i < nvqs; ++i) { + if (vhost_vq_init(dev, i)) goto err_vq_init; } + for (; i < dev->max_nvqs; ++i) + /* Just prep/clear the fields and set initialized=false */ + __vhost_vq_init(dev, dev->vqs[i]); + return 0; err_vq_init: @@ -653,7 +675,7 @@ void vhost_dev_reset_owner(struct vhost_dev *dev, struct vhost_iotlb *umem) /* We don't need VQ locks below since vhost_dev_cleanup makes sure * VQs aren't running. */ - for (i = 0; i < dev->nvqs; ++i) + for (i = 0; i < dev->max_nvqs; ++i) dev->vqs[i]->umem = umem; } EXPORT_SYMBOL_GPL(vhost_dev_reset_owner); @@ -662,7 +684,7 @@ void vhost_dev_stop(struct vhost_dev *dev) { int i; - for (i = 0; i < dev->nvqs; ++i) { + for (i = 0; i < dev->max_nvqs; ++i) { if (dev->vqs[i]->kick && dev->vqs[i]->handle_kick) { vhost_poll_stop(&dev->vqs[i]->poll); vhost_poll_flush(&dev->vqs[i]->poll); @@ -694,7 +716,7 @@ void vhost_dev_cleanup(struct vhost_dev *dev) { int i; - for (i = 0; i < dev->nvqs; ++i) { + for (i = 0; i < dev->max_nvqs; ++i) { if (dev->vqs[i]->error_ctx) eventfd_ctx_put(dev->vqs[i]->error_ctx); if (dev->vqs[i]->kick) @@ -788,7 +810,7 @@ static bool memory_access_ok(struct vhost_dev *d, struct vhost_iotlb *umem, { int i; - for (i = 0; i < d->nvqs; ++i) { + for (i = 0; i < d->max_nvqs; ++i) { bool ok; bool log; @@ -1000,14 +1022,14 @@ static inline int vhost_put_used_idx(struct vhost_virtqueue *vq) static void vhost_dev_lock_vqs(struct vhost_dev *d) { int i = 0; - for (i = 0; i < d->nvqs; ++i) + for (i = 0; i < d->max_nvqs; ++i) mutex_lock_nested(&d->vqs[i]->mutex, i); } static void vhost_dev_unlock_vqs(struct vhost_dev *d) { int i = 0; - for (i = 0; i < d->nvqs; ++i) + for (i = 0; i < d->max_nvqs; ++i) mutex_unlock(&d->vqs[i]->mutex); } @@ -1463,7 +1485,7 @@ static long vhost_set_memory(struct vhost_dev *d, struct vhost_memory __user *m) d->umem = newumem; /* All memory accesses are done under some VQ mutex. */ - for (i = 0; i < d->nvqs; ++i) { + for (i = 0; i < d->max_nvqs; ++i) { mutex_lock(&d->vqs[i]->mutex); d->vqs[i]->umem = newumem; mutex_unlock(&d->vqs[i]->mutex); @@ -1591,11 +1613,14 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg r = get_user(idx, idxp); if (r < 0) return r; - if (idx >= d->nvqs) - return -ENOBUFS; - idx = array_index_nospec(idx, d->nvqs); + idx = array_index_nospec(idx, d->max_nvqs); vq = d->vqs[idx]; + if (!vq->initialized) { + r = vhost_vq_init(d, idx); + if (r) + return r; + } if (ioctl == VHOST_SET_VRING_NUM || ioctl == VHOST_SET_VRING_ADDR) { @@ -1727,7 +1752,7 @@ int vhost_init_device_iotlb(struct vhost_dev *d, bool enabled) oiotlb = d->iotlb; d->iotlb = niotlb; - for (i = 0; i < d->nvqs; ++i) { + for (i = 0; i < d->max_nvqs; ++i) { struct vhost_virtqueue *vq = d->vqs[i]; mutex_lock(&vq->mutex); @@ -1774,7 +1799,7 @@ long vhost_dev_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *argp) r = -EFAULT; break; } - for (i = 0; i < d->nvqs; ++i) { + for (i = 0; i < d->max_nvqs; ++i) { struct vhost_virtqueue *vq; void __user *base = (void __user *)(unsigned long)p; vq = d->vqs[i]; @@ -1797,7 +1822,7 @@ long vhost_dev_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *argp) break; } swap(ctx, d->log_ctx); - for (i = 0; i < d->nvqs; ++i) { + for (i = 0; i < d->max_nvqs; ++i) { mutex_lock(&d->vqs[i]->mutex); d->vqs[i]->log_ctx = d->log_ctx; mutex_unlock(&d->vqs[i]->mutex); @@ -2612,7 +2637,7 @@ void vhost_set_backend_features(struct vhost_dev *dev, u64 features) int i; mutex_lock(&dev->mutex); - for (i = 0; i < dev->nvqs; ++i) { + for (i = 0; i < dev->max_nvqs; ++i) { vq = dev->vqs[i]; mutex_lock(&vq->mutex); vq->acked_backend_features = features; diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h index a053318..2c6473e 100644 --- a/drivers/vhost/vhost.h +++ b/drivers/vhost/vhost.h @@ -132,6 +132,8 @@ struct vhost_virtqueue { bool user_be; #endif u32 busyloop_timeout; + + bool initialized; }; struct vhost_msg_node { @@ -148,6 +150,7 @@ struct vhost_dev { struct mutex mutex; struct vhost_virtqueue **vqs; int nvqs; + int max_nvqs; struct eventfd_ctx *log_ctx; struct llist_head work_list; struct task_struct *worker; @@ -168,8 +171,8 @@ struct vhost_dev { bool vhost_exceeds_weight(struct vhost_virtqueue *vq, int pkts, int total_len); int vhost_dev_init(struct vhost_dev *dev, struct vhost_virtqueue **vqs, - int nvqs, int iov_limit, int weight, int byte_weight, - bool use_worker, + int nvqs, int max_nvqs, int iov_limit, int weight, + int byte_weight, bool use_worker, int (*msg_handler)(struct vhost_dev *dev, struct vhost_iotlb_msg *msg)); long vhost_dev_set_owner(struct vhost_dev *dev); diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c index a1a35e1..9200868 100644 --- a/drivers/vhost/vsock.c +++ b/drivers/vhost/vsock.c @@ -606,7 +606,7 @@ static int vhost_vsock_dev_open(struct inode *inode, struct file *file) { struct vhost_virtqueue **vqs; struct vhost_vsock *vsock; - int ret; + int ret, nvqs; /* This struct is large and allocation could fail, fall back to vmalloc * if there is no other way. @@ -615,7 +615,8 @@ static int vhost_vsock_dev_open(struct inode *inode, struct file *file) if (!vsock) return -ENOMEM; - vqs = kmalloc_array(ARRAY_SIZE(vsock->vqs), sizeof(*vqs), GFP_KERNEL); + nvqs = ARRAY_SIZE(vsock->vqs); + vqs = kmalloc_array(nvqs, sizeof(*vqs), GFP_KERNEL); if (!vqs) { ret = -ENOMEM; goto out; @@ -630,9 +631,9 @@ static int vhost_vsock_dev_open(struct inode *inode, struct file *file) vsock->vqs[VSOCK_VQ_TX].handle_kick = vhost_vsock_handle_tx_kick; vsock->vqs[VSOCK_VQ_RX].handle_kick = vhost_vsock_handle_rx_kick; - if (vhost_dev_init(&vsock->dev, vqs, ARRAY_SIZE(vsock->vqs), - UIO_MAXIOV, VHOST_VSOCK_PKT_WEIGHT, - VHOST_VSOCK_WEIGHT, true, NULL)) + if (vhost_dev_init(&vsock->dev, vqs, nvqs, nvqs, UIO_MAXIOV, + VHOST_VSOCK_PKT_WEIGHT, VHOST_VSOCK_WEIGHT, true, + NULL)) goto err_dev_init; file->private_data = vsock; From patchwork Wed Oct 7 20:54:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 11821789 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6963217D1 for ; Wed, 7 Oct 2020 20:55:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 49D942083B for ; Wed, 7 Oct 2020 20:55:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Yrf0AQm+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728808AbgJGUz2 (ORCPT ); Wed, 7 Oct 2020 16:55:28 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:43320 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728789AbgJGUz0 (ORCPT ); Wed, 7 Oct 2020 16:55:26 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KnSF6038252; Wed, 7 Oct 2020 20:55:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=PoAgr+goPsYoyHGUHDIojY7cz7/Vh87XeLNAz04rgyI=; b=Yrf0AQm+BGYxk4yyJFDTU4MwqG3c2nP2qPhYmV8avMClxF53FopATqC80KnIJKnvtdJa uMudyj6+Q1ZD/DKaH/PMS044N24BuhM3KMWuUPVuHMq+wVqTdZvfvZx/lF9gmwzvXtax OWOocPZJmkJyjJiwifu4pcKKO+43in2EReyIc7DIeYJoN0tczyFmXh+2z3f31K2EWnhA I1+T582Tja9mH8u9DIaNrB6Pm4FF9glus09V+vDUmKE2ly3oU+eZtUzeBgou+LLKJWL0 0XF1jfXoj2t+QBg3Yl8KCvbP8GtHwmp74ktHbRY8nV5ct0isFWK9nk80hwFy5wI29kP6 Vw== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by userp2120.oracle.com with ESMTP id 33xhxn475g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 07 Oct 2020 20:55:22 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KpMjn166497; Wed, 7 Oct 2020 20:55:21 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserp3030.oracle.com with ESMTP id 33y2vpyuxq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 07 Oct 2020 20:55:21 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 097KtKcb005642; Wed, 7 Oct 2020 20:55:20 GMT Received: from ol2.localdomain (/73.88.28.6) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Oct 2020 13:55:20 -0700 From: Mike Christie To: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, pbonzini@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Subject: [PATCH 07/16] vhost scsi: support delayed IO vq creation Date: Wed, 7 Oct 2020 15:54:52 -0500 Message-Id: <1602104101-5592-8-git-send-email-michael.christie@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> References: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 malwarescore=0 suspectscore=0 spamscore=0 phishscore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 bulkscore=0 impostorscore=0 lowpriorityscore=0 suspectscore=0 phishscore=0 mlxlogscore=999 adultscore=0 clxscore=1015 spamscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org Each vhost-scsi device will need a evt and ctl queue, but the number of IO queues depends on whatever the user has configured in userspace. This patch has vhost-scsi create the evt, ctl and one IO vq at device open time. We then create the other IO vqs when userspace starts to set them up. We still waste some mem on the vq and scsi vq structs, but we don't waste mem on iovec related arrays and for later patches we know which queues are used by the dev->nvqs value. Signed-off-by: Mike Christie --- drivers/vhost/scsi.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 5d412f1..ab1b656 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -1401,7 +1401,7 @@ static void vhost_scsi_flush(struct vhost_scsi *vs) mutex_lock(&vs->dev.mutex); /* Verify that ring has been setup correctly. */ - for (index = 0; index < vs->dev.nvqs; ++index) { + for (index = 0; index < vs->dev.max_nvqs; ++index) { /* Verify that ring has been setup correctly. */ if (!vhost_vq_access_ok(&vs->vqs[index].vq)) { ret = -EFAULT; @@ -1464,6 +1464,9 @@ static void vhost_scsi_flush(struct vhost_scsi *vs) sizeof(vs->vs_vhost_wwpn)); for (i = 0; i < VHOST_SCSI_MAX_VQ; i++) { vq = &vs->vqs[i].vq; + if (!vq->initialized) + continue; + mutex_lock(&vq->mutex); vhost_vq_set_backend(vq, vs_tpg); vhost_vq_init_access(vq); @@ -1503,7 +1506,7 @@ static void vhost_scsi_flush(struct vhost_scsi *vs) mutex_lock(&vhost_scsi_mutex); mutex_lock(&vs->dev.mutex); /* Verify that ring has been setup correctly. */ - for (index = 0; index < vs->dev.nvqs; ++index) { + for (index = 0; index < vs->dev.max_nvqs; ++index) { if (!vhost_vq_access_ok(&vs->vqs[index].vq)) { ret = -EFAULT; goto err_dev; @@ -1551,6 +1554,9 @@ static void vhost_scsi_flush(struct vhost_scsi *vs) if (match) { for (i = 0; i < VHOST_SCSI_MAX_VQ; i++) { vq = &vs->vqs[i].vq; + if (!vq->initialized) + continue; + mutex_lock(&vq->mutex); vhost_vq_set_backend(vq, NULL); mutex_unlock(&vq->mutex); @@ -1632,8 +1638,13 @@ static int vhost_scsi_open(struct inode *inode, struct file *f) vqs[i] = &vs->vqs[i].vq; vs->vqs[i].vq.handle_kick = vhost_scsi_handle_kick; } - if (vhost_dev_init(&vs->dev, vqs, VHOST_SCSI_MAX_VQ, VHOST_SCSI_MAX_VQ, - UIO_MAXIOV, VHOST_SCSI_WEIGHT, 0, true, NULL)) + + /* + * We will always need the ctl, evt and at least 1 IO vq. Create more + * IO vqs if userspace requests them. + */ + if (vhost_dev_init(&vs->dev, vqs, 3, VHOST_SCSI_MAX_VQ, UIO_MAXIOV, + VHOST_SCSI_WEIGHT, 0, true, NULL)) goto err_dev_init; vhost_scsi_init_inflight(vs, NULL); From patchwork Wed Oct 7 20:54:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 11821803 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2360E17D5 for ; Wed, 7 Oct 2020 20:55:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id F2E912087D for ; Wed, 7 Oct 2020 20:55:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="iNFSF82Y" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728822AbgJGUze (ORCPT ); Wed, 7 Oct 2020 16:55:34 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:48248 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728798AbgJGUza (ORCPT ); Wed, 7 Oct 2020 16:55:30 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KoBsP028846; Wed, 7 Oct 2020 20:55:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=PwNFczbC8s1H1iexbFxiXSr3V+MYluFMMJIQZWiW7mg=; b=iNFSF82YemKWysNav0QFxe7TOrXtLeq18dmZrGrVzgIlFVacewuCfAhH/pu+bpuVTiqt n5CQqMKD3FWB/aBt3KJEbYGOSrWYAcFBHEwQ6qG2IWit6PZw7YPG5b3rzsRoia3migak 8b1KMSZurN9VGLM7aSyounBBGRYmI01k7EB4Lr6Tlq9VbY0Zqya2JnR9j5k1r499c81b 3W4ZIBdKPjzaJytxLUeWxEKbtTpm8Ed1QnUiHaMQDFO8lKxbJgADrcMV/mYv06U/PyJf n52OAT/dppyARWatzb0FEYQRghYtKWmGhN5/baIT2Id3kQvq/SljnG+E6FGfe+VhVtto SQ== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2130.oracle.com with ESMTP id 33xetb4ef3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 07 Oct 2020 20:55:23 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KpL2l166319; Wed, 7 Oct 2020 20:55:23 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserp3030.oracle.com with ESMTP id 33y2vpyv08-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 07 Oct 2020 20:55:22 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 097KtLNu032109; Wed, 7 Oct 2020 20:55:21 GMT Received: from ol2.localdomain (/73.88.28.6) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Oct 2020 13:55:21 -0700 From: Mike Christie To: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, pbonzini@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Subject: [PATCH 08/16] vhost scsi: alloc cmds per vq instead of session Date: Wed, 7 Oct 2020 15:54:53 -0500 Message-Id: <1602104101-5592-9-git-send-email-michael.christie@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> References: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 malwarescore=0 suspectscore=2 spamscore=0 phishscore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 mlxscore=0 clxscore=1015 priorityscore=1501 adultscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 malwarescore=0 suspectscore=2 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org We currently are limited to 256 cmds per session. This leads to problems where if the user has increased virtqueue_size to more than 2 or cmd_per_lun to more than 256 vhost_scsi_get_tag can fail and the guest will get IO errors. This patch moves the cmd allocation to per vq so we can easily match whatever the user has specified for num_queues and virtqueue_size/cmd_per_lun. It also makes it easier to control how much memory we preallocate. For cases, where perf is not as important and we can use the current defaults (1 vq and 128 cmds per vq) memory use from preallocate cmds is cut in half. For cases, where we are willing to use more memory for higher perf, cmd mem use will now increase as the num queues and queue depth increases. Signed-off-by: Mike Christie --- V2 - fix use before set cpu var errors - drop vhost_vq_is_setup drivers/vhost/scsi.c | 207 +++++++++++++++++++++++++++++++-------------------- 1 file changed, 128 insertions(+), 79 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index ab1b656..f6b9010 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -52,7 +52,6 @@ #define VHOST_SCSI_VERSION "v0.1" #define VHOST_SCSI_NAMELEN 256 #define VHOST_SCSI_MAX_CDB_SIZE 32 -#define VHOST_SCSI_DEFAULT_TAGS 256 #define VHOST_SCSI_PREALLOC_SGLS 2048 #define VHOST_SCSI_PREALLOC_UPAGES 2048 #define VHOST_SCSI_PREALLOC_PROT_SGLS 2048 @@ -189,6 +188,9 @@ struct vhost_scsi_virtqueue { * Writers must also take dev mutex and flush under it. */ int inflight_idx; + struct vhost_scsi_cmd *scsi_cmds; + struct sbitmap scsi_tags; + int max_cmds; }; struct vhost_scsi { @@ -324,7 +326,9 @@ static void vhost_scsi_release_cmd(struct se_cmd *se_cmd) { struct vhost_scsi_cmd *tv_cmd = container_of(se_cmd, struct vhost_scsi_cmd, tvc_se_cmd); - struct se_session *se_sess = tv_cmd->tvc_nexus->tvn_se_sess; + struct vhost_scsi_virtqueue *svq = container_of(tv_cmd->tvc_vq, + struct vhost_scsi_virtqueue, vq); + struct vhost_scsi_inflight *inflight = tv_cmd->inflight; int i; if (tv_cmd->tvc_sgl_count) { @@ -336,8 +340,8 @@ static void vhost_scsi_release_cmd(struct se_cmd *se_cmd) put_page(sg_page(&tv_cmd->tvc_prot_sgl[i])); } - vhost_scsi_put_inflight(tv_cmd->inflight); - target_free_tag(se_sess, se_cmd); + sbitmap_clear_bit(&svq->scsi_tags, se_cmd->map_tag); + vhost_scsi_put_inflight(inflight); } static u32 vhost_scsi_sess_get_index(struct se_session *se_sess) @@ -566,31 +570,31 @@ static void vhost_scsi_complete_cmd_work(struct vhost_work *work) } static struct vhost_scsi_cmd * -vhost_scsi_get_tag(struct vhost_virtqueue *vq, struct vhost_scsi_tpg *tpg, +vhost_scsi_get_cmd(struct vhost_virtqueue *vq, struct vhost_scsi_tpg *tpg, unsigned char *cdb, u64 scsi_tag, u16 lun, u8 task_attr, u32 exp_data_len, int data_direction) { + struct vhost_scsi_virtqueue *svq = container_of(vq, + struct vhost_scsi_virtqueue, vq); struct vhost_scsi_cmd *cmd; struct vhost_scsi_nexus *tv_nexus; - struct se_session *se_sess; struct scatterlist *sg, *prot_sg; struct page **pages; - int tag, cpu; + int tag; tv_nexus = tpg->tpg_nexus; if (!tv_nexus) { pr_err("Unable to locate active struct vhost_scsi_nexus\n"); return ERR_PTR(-EIO); } - se_sess = tv_nexus->tvn_se_sess; - tag = sbitmap_queue_get(&se_sess->sess_tag_pool, &cpu); + tag = sbitmap_get(&svq->scsi_tags, 0, false); if (tag < 0) { pr_err("Unable to obtain tag for vhost_scsi_cmd\n"); return ERR_PTR(-ENOMEM); } - cmd = &((struct vhost_scsi_cmd *)se_sess->sess_cmd_map)[tag]; + cmd = &svq->scsi_cmds[tag]; sg = cmd->tvc_sgl; prot_sg = cmd->tvc_prot_sgl; pages = cmd->tvc_upages; @@ -599,7 +603,6 @@ static void vhost_scsi_complete_cmd_work(struct vhost_work *work) cmd->tvc_prot_sgl = prot_sg; cmd->tvc_upages = pages; cmd->tvc_se_cmd.map_tag = tag; - cmd->tvc_se_cmd.map_cpu = cpu; cmd->tvc_tag = scsi_tag; cmd->tvc_lun = lun; cmd->tvc_task_attr = task_attr; @@ -1070,11 +1073,11 @@ static u16 vhost_buf_to_lun(u8 *lun_buf) scsi_command_size(cdb), VHOST_SCSI_MAX_CDB_SIZE); goto err; } - cmd = vhost_scsi_get_tag(vq, tpg, cdb, tag, lun, task_attr, + cmd = vhost_scsi_get_cmd(vq, tpg, cdb, tag, lun, task_attr, exp_data_len + prot_bytes, data_direction); if (IS_ERR(cmd)) { - vq_err(vq, "vhost_scsi_get_tag failed %ld\n", + vq_err(vq, "vhost_scsi_get_cmd failed %ld\n", PTR_ERR(cmd)); goto err; } @@ -1378,6 +1381,83 @@ static void vhost_scsi_flush(struct vhost_scsi *vs) wait_for_completion(&old_inflight[i]->comp); } +static void vhost_scsi_destroy_vq_cmds(struct vhost_virtqueue *vq) +{ + struct vhost_scsi_virtqueue *svq = container_of(vq, + struct vhost_scsi_virtqueue, vq); + struct vhost_scsi_cmd *tv_cmd; + unsigned int i; + + if (!svq->scsi_cmds) + return; + + for (i = 0; i < svq->max_cmds; i++) { + tv_cmd = &svq->scsi_cmds[i]; + + kfree(tv_cmd->tvc_sgl); + kfree(tv_cmd->tvc_prot_sgl); + kfree(tv_cmd->tvc_upages); + } + + sbitmap_free(&svq->scsi_tags); + kfree(svq->scsi_cmds); + svq->scsi_cmds = NULL; +} + +static int vhost_scsi_setup_vq_cmds(struct vhost_virtqueue *vq, int max_cmds) +{ + struct vhost_scsi_virtqueue *svq = container_of(vq, + struct vhost_scsi_virtqueue, vq); + struct vhost_scsi_cmd *tv_cmd; + unsigned int i; + + if (svq->scsi_cmds) + return 0; + + if (sbitmap_init_node(&svq->scsi_tags, max_cmds, -1, GFP_KERNEL, + NUMA_NO_NODE)) + return -ENOMEM; + svq->max_cmds = max_cmds; + + svq->scsi_cmds = kcalloc(max_cmds, sizeof(*tv_cmd), GFP_KERNEL); + if (!svq->scsi_cmds) { + sbitmap_free(&svq->scsi_tags); + return -ENOMEM; + } + + for (i = 0; i < max_cmds; i++) { + tv_cmd = &svq->scsi_cmds[i]; + + tv_cmd->tvc_sgl = kcalloc(VHOST_SCSI_PREALLOC_SGLS, + sizeof(struct scatterlist), + GFP_KERNEL); + if (!tv_cmd->tvc_sgl) { + pr_err("Unable to allocate tv_cmd->tvc_sgl\n"); + goto out; + } + + tv_cmd->tvc_upages = kcalloc(VHOST_SCSI_PREALLOC_UPAGES, + sizeof(struct page *), + GFP_KERNEL); + if (!tv_cmd->tvc_upages) { + pr_err("Unable to allocate tv_cmd->tvc_upages\n"); + goto out; + } + + tv_cmd->tvc_prot_sgl = kcalloc(VHOST_SCSI_PREALLOC_PROT_SGLS, + sizeof(struct scatterlist), + GFP_KERNEL); + if (!tv_cmd->tvc_prot_sgl) { + pr_err("Unable to allocate tv_cmd->tvc_prot_sgl\n"); + goto out; + } + } + return 0; +out: + vhost_scsi_destroy_vq_cmds(vq); + return -ENOMEM; +} + /* * Called from vhost_scsi_ioctl() context to walk the list of available * vhost_scsi_tpg with an active struct vhost_scsi_nexus @@ -1432,10 +1512,9 @@ static void vhost_scsi_flush(struct vhost_scsi *vs) if (!strcmp(tv_tport->tport_name, t->vhost_wwpn)) { if (vs->vs_tpg && vs->vs_tpg[tpg->tport_tpgt]) { - kfree(vs_tpg); mutex_unlock(&tpg->tv_tpg_mutex); ret = -EEXIST; - goto out; + goto undepend; } /* * In order to ensure individual vhost-scsi configfs @@ -1447,9 +1526,8 @@ static void vhost_scsi_flush(struct vhost_scsi *vs) ret = target_depend_item(&se_tpg->tpg_group.cg_item); if (ret) { pr_warn("target_depend_item() failed: %d\n", ret); - kfree(vs_tpg); mutex_unlock(&tpg->tv_tpg_mutex); - goto out; + goto undepend; } tpg->tv_tpg_vhost_count++; tpg->vhost_scsi = vs; @@ -1462,6 +1540,16 @@ static void vhost_scsi_flush(struct vhost_scsi *vs) if (match) { memcpy(vs->vs_vhost_wwpn, t->vhost_wwpn, sizeof(vs->vs_vhost_wwpn)); + + for (i = VHOST_SCSI_VQ_IO; i < VHOST_SCSI_MAX_VQ; i++) { + vq = &vs->vqs[i].vq; + if (!vq->initialized) + continue; + + if (vhost_scsi_setup_vq_cmds(vq, vq->num)) + goto destroy_vq_cmds; + } + for (i = 0; i < VHOST_SCSI_MAX_VQ; i++) { vq = &vs->vqs[i].vq; if (!vq->initialized) @@ -1484,7 +1572,22 @@ static void vhost_scsi_flush(struct vhost_scsi *vs) vhost_scsi_flush(vs); kfree(vs->vs_tpg); vs->vs_tpg = vs_tpg; + goto out; +destroy_vq_cmds: + for (i--; i >= VHOST_SCSI_VQ_IO; i--) { + if (!vhost_vq_get_backend(&vs->vqs[i].vq)) + vhost_scsi_destroy_vq_cmds(&vs->vqs[i].vq); + } +undepend: + for (i = 0; i < VHOST_SCSI_MAX_TARGET; i++) { + tpg = vs_tpg[i]; + if (tpg) { + tpg->tv_tpg_vhost_count--; + target_undepend_item(&tpg->se_tpg.tpg_group.cg_item); + } + } + kfree(vs_tpg); out: mutex_unlock(&vs->dev.mutex); mutex_unlock(&vhost_scsi_mutex); @@ -1560,6 +1663,12 @@ static void vhost_scsi_flush(struct vhost_scsi *vs) mutex_lock(&vq->mutex); vhost_vq_set_backend(vq, NULL); mutex_unlock(&vq->mutex); + /* + * Make sure cmds are not running before tearing them + * down. + */ + vhost_scsi_flush(vs); + vhost_scsi_destroy_vq_cmds(vq); } } /* @@ -1861,23 +1970,6 @@ static void vhost_scsi_port_unlink(struct se_portal_group *se_tpg, mutex_unlock(&vhost_scsi_mutex); } -static void vhost_scsi_free_cmd_map_res(struct se_session *se_sess) -{ - struct vhost_scsi_cmd *tv_cmd; - unsigned int i; - - if (!se_sess->sess_cmd_map) - return; - - for (i = 0; i < VHOST_SCSI_DEFAULT_TAGS; i++) { - tv_cmd = &((struct vhost_scsi_cmd *)se_sess->sess_cmd_map)[i]; - - kfree(tv_cmd->tvc_sgl); - kfree(tv_cmd->tvc_prot_sgl); - kfree(tv_cmd->tvc_upages); - } -} - static ssize_t vhost_scsi_tpg_attrib_fabric_prot_type_store( struct config_item *item, const char *page, size_t count) { @@ -1917,45 +2009,6 @@ static ssize_t vhost_scsi_tpg_attrib_fabric_prot_type_show( NULL, }; -static int vhost_scsi_nexus_cb(struct se_portal_group *se_tpg, - struct se_session *se_sess, void *p) -{ - struct vhost_scsi_cmd *tv_cmd; - unsigned int i; - - for (i = 0; i < VHOST_SCSI_DEFAULT_TAGS; i++) { - tv_cmd = &((struct vhost_scsi_cmd *)se_sess->sess_cmd_map)[i]; - - tv_cmd->tvc_sgl = kcalloc(VHOST_SCSI_PREALLOC_SGLS, - sizeof(struct scatterlist), - GFP_KERNEL); - if (!tv_cmd->tvc_sgl) { - pr_err("Unable to allocate tv_cmd->tvc_sgl\n"); - goto out; - } - - tv_cmd->tvc_upages = kcalloc(VHOST_SCSI_PREALLOC_UPAGES, - sizeof(struct page *), - GFP_KERNEL); - if (!tv_cmd->tvc_upages) { - pr_err("Unable to allocate tv_cmd->tvc_upages\n"); - goto out; - } - - tv_cmd->tvc_prot_sgl = kcalloc(VHOST_SCSI_PREALLOC_PROT_SGLS, - sizeof(struct scatterlist), - GFP_KERNEL); - if (!tv_cmd->tvc_prot_sgl) { - pr_err("Unable to allocate tv_cmd->tvc_prot_sgl\n"); - goto out; - } - } - return 0; -out: - vhost_scsi_free_cmd_map_res(se_sess); - return -ENOMEM; -} - static int vhost_scsi_make_nexus(struct vhost_scsi_tpg *tpg, const char *name) { @@ -1979,12 +2032,9 @@ static int vhost_scsi_make_nexus(struct vhost_scsi_tpg *tpg, * struct se_node_acl for the vhost_scsi struct se_portal_group with * the SCSI Initiator port name of the passed configfs group 'name'. */ - tv_nexus->tvn_se_sess = target_setup_session(&tpg->se_tpg, - VHOST_SCSI_DEFAULT_TAGS, - sizeof(struct vhost_scsi_cmd), + tv_nexus->tvn_se_sess = target_setup_session(&tpg->se_tpg, 0, 0, TARGET_PROT_DIN_PASS | TARGET_PROT_DOUT_PASS, - (unsigned char *)name, tv_nexus, - vhost_scsi_nexus_cb); + (unsigned char *)name, tv_nexus, NULL); if (IS_ERR(tv_nexus->tvn_se_sess)) { mutex_unlock(&tpg->tv_tpg_mutex); kfree(tv_nexus); @@ -2034,7 +2084,6 @@ static int vhost_scsi_drop_nexus(struct vhost_scsi_tpg *tpg) " %s Initiator Port: %s\n", vhost_scsi_dump_proto_id(tpg->tport), tv_nexus->tvn_se_sess->se_node_acl->initiatorname); - vhost_scsi_free_cmd_map_res(se_sess); /* * Release the SCSI I_T Nexus to the emulated vhost Target Port */ From patchwork Wed Oct 7 20:54:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 11821795 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CA6A014D5 for ; Wed, 7 Oct 2020 20:55:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A8BD12083B for ; Wed, 7 Oct 2020 20:55:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="p5bbpASx" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728373AbgJGUzd (ORCPT ); Wed, 7 Oct 2020 16:55:33 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:43336 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728183AbgJGUz1 (ORCPT ); Wed, 7 Oct 2020 16:55:27 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KnSPj038239; Wed, 7 Oct 2020 20:55:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=aYEB9oNwgE3rL1pWC/WbwJ3rVzn4m1SbcZXmZ3XDWmI=; b=p5bbpASxPHqXtu3uPe6Sknw95w4NL949frRVtZpLaXFyqsleB19GpIjTKH1lhBXojjeB H/+3VDxSJYjPs8//bm5J3OZN79fyuXsl+ITbtJTkYDWhRONXBbvrG6CHsNkylWgDKtND SGXvwZAnOzwxSAexcWQQTrs3PEEsizylddvU8Jz1Xnq5urvgyRyKjwpAjhqvEY4/CNl7 jw3zOclVFHeCqsU+z+GGk3o6pKWQcslpJpvGNxkRWfr94Py9/MfhVUk7H5vwsfi2gjKy S7W1B9m5x55jjai4ksIKri7B1rH3RJuTmORUmm3q32thttlu9zRUbjuUaCL/kuZV/Eyj Cw== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by userp2120.oracle.com with ESMTP id 33xhxn475m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 07 Oct 2020 20:55:23 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KpC1w028966; Wed, 7 Oct 2020 20:55:23 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userp3020.oracle.com with ESMTP id 33yyjhpuh9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 07 Oct 2020 20:55:23 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 097KtLt9009077; Wed, 7 Oct 2020 20:55:21 GMT Received: from ol2.localdomain (/73.88.28.6) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Oct 2020 13:55:21 -0700 From: Mike Christie To: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, pbonzini@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Subject: [PATCH 09/16] vhost scsi: fix cmd completion race Date: Wed, 7 Oct 2020 15:54:54 -0500 Message-Id: <1602104101-5592-10-git-send-email-michael.christie@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> References: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=2 adultscore=0 bulkscore=0 phishscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 bulkscore=0 impostorscore=0 lowpriorityscore=0 suspectscore=2 phishscore=0 mlxlogscore=999 adultscore=0 clxscore=1015 spamscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org We might not do the final se_cmd put from vhost_scsi_complete_cmd_work. When the last put happens a little later then we could race where vhost_scsi_complete_cmd_work does vhost_signal, the guest runs and sends more IO, and vhost_scsi_handle_vq runs but does not find any free cmds. This patch has us delay completing the cmd until the last lio core ref is dropped. We then know that once we signal to the guest that the cmd is completed that if it queues a new command it will find a free cmd. Signed-off-by: Mike Christie --- drivers/vhost/scsi.c | 42 +++++++++++++++--------------------------- 1 file changed, 15 insertions(+), 27 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index f6b9010..2fa48dd 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -322,7 +322,7 @@ static u32 vhost_scsi_tpg_get_inst_index(struct se_portal_group *se_tpg) return 1; } -static void vhost_scsi_release_cmd(struct se_cmd *se_cmd) +static void vhost_scsi_release_cmd_res(struct se_cmd *se_cmd) { struct vhost_scsi_cmd *tv_cmd = container_of(se_cmd, struct vhost_scsi_cmd, tvc_se_cmd); @@ -344,6 +344,16 @@ static void vhost_scsi_release_cmd(struct se_cmd *se_cmd) vhost_scsi_put_inflight(inflight); } +static void vhost_scsi_release_cmd(struct se_cmd *se_cmd) +{ + struct vhost_scsi_cmd *cmd = container_of(se_cmd, + struct vhost_scsi_cmd, tvc_se_cmd); + struct vhost_scsi *vs = cmd->tvc_vhost; + + llist_add(&cmd->tvc_completion_list, &vs->vs_completion_list); + vhost_work_queue(&vs->dev, &vs->vs_completion_work); +} + static u32 vhost_scsi_sess_get_index(struct se_session *se_sess) { return 0; @@ -366,28 +376,15 @@ static int vhost_scsi_get_cmd_state(struct se_cmd *se_cmd) return 0; } -static void vhost_scsi_complete_cmd(struct vhost_scsi_cmd *cmd) -{ - struct vhost_scsi *vs = cmd->tvc_vhost; - - llist_add(&cmd->tvc_completion_list, &vs->vs_completion_list); - - vhost_work_queue(&vs->dev, &vs->vs_completion_work); -} - static int vhost_scsi_queue_data_in(struct se_cmd *se_cmd) { - struct vhost_scsi_cmd *cmd = container_of(se_cmd, - struct vhost_scsi_cmd, tvc_se_cmd); - vhost_scsi_complete_cmd(cmd); + transport_generic_free_cmd(se_cmd, 0); return 0; } static int vhost_scsi_queue_status(struct se_cmd *se_cmd) { - struct vhost_scsi_cmd *cmd = container_of(se_cmd, - struct vhost_scsi_cmd, tvc_se_cmd); - vhost_scsi_complete_cmd(cmd); + transport_generic_free_cmd(se_cmd, 0); return 0; } @@ -433,15 +430,6 @@ static void vhost_scsi_free_evt(struct vhost_scsi *vs, struct vhost_scsi_evt *ev return evt; } -static void vhost_scsi_free_cmd(struct vhost_scsi_cmd *cmd) -{ - struct se_cmd *se_cmd = &cmd->tvc_se_cmd; - - /* TODO locking against target/backend threads? */ - transport_generic_free_cmd(se_cmd, 0); - -} - static int vhost_scsi_check_stop_free(struct se_cmd *se_cmd) { return target_put_sess_cmd(se_cmd); @@ -560,7 +548,7 @@ static void vhost_scsi_complete_cmd_work(struct vhost_work *work) } else pr_err("Faulted on virtio_scsi_cmd_resp\n"); - vhost_scsi_free_cmd(cmd); + vhost_scsi_release_cmd_res(se_cmd); } vq = -1; @@ -1096,7 +1084,7 @@ static u16 vhost_buf_to_lun(u8 *lun_buf) &prot_iter, exp_data_len, &data_iter))) { vq_err(vq, "Failed to map iov to sgl\n"); - vhost_scsi_release_cmd(&cmd->tvc_se_cmd); + vhost_scsi_release_cmd_res(&cmd->tvc_se_cmd); goto err; } } From patchwork Wed Oct 7 20:54:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 11821831 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D05BA6CA for ; Wed, 7 Oct 2020 20:57:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9E9BD20782 for ; Wed, 7 Oct 2020 20:57:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Z+6Xdf1A" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728336AbgJGU5a (ORCPT ); Wed, 7 Oct 2020 16:57:30 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:44724 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726152AbgJGU5a (ORCPT ); Wed, 7 Oct 2020 16:57:30 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KnTUl038295; Wed, 7 Oct 2020 20:57:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=yhya1YvqWr786+HzbwNs8Jo4GH88EaM6i2YMxTq3ajA=; b=Z+6Xdf1Asm+gK8fxV7VcaHEFz9cajjKkdLciOpmQbWUhaNvjgTZsOh4v07YIU3kEOdTT e6lppLOxKwdqwX0U7fkwJIsMmEe6SxwIOYrhrD/9hyr8tEpSpsoqozEIhtToEIbg2fHj sXyF+6TXjP/rLmwcnxgxY7hwKMU64JRkjw0Uq37C5bhnpWMf5jy40MqXznF6woPg2L+s p1Om0HMezMNnRnkP39SdGInmWRLjdeywnWG4bCX+WlRFQDprri3+G8hcFdlD6AO0Hhui RQ4KDz4FPagxqWA+PpYmZt9oUYtXJ/gEPaJ/JxpYgVDMg/h3Tgeev6vUtSAIJhcAZnqh 4g== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by userp2120.oracle.com with ESMTP id 33xhxn47dg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 07 Oct 2020 20:57:23 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KpLhj166352; Wed, 7 Oct 2020 20:55:23 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserp3030.oracle.com with ESMTP id 33y2vpyv16-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 07 Oct 2020 20:55:23 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 097KtMuV005654; Wed, 7 Oct 2020 20:55:22 GMT Received: from ol2.localdomain (/73.88.28.6) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Oct 2020 13:55:22 -0700 From: Mike Christie To: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, pbonzini@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Subject: [PATCH 10/16] vhost scsi: Add support for LUN resets. Date: Wed, 7 Oct 2020 15:54:55 -0500 Message-Id: <1602104101-5592-11-git-send-email-michael.christie@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> References: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 malwarescore=0 suspectscore=2 spamscore=0 phishscore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 bulkscore=0 impostorscore=0 lowpriorityscore=0 suspectscore=2 phishscore=0 mlxlogscore=999 adultscore=0 clxscore=1015 spamscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org In newer versions of virtio-scsi we just reset the timer when an a command times out, so TMFs are never sent for the cmd time out case. However, in older kernels and for the TMF inject cases, we can still get resets and we end up just failing immediately so the guest might see the device get offlined and IO errors. For the older kernel cases, we want the same end result as the modern virtio-scsi driver where we let the lower levels fire their error handling and handle the problem. And at the upper levels we want to wait. This patch ties the LUN reset handling into the LIO TMF code which will just wait for outstanding commands to complete like we are doing in the modern virtio-scsi case. Note: I did not handle the ABORT case to keep this simple. For ABORTs LIO just waits on the cmd like how it does for the RESET case. If an ABORT fails, the guest OS ends up escalating to LUN RESET, so in the end we get the same behavior where we wait on the outstanding cmds. Signed-off-by: Mike Christie --- drivers/vhost/scsi.c | 147 ++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 134 insertions(+), 13 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 2fa48dd..f543fa0 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -139,6 +139,7 @@ struct vhost_scsi_tpg { struct se_portal_group se_tpg; /* Pointer back to vhost_scsi, protected by tv_tpg_mutex */ struct vhost_scsi *vhost_scsi; + struct list_head tmf_queue; }; struct vhost_scsi_tport { @@ -211,6 +212,20 @@ struct vhost_scsi { int vs_events_nr; /* num of pending events, protected by vq->mutex */ }; +struct vhost_scsi_tmf { + struct vhost_work vwork; + struct vhost_scsi_tpg *tpg; + struct vhost_scsi *vhost; + struct vhost_scsi_virtqueue *svq; + struct list_head queue_entry; + + struct se_cmd se_cmd; + struct vhost_scsi_inflight *inflight; + struct iovec resp_iov; + int in_iovs; + int vq_desc; +}; + /* * Context for processing request and control queue operations. */ @@ -344,14 +359,32 @@ static void vhost_scsi_release_cmd_res(struct se_cmd *se_cmd) vhost_scsi_put_inflight(inflight); } +static void vhost_scsi_release_tmf_res(struct vhost_scsi_tmf *tmf) +{ + struct vhost_scsi_tpg *tpg = tmf->tpg; + struct vhost_scsi_inflight *inflight = tmf->inflight; + + mutex_lock(&tpg->tv_tpg_mutex); + list_add_tail(&tpg->tmf_queue, &tmf->queue_entry); + mutex_unlock(&tpg->tv_tpg_mutex); + vhost_scsi_put_inflight(inflight); +} + static void vhost_scsi_release_cmd(struct se_cmd *se_cmd) { - struct vhost_scsi_cmd *cmd = container_of(se_cmd, + if (se_cmd->se_cmd_flags & SCF_SCSI_TMR_CDB) { + struct vhost_scsi_tmf *tmf = container_of(se_cmd, + struct vhost_scsi_tmf, se_cmd); + + vhost_work_queue(&tmf->vhost->dev, &tmf->vwork); + } else { + struct vhost_scsi_cmd *cmd = container_of(se_cmd, struct vhost_scsi_cmd, tvc_se_cmd); - struct vhost_scsi *vs = cmd->tvc_vhost; + struct vhost_scsi *vs = cmd->tvc_vhost; - llist_add(&cmd->tvc_completion_list, &vs->vs_completion_list); - vhost_work_queue(&vs->dev, &vs->vs_completion_work); + llist_add(&cmd->tvc_completion_list, &vs->vs_completion_list); + vhost_work_queue(&vs->dev, &vs->vs_completion_work); + } } static u32 vhost_scsi_sess_get_index(struct se_session *se_sess) @@ -390,7 +423,10 @@ static int vhost_scsi_queue_status(struct se_cmd *se_cmd) static void vhost_scsi_queue_tm_rsp(struct se_cmd *se_cmd) { - return; + struct vhost_scsi_tmf *tmf = container_of(se_cmd, struct vhost_scsi_tmf, + se_cmd); + + transport_generic_free_cmd(&tmf->se_cmd, 0); } static void vhost_scsi_aborted_task(struct se_cmd *se_cmd) @@ -1120,9 +1156,9 @@ static u16 vhost_buf_to_lun(u8 *lun_buf) } static void -vhost_scsi_send_tmf_reject(struct vhost_scsi *vs, - struct vhost_virtqueue *vq, - struct vhost_scsi_ctx *vc) +vhost_scsi_send_tmf_resp(struct vhost_scsi *vs, struct vhost_virtqueue *vq, + int in_iovs, int vq_desc, struct iovec *resp_iov, + int tmf_resp_code) { struct virtio_scsi_ctrl_tmf_resp rsp; struct iov_iter iov_iter; @@ -1130,17 +1166,87 @@ static u16 vhost_buf_to_lun(u8 *lun_buf) pr_debug("%s\n", __func__); memset(&rsp, 0, sizeof(rsp)); - rsp.response = VIRTIO_SCSI_S_FUNCTION_REJECTED; + rsp.response = tmf_resp_code; - iov_iter_init(&iov_iter, READ, &vq->iov[vc->out], vc->in, sizeof(rsp)); + iov_iter_init(&iov_iter, READ, resp_iov, in_iovs, sizeof(rsp)); ret = copy_to_iter(&rsp, sizeof(rsp), &iov_iter); if (likely(ret == sizeof(rsp))) - vhost_add_used_and_signal(&vs->dev, vq, vc->head, 0); + vhost_add_used_and_signal(&vs->dev, vq, vq_desc, 0); else pr_err("Faulted on virtio_scsi_ctrl_tmf_resp\n"); } +static void vhost_scsi_tmf_resp_work(struct vhost_work *work) +{ + struct vhost_scsi_tmf *tmf = container_of(work, struct vhost_scsi_tmf, + vwork); + int resp_code; + + if (tmf->se_cmd.se_tmr_req->response == TMR_FUNCTION_COMPLETE) + resp_code = VIRTIO_SCSI_S_FUNCTION_SUCCEEDED; + else + resp_code = VIRTIO_SCSI_S_FUNCTION_REJECTED; + + vhost_scsi_send_tmf_resp(tmf->vhost, &tmf->svq->vq, tmf->in_iovs, + tmf->vq_desc, &tmf->resp_iov, resp_code); + vhost_scsi_release_tmf_res(tmf); +} + +static void +vhost_scsi_handle_tmf(struct vhost_scsi *vs, struct vhost_scsi_tpg *tpg, + struct vhost_virtqueue *vq, + struct virtio_scsi_ctrl_tmf_req *vtmf, + struct vhost_scsi_ctx *vc) +{ + struct vhost_scsi_virtqueue *svq = container_of(vq, + struct vhost_scsi_virtqueue, vq); + struct vhost_scsi_tmf *tmf; + + if (vhost32_to_cpu(vq, vtmf->subtype) != + VIRTIO_SCSI_T_TMF_LOGICAL_UNIT_RESET) + goto send_reject; + + if (!tpg->tpg_nexus || !tpg->tpg_nexus->tvn_se_sess) { + pr_err("Unable to locate active struct vhost_scsi_nexus for LUN RESET.\n"); + goto send_reject; + } + + mutex_lock(&tpg->tv_tpg_mutex); + if (list_empty(&tpg->tmf_queue)) { + pr_err("Missing reserve TMF. Could not handle LUN RESET.\n"); + mutex_unlock(&tpg->tv_tpg_mutex); + goto send_reject; + } + + tmf = list_first_entry(&tpg->tmf_queue, struct vhost_scsi_tmf, + queue_entry); + list_del_init(&tmf->queue_entry); + mutex_unlock(&tpg->tv_tpg_mutex); + + tmf->tpg = tpg; + tmf->vhost = vs; + tmf->svq = svq; + tmf->resp_iov = vq->iov[vc->out]; + tmf->vq_desc = vc->head; + tmf->in_iovs = vc->in; + tmf->inflight = vhost_scsi_get_inflight(vq); + + if (target_submit_tmr(&tmf->se_cmd, tpg->tpg_nexus->tvn_se_sess, NULL, + vhost_buf_to_lun(vtmf->lun), NULL, + TMR_LUN_RESET, GFP_KERNEL, 0, + TARGET_SCF_ACK_KREF) < 0) { + vhost_scsi_release_tmf_res(tmf); + goto send_reject; + } + + return; + +send_reject: + vhost_scsi_send_tmf_resp(vs, vq, vc->in, vc->head, &vq->iov[vc->out], + VIRTIO_SCSI_S_FUNCTION_REJECTED); +} + static void vhost_scsi_send_an_resp(struct vhost_scsi *vs, struct vhost_virtqueue *vq, @@ -1166,6 +1272,7 @@ static u16 vhost_buf_to_lun(u8 *lun_buf) static void vhost_scsi_ctl_handle_vq(struct vhost_scsi *vs, struct vhost_virtqueue *vq) { + struct vhost_scsi_tpg *tpg; union { __virtio32 type; struct virtio_scsi_ctrl_an_req an; @@ -1247,12 +1354,12 @@ static u16 vhost_buf_to_lun(u8 *lun_buf) vc.req += typ_size; vc.req_size -= typ_size; - ret = vhost_scsi_get_req(vq, &vc, NULL); + ret = vhost_scsi_get_req(vq, &vc, &tpg); if (ret) goto err; if (v_req.type == VIRTIO_SCSI_T_TMF) - vhost_scsi_send_tmf_reject(vs, vq, &vc); + vhost_scsi_handle_tmf(vs, tpg, vq, &v_req.tmf, &vc); else vhost_scsi_send_an_resp(vs, vq, &vc); err: @@ -1927,11 +2034,19 @@ static int vhost_scsi_port_link(struct se_portal_group *se_tpg, { struct vhost_scsi_tpg *tpg = container_of(se_tpg, struct vhost_scsi_tpg, se_tpg); + struct vhost_scsi_tmf *tmf; + + tmf = kzalloc(sizeof(*tmf), GFP_KERNEL); + if (!tmf) + return -ENOMEM; + INIT_LIST_HEAD(&tmf->queue_entry); + vhost_work_init(&tmf->vwork, vhost_scsi_tmf_resp_work); mutex_lock(&vhost_scsi_mutex); mutex_lock(&tpg->tv_tpg_mutex); tpg->tv_tpg_port_count++; + list_add_tail(&tmf->queue_entry, &tpg->tmf_queue); mutex_unlock(&tpg->tv_tpg_mutex); vhost_scsi_hotplug(tpg, lun); @@ -1946,11 +2061,16 @@ static void vhost_scsi_port_unlink(struct se_portal_group *se_tpg, { struct vhost_scsi_tpg *tpg = container_of(se_tpg, struct vhost_scsi_tpg, se_tpg); + struct vhost_scsi_tmf *tmf; mutex_lock(&vhost_scsi_mutex); mutex_lock(&tpg->tv_tpg_mutex); tpg->tv_tpg_port_count--; + tmf = list_first_entry(&tpg->tmf_queue, struct vhost_scsi_tmf, + queue_entry); + list_del(&tmf->queue_entry); + kfree(tmf); mutex_unlock(&tpg->tv_tpg_mutex); vhost_scsi_hotunplug(tpg, lun); @@ -2211,6 +2331,7 @@ static ssize_t vhost_scsi_tpg_nexus_store(struct config_item *item, } mutex_init(&tpg->tv_tpg_mutex); INIT_LIST_HEAD(&tpg->tv_tpg_list); + INIT_LIST_HEAD(&tpg->tmf_queue); tpg->tport = tport; tpg->tport_tpgt = tpgt; From patchwork Wed Oct 7 20:54:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 11821821 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2A1AB6CA for ; Wed, 7 Oct 2020 20:55:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 03E122083B for ; Wed, 7 Oct 2020 20:55:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="MFJUGCrH" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728830AbgJGUzh (ORCPT ); Wed, 7 Oct 2020 16:55:37 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:48256 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728801AbgJGUz2 (ORCPT ); Wed, 7 Oct 2020 16:55:28 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KoCgh028888; Wed, 7 Oct 2020 20:55:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=60uJMfmHb5c6yTfXQfCI6Hs08UShLQ9VnZ7vDwQNZPo=; b=MFJUGCrHFokuPMSEeO7/iKiVn4V7mFYy5XkP4bIRsXfkugj+c2XYtkMrk9S4zPe5Nvbf U3dHkivQh+RUEBAGb2J65M7soXA+Lsm8nUJn5LA9ULwNwrNETUZjQs6jyhXqTyyppgxR M5LcvSplA+4Hd+fB9T+Fra+kyG30mAV+mFjavp7nwAJGp6Z8Rs1ChGmrxGxD0t5S/e4o FntKWw9sb8fQabLVfuJRtEYEeUP8TuFSODTgKjYfuSCMo+kNXTIo056u2lnmxieAhLFV ELUJHkGAnWC030O6VZCokjMaANEtqFuv7Ir2z/jz33dkOWj0azwf7t/vqCwu9BadLdwc ag== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2130.oracle.com with ESMTP id 33xetb4ef6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 07 Oct 2020 20:55:25 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KpCWS028972; Wed, 7 Oct 2020 20:55:24 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userp3020.oracle.com with ESMTP id 33yyjhpuj9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 07 Oct 2020 20:55:24 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 097KtNiv005657; Wed, 7 Oct 2020 20:55:23 GMT Received: from ol2.localdomain (/73.88.28.6) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Oct 2020 13:55:22 -0700 From: Mike Christie To: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, pbonzini@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Subject: [PATCH 11/16] vhost scsi: remove extra flushes Date: Wed, 7 Oct 2020 15:54:56 -0500 Message-Id: <1602104101-5592-12-git-send-email-michael.christie@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> References: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 bulkscore=0 phishscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 mlxscore=0 clxscore=1015 priorityscore=1501 adultscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 malwarescore=0 suspectscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org The vhost work flush function was flushing the entire work queue, so there is no need for the double vhost_work_dev_flush calls in vhost_scsi_flush. And we do not need to call vhost_poll_flush for each poller because that call also ends up flushing the same work queue thread the vhost_work_dev_flush call flushed. Signed-off-by: Mike Christie --- drivers/vhost/scsi.c | 8 -------- 1 file changed, 8 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index f543fa0..b348e9c 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -1443,11 +1443,6 @@ static void vhost_scsi_handle_kick(struct vhost_work *work) vhost_scsi_handle_vq(vs, vq); } -static void vhost_scsi_flush_vq(struct vhost_scsi *vs, int index) -{ - vhost_poll_flush(&vs->vqs[index].vq.poll); -} - /* Callers must hold dev mutex */ static void vhost_scsi_flush(struct vhost_scsi *vs) { @@ -1466,9 +1461,6 @@ static void vhost_scsi_flush(struct vhost_scsi *vs) kref_put(&old_inflight[i]->kref, vhost_scsi_done_inflight); /* Flush both the vhost poll and vhost work */ - for (i = 0; i < VHOST_SCSI_MAX_VQ; i++) - vhost_scsi_flush_vq(vs, i); - vhost_work_dev_flush(&vs->dev); vhost_work_dev_flush(&vs->dev); /* Wait for all reqs issued before the flush to be finished */ From patchwork Wed Oct 7 20:54:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 11821835 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9994817D1 for ; Wed, 7 Oct 2020 20:57:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 740E120782 for ; Wed, 7 Oct 2020 20:57:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="eiDBb5rD" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728385AbgJGU5d (ORCPT ); Wed, 7 Oct 2020 16:57:33 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:36572 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727863AbgJGU5b (ORCPT ); Wed, 7 Oct 2020 16:57:31 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KoGb3113298; Wed, 7 Oct 2020 20:57:24 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=kEsw8MaL3Rqz6MCZK7TzqKzYFxtra7l/p0GyZvcJHqk=; b=eiDBb5rDn8G10xYB++6+IRtC90KDHKs0Gi52ZEO43BPfkmVXy8UOYOqib06imEo8kPiT IiVqYi1eduIP8liSZY3MZey7WgKFycYs4CAkVLf7ky0kw8zaJyDqEp483XY9HW6MlE/+ 9555/SwKq2F5mRrOYMIhNyufLLomQ+pNPac4WDQT39f9E2NVviC5OKh5WGk6BzXCfWyv k9jA8g8T4YHhWoStsqhZ/0UTl2aSAuQlmRmjPXDALsChDZBQTSF7rBcyYnaRKIyvi9az WZroGbkj3mftb8ojyZ0oX/I0iU3whX880UotCEQTfZPHagSDk2aOwBUHYiL7EF7mlof7 oA== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by aserp2120.oracle.com with ESMTP id 33ym34scyd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 07 Oct 2020 20:57:24 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097Kor1g125510; Wed, 7 Oct 2020 20:55:24 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserp3020.oracle.com with ESMTP id 3410k03uu1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 07 Oct 2020 20:55:24 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 097KtNmt005660; Wed, 7 Oct 2020 20:55:23 GMT Received: from ol2.localdomain (/73.88.28.6) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Oct 2020 13:55:23 -0700 From: Mike Christie To: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, pbonzini@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Subject: [PATCH 12/16] vhost: support multiple worker threads Date: Wed, 7 Oct 2020 15:54:57 -0500 Message-Id: <1602104101-5592-13-git-send-email-michael.christie@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> References: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 mlxlogscore=999 spamscore=0 adultscore=0 bulkscore=0 malwarescore=0 suspectscore=2 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 impostorscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 clxscore=1015 bulkscore=0 spamscore=0 malwarescore=0 phishscore=0 suspectscore=2 adultscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org This is a prep patch to support multiple vhost worker threads per vhost dev. This patch converts the code that had assumed a single worker thread by: 1. Moving worker related fields to a new struct vhost_worker. 2. Converting vhost.c code to use the new struct and assume we will have an array of workers. 3. It also exports a helper function that will be used in the last patch when vhost-scsi is converted to use this new functionality. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 232 +++++++++++++++++++++++++++++++++++++++----------- drivers/vhost/vhost.h | 12 ++- 2 files changed, 190 insertions(+), 54 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 1507400..e46b39e 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -231,18 +231,48 @@ void vhost_poll_stop(struct vhost_poll *poll) } EXPORT_SYMBOL_GPL(vhost_poll_stop); -void vhost_work_dev_flush(struct vhost_dev *dev) +static void vhost_work_queue_on(struct vhost_dev *dev, struct vhost_work *work, + int worker_id) +{ + if (!dev->workers) + return; + + if (!test_and_set_bit(VHOST_WORK_QUEUED, &work->flags)) { + /* We can only add the work to the list after we're + * sure it was not in the list. + * test_and_set_bit() implies a memory barrier. + */ + llist_add(&work->node, &dev->workers[worker_id]->work_list); + wake_up_process(dev->workers[worker_id]->task); + } +} + +void vhost_work_queue(struct vhost_dev *dev, struct vhost_work *work) +{ + vhost_work_queue_on(dev, work, 0); +} +EXPORT_SYMBOL_GPL(vhost_work_queue); + +static void vhost_work_flush_on(struct vhost_dev *dev, int worker_id) { struct vhost_flush_struct flush; - if (dev->worker) { + if (dev->workers) { init_completion(&flush.wait_event); vhost_work_init(&flush.work, vhost_flush_work); - vhost_work_queue(dev, &flush.work); + vhost_work_queue_on(dev, &flush.work, worker_id); wait_for_completion(&flush.wait_event); } } + +void vhost_work_dev_flush(struct vhost_dev *dev) +{ + int i; + + for (i = 0; i < dev->num_workers; i++) + vhost_work_flush_on(dev, i); +} EXPORT_SYMBOL_GPL(vhost_work_dev_flush); /* Flush any work that has been scheduled. When calling this, don't hold any @@ -253,26 +283,20 @@ void vhost_poll_flush(struct vhost_poll *poll) } EXPORT_SYMBOL_GPL(vhost_poll_flush); -void vhost_work_queue(struct vhost_dev *dev, struct vhost_work *work) +/* A lockless hint for busy polling code to exit the loop */ +bool vhost_has_work(struct vhost_dev *dev) { - if (!dev->worker) - return; + int i; - if (!test_and_set_bit(VHOST_WORK_QUEUED, &work->flags)) { - /* We can only add the work to the list after we're - * sure it was not in the list. - * test_and_set_bit() implies a memory barrier. - */ - llist_add(&work->node, &dev->work_list); - wake_up_process(dev->worker); + if (!dev->workers) + return false; + + for (i = 0; i < dev->num_workers; i++) { + if (!llist_empty(&dev->workers[i]->work_list)) + return true; } -} -EXPORT_SYMBOL_GPL(vhost_work_queue); -/* A lockless hint for busy polling code to exit the loop */ -bool vhost_has_work(struct vhost_dev *dev) -{ - return !llist_empty(&dev->work_list); + return false; } EXPORT_SYMBOL_GPL(vhost_has_work); @@ -339,7 +363,8 @@ static void vhost_vq_reset(struct vhost_dev *dev, static int vhost_worker(void *data) { - struct vhost_dev *dev = data; + struct vhost_worker *worker = data; + struct vhost_dev *dev = worker->dev; struct vhost_work *work, *work_next; struct llist_node *node; @@ -353,8 +378,7 @@ static int vhost_worker(void *data) __set_current_state(TASK_RUNNING); break; } - - node = llist_del_all(&dev->work_list); + node = llist_del_all(&worker->work_list); if (!node) schedule(); @@ -507,13 +531,13 @@ int vhost_dev_init(struct vhost_dev *dev, dev->umem = NULL; dev->iotlb = NULL; dev->mm = NULL; - dev->worker = NULL; + dev->workers = NULL; + dev->num_workers = 0; dev->iov_limit = iov_limit; dev->weight = weight; - dev->byte_weight = byte_weight; dev->use_worker = use_worker; + dev->byte_weight = byte_weight; dev->msg_handler = msg_handler; - init_llist_head(&dev->work_list); init_waitqueue_head(&dev->wait); INIT_LIST_HEAD(&dev->read_list); INIT_LIST_HEAD(&dev->pending_list); @@ -559,16 +583,28 @@ static void vhost_attach_cgroups_work(struct vhost_work *work) s->ret = cgroup_attach_task_all(s->owner, current); } -static int vhost_attach_cgroups(struct vhost_dev *dev) +static int vhost_attach_cgroups_on(struct vhost_dev *dev, int worker_id) { struct vhost_attach_cgroups_struct attach; attach.owner = current; vhost_work_init(&attach.work, vhost_attach_cgroups_work); - vhost_work_queue(dev, &attach.work); - vhost_work_dev_flush(dev); + vhost_work_queue_on(dev, &attach.work, worker_id); + vhost_work_flush_on(dev, worker_id); return attach.ret; } +static int vhost_attach_cgroups(struct vhost_dev *dev, int first_worker) +{ + int i, ret = 0; + + for (i = first_worker; i < dev->num_workers; i++) { + ret = vhost_attach_cgroups_on(dev, i); + if (ret) + break; + } + + return ret; +} /* Caller should have device mutex */ bool vhost_dev_has_owner(struct vhost_dev *dev) @@ -607,10 +643,117 @@ static void vhost_detach_mm(struct vhost_dev *dev) dev->mm = NULL; } +static void vhost_workers_free(struct vhost_dev *dev) +{ + struct vhost_worker *worker; + int i; + + if (!dev->workers) + return; + + for (i = 0; i < dev->num_workers; i++) { + worker = dev->workers[i]; + + WARN_ON(!llist_empty(&worker->work_list)); + kthread_stop(worker->task); + kfree(worker); + } + + kfree(dev->workers); + dev->workers = NULL; + dev->num_workers = 0; +} + +static int vhost_worker_create(struct vhost_dev *dev, int worker_id) +{ + struct vhost_worker *worker; + struct task_struct *task; + int ret; + + worker = kzalloc(sizeof(*worker), GFP_KERNEL); + if (!worker) + return -ENOMEM; + + init_llist_head(&worker->work_list); + worker->dev = dev; + + task = kthread_create(vhost_worker, worker, "vhost-%d", current->pid); + if (IS_ERR(task)) { + ret = PTR_ERR(task); + goto free_worker; + } + + dev->workers[worker_id] = worker; + worker->task = task; + wake_up_process(task); /* avoid contributing to loadavg */ + return 0; + +free_worker: + kfree(worker); + return ret; +} + +/** + * vhost_workers_create - create vhost workers and attach to cgroup + * @dev: vhost device + * @new_num_workers: the total number of workers we want after this returns + * + * Caller must have the device mutex and have stopped operations that + * can access the workers array. + */ +int vhost_workers_create(struct vhost_dev *dev, int new_num_workers) +{ + struct vhost_worker **new_workers; + struct mm_struct *mm; + bool owner_match = true; + int i, err, start; + + if (new_num_workers == dev->num_workers) + return 0; + + if (new_num_workers < dev->num_workers) + return -EINVAL; + + if (vhost_dev_has_owner(dev)) { + mm = get_task_mm(current); + if (mm != dev->mm) + owner_match = false; + mmput(mm); + if (!owner_match) + return -EBUSY; + } + + new_workers = krealloc(dev->workers, new_num_workers * sizeof(*new_workers), + GFP_KERNEL); + if (!new_workers) { + err = -ENOMEM; + goto free_workers; + } + dev->workers = new_workers; + + start = dev->num_workers; + for (i = start; i < new_num_workers; i++) { + err = vhost_worker_create(dev, i); + if (err) + goto free_workers; + dev->num_workers++; + } + + err = vhost_attach_cgroups(dev, start); + if (err) + goto free_workers; + + return 0; + +free_workers: + vhost_workers_free(dev); + return err; +} +EXPORT_SYMBOL_GPL(vhost_workers_create); + /* Caller should have device mutex */ long vhost_dev_set_owner(struct vhost_dev *dev) { - struct task_struct *worker; int err; /* Is there an owner already? */ @@ -623,27 +766,16 @@ long vhost_dev_set_owner(struct vhost_dev *dev) dev->kcov_handle = kcov_common_handle(); if (dev->use_worker) { - worker = kthread_create(vhost_worker, dev, - "vhost-%d", current->pid); - if (IS_ERR(worker)) { - err = PTR_ERR(worker); - goto err_worker; - } - - dev->worker = worker; - wake_up_process(worker); /* avoid contributing to loadavg */ - - err = vhost_attach_cgroups(dev); + /* + * All drivers that set use_worker=true, use at least one + * worker. Drivers like vhost-scsi may override this later. + */ + err = vhost_workers_create(dev, 1); if (err) - goto err_cgroup; + goto err_worker; } return 0; -err_cgroup: - if (dev->worker) { - kthread_stop(dev->worker); - dev->worker = NULL; - } err_worker: vhost_detach_mm(dev); dev->kcov_handle = 0; @@ -736,12 +868,8 @@ void vhost_dev_cleanup(struct vhost_dev *dev) dev->iotlb = NULL; vhost_clear_msg(dev); wake_up_interruptible_poll(&dev->wait, EPOLLIN | EPOLLRDNORM); - WARN_ON(!llist_empty(&dev->work_list)); - if (dev->worker) { - kthread_stop(dev->worker); - dev->worker = NULL; - dev->kcov_handle = 0; - } + vhost_workers_free(dev); + dev->kcov_handle = 0; vhost_detach_mm(dev); } EXPORT_SYMBOL_GPL(vhost_dev_cleanup); diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h index 2c6473e..a91be8a 100644 --- a/drivers/vhost/vhost.h +++ b/drivers/vhost/vhost.h @@ -25,6 +25,12 @@ struct vhost_work { unsigned long flags; }; +struct vhost_worker { + struct task_struct *task; + struct llist_head work_list; + struct vhost_dev *dev; +}; + /* Poll a file (eventfd or socket) */ /* Note: there's nothing vhost specific about this structure. */ struct vhost_poll { @@ -39,6 +45,7 @@ struct vhost_poll { void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn); void vhost_work_queue(struct vhost_dev *dev, struct vhost_work *work); bool vhost_has_work(struct vhost_dev *dev); +int vhost_workers_create(struct vhost_dev *dev, int new_num_workers); void vhost_poll_init(struct vhost_poll *poll, vhost_work_fn_t fn, __poll_t mask, struct vhost_dev *dev); @@ -152,8 +159,8 @@ struct vhost_dev { int nvqs; int max_nvqs; struct eventfd_ctx *log_ctx; - struct llist_head work_list; - struct task_struct *worker; + struct vhost_worker **workers; + int num_workers; struct vhost_iotlb *umem; struct vhost_iotlb *iotlb; spinlock_t iotlb_lock; @@ -175,6 +182,7 @@ int vhost_dev_init(struct vhost_dev *dev, struct vhost_virtqueue **vqs, int byte_weight, bool use_worker, int (*msg_handler)(struct vhost_dev *dev, struct vhost_iotlb_msg *msg)); +int vhost_vq_set_worker(struct vhost_virtqueue *vq, int worker_id); long vhost_dev_set_owner(struct vhost_dev *dev); bool vhost_dev_has_owner(struct vhost_dev *dev); long vhost_dev_check_owner(struct vhost_dev *); From patchwork Wed Oct 7 20:54:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 11821815 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E52A617D1 for ; Wed, 7 Oct 2020 20:55:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C0E272083B for ; Wed, 7 Oct 2020 20:55:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="K1jWNimz" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728798AbgJGUzh (ORCPT ); Wed, 7 Oct 2020 16:55:37 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:48268 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727816AbgJGUza (ORCPT ); Wed, 7 Oct 2020 16:55:30 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KoBsQ028846; Wed, 7 Oct 2020 20:55:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=qyACbUygmzt92DFQ1NzUiVRE5GH5AUspf3Vl9ti3Q4Y=; b=K1jWNimzdBmO8gLFe9tKH4udtJM6eeBWD/p/49FJRRS2quKT2wxEFrtJLqN/mb6ujYPA MJTmlIN5G6DMtJYGCd9SgJ5erdb565IHoeC3Qk5DcbMUSHLtpHoCmO3qOpYQLgq2enLL KqW9bOQ/62uyjmpbDvPDTpb9ms4Pfz5YLdvB/LBR2PJ9csM4zKKJ09LwHSobH2VItE6h KBCl55OfYYncZANOtE2hSZBa3hIOv9kjsYiRrAPgft8GoAJzrp+8TeaYZDjxX0LW0Q/W ngBl53/4e2J9j8tJDxsA5xqlpESLo6AOlDPKhJOxJMLEhnZsAgMWK66/8bbqN0imN8oS qg== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2130.oracle.com with ESMTP id 33xetb4ef9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 07 Oct 2020 20:55:26 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KpLQt166325; Wed, 7 Oct 2020 20:55:25 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserp3030.oracle.com with ESMTP id 33y2vpyv39-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 07 Oct 2020 20:55:25 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 097KtOje021076; Wed, 7 Oct 2020 20:55:24 GMT Received: from ol2.localdomain (/73.88.28.6) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Oct 2020 13:55:24 -0700 From: Mike Christie To: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, pbonzini@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Subject: [PATCH 13/16] vhost poll: fix coding style Date: Wed, 7 Oct 2020 15:54:58 -0500 Message-Id: <1602104101-5592-14-git-send-email-michael.christie@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> References: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 malwarescore=0 suspectscore=0 spamscore=0 phishscore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 mlxscore=0 clxscore=1015 priorityscore=1501 adultscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 malwarescore=0 suspectscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org We use like 3 coding styles in this struct. Switch to just tabs. Signed-off-by: Mike Christie --- drivers/vhost/vhost.h | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h index a91be8a..2bbe85f 100644 --- a/drivers/vhost/vhost.h +++ b/drivers/vhost/vhost.h @@ -34,12 +34,12 @@ struct vhost_worker { /* Poll a file (eventfd or socket) */ /* Note: there's nothing vhost specific about this structure. */ struct vhost_poll { - poll_table table; - wait_queue_head_t *wqh; - wait_queue_entry_t wait; - struct vhost_work work; - __poll_t mask; - struct vhost_dev *dev; + poll_table table; + wait_queue_head_t *wqh; + wait_queue_entry_t wait; + struct vhost_work work; + __poll_t mask; + struct vhost_dev *dev; }; void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn); From patchwork Wed Oct 7 20:54:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 11821813 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6B9A76CA for ; Wed, 7 Oct 2020 20:55:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4A1802083B for ; Wed, 7 Oct 2020 20:55:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="lEgeUPQK" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728692AbgJGUzd (ORCPT ); Wed, 7 Oct 2020 16:55:33 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:43366 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728816AbgJGUzb (ORCPT ); Wed, 7 Oct 2020 16:55:31 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KnUFr038310; Wed, 7 Oct 2020 20:55:27 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=nMSyPzjtY25TB3KiQcKSoQOhg9Eo04FswU4uDxvkkc0=; b=lEgeUPQKGy0cbXFcG3j+dI0sxbw5tMb65C2ZXgRv8cFvMmbbVDuAZ0P4Yr+k/BNGGtsp lcVcBTVnqXETrM3q2zoORGoRQIgk/PS3wFHIPblNlfterwRQlBBZWj/7efx2tAhcYVfV rQt2W5HvIAIU1S9hjwlPnQxjh0Acl3yLxB1moq5c4PgVddNnTtZwlG6rOgHZfyk0wxE4 h6Y4to0lby4jmHwM4MLr8plc9B9BU9QOabolz26YyErh2VcHXK58ARkOK9ff8SOqBysp GGOfT7VZ1GGdFzNndnaPe2CbIPB1pPlovFqwlDCCNO27ujT4nfJsXYzDPpsuEKwCpbQE ww== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by userp2120.oracle.com with ESMTP id 33xhxn475t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 07 Oct 2020 20:55:27 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KpLYp166298; Wed, 7 Oct 2020 20:55:26 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserp3030.oracle.com with ESMTP id 33y2vpyv3u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 07 Oct 2020 20:55:26 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 097KtOin021079; Wed, 7 Oct 2020 20:55:24 GMT Received: from ol2.localdomain (/73.88.28.6) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Oct 2020 13:55:24 -0700 From: Mike Christie To: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, pbonzini@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Subject: [PATCH 14/16] vhost: poll support support multiple workers Date: Wed, 7 Oct 2020 15:54:59 -0500 Message-Id: <1602104101-5592-15-git-send-email-michael.christie@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> References: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 malwarescore=0 suspectscore=2 spamscore=0 phishscore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 bulkscore=0 impostorscore=0 lowpriorityscore=0 suspectscore=2 phishscore=0 mlxlogscore=999 adultscore=0 clxscore=1015 spamscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org The final patches are going to have vhost scsi create a vhost worker per IO vq. This patch converts the poll code to poll and queue work on the worker that is tied to the vq (in this patch we maintain the old behavior where all vqs use a single worker). For drivers that do not convert over to the multiple worker support or for the case where the user just does not want to allocate the resources then we maintain support for the single worker case. Signed-off-by: Mike Christie Reported-by: kernel test robot --- drivers/vhost/net.c | 6 ++++-- drivers/vhost/vhost.c | 25 ++++++++++++++++++++++--- drivers/vhost/vhost.h | 9 +++++++-- 3 files changed, 33 insertions(+), 7 deletions(-) diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index fce46f0..a316ed0 100644 --- a/drivers/vhost/net.c +++ b/drivers/vhost/net.c @@ -1322,8 +1322,10 @@ static int vhost_net_open(struct inode *inode, struct file *f) NULL)) goto err_dev_init; - vhost_poll_init(n->poll + VHOST_NET_VQ_TX, handle_tx_net, EPOLLOUT, dev); - vhost_poll_init(n->poll + VHOST_NET_VQ_RX, handle_rx_net, EPOLLIN, dev); + vhost_poll_init(n->poll + VHOST_NET_VQ_TX, handle_tx_net, EPOLLOUT, dev, + vqs[VHOST_NET_VQ_TX]); + vhost_poll_init(n->poll + VHOST_NET_VQ_RX, handle_rx_net, EPOLLIN, dev, + vqs[VHOST_NET_VQ_RX]); f->private_data = n; n->page_frag.page = NULL; diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index e46b39e..f0038d6 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -187,13 +187,15 @@ void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn) /* Init poll structure */ void vhost_poll_init(struct vhost_poll *poll, vhost_work_fn_t fn, - __poll_t mask, struct vhost_dev *dev) + __poll_t mask, struct vhost_dev *dev, + struct vhost_virtqueue *vq) { init_waitqueue_func_entry(&poll->wait, vhost_poll_wakeup); init_poll_funcptr(&poll->table, vhost_poll_func); poll->mask = mask; poll->dev = dev; poll->wqh = NULL; + poll->vq = vq; vhost_work_init(&poll->work, fn); } @@ -283,6 +285,12 @@ void vhost_poll_flush(struct vhost_poll *poll) } EXPORT_SYMBOL_GPL(vhost_poll_flush); +void vhost_vq_work_queue(struct vhost_virtqueue *vq, struct vhost_work *work) +{ + vhost_work_queue_on(vq->dev, work, vq->worker_id); +} +EXPORT_SYMBOL_GPL(vhost_vq_work_queue); + /* A lockless hint for busy polling code to exit the loop */ bool vhost_has_work(struct vhost_dev *dev) { @@ -302,7 +310,7 @@ bool vhost_has_work(struct vhost_dev *dev) void vhost_poll_queue(struct vhost_poll *poll) { - vhost_work_queue(poll->dev, &poll->work); + vhost_vq_work_queue(poll->vq, &poll->work); } EXPORT_SYMBOL_GPL(vhost_poll_queue); @@ -488,11 +496,12 @@ static void __vhost_vq_init(struct vhost_dev *dev, struct vhost_virtqueue *vq) vq->indirect = NULL; vq->heads = NULL; vq->dev = dev; + vq->worker_id = 0; mutex_init(&vq->mutex); vhost_vq_reset(dev, vq); if (vq->handle_kick) - vhost_poll_init(&vq->poll, vq->handle_kick, EPOLLIN, dev); + vhost_poll_init(&vq->poll, vq->handle_kick, EPOLLIN, dev, vq); } static int vhost_vq_init(struct vhost_dev *dev, int vq_idx) @@ -643,6 +652,16 @@ static void vhost_detach_mm(struct vhost_dev *dev) dev->mm = NULL; } +int vhost_vq_set_worker(struct vhost_virtqueue *vq, int worker_id) +{ + if (vhost_vq_get_backend(vq)) + return -EBUSY; + + vq->worker_id = worker_id; + return 0; +} +EXPORT_SYMBOL_GPL(vhost_vq_set_worker); + static void vhost_workers_free(struct vhost_dev *dev) { struct vhost_worker *worker; diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h index 2bbe85f..6e9643d 100644 --- a/drivers/vhost/vhost.h +++ b/drivers/vhost/vhost.h @@ -16,6 +16,7 @@ #include struct vhost_work; +struct vhost_virtqueue *vq; typedef void (*vhost_work_fn_t)(struct vhost_work *work); #define VHOST_WORK_QUEUED 1 @@ -32,7 +33,6 @@ struct vhost_worker { }; /* Poll a file (eventfd or socket) */ -/* Note: there's nothing vhost specific about this structure. */ struct vhost_poll { poll_table table; wait_queue_head_t *wqh; @@ -40,15 +40,19 @@ struct vhost_poll { struct vhost_work work; __poll_t mask; struct vhost_dev *dev; + struct vhost_virtqueue *vq; }; +int vhost_vq_set_worker(struct vhost_virtqueue *vq, int worker_id); +void vhost_vq_work_queue(struct vhost_virtqueue *vq, struct vhost_work *work); void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn); void vhost_work_queue(struct vhost_dev *dev, struct vhost_work *work); bool vhost_has_work(struct vhost_dev *dev); int vhost_workers_create(struct vhost_dev *dev, int new_num_workers); void vhost_poll_init(struct vhost_poll *poll, vhost_work_fn_t fn, - __poll_t mask, struct vhost_dev *dev); + __poll_t mask, struct vhost_dev *dev, + struct vhost_virtqueue *vq); int vhost_poll_start(struct vhost_poll *poll, struct file *file); void vhost_poll_stop(struct vhost_poll *poll); void vhost_poll_flush(struct vhost_poll *poll); @@ -92,6 +96,7 @@ struct vhost_virtqueue { struct vhost_poll poll; + int worker_id; /* The routine to call when the Guest pings us, or timeout. */ vhost_work_fn_t handle_kick; From patchwork Wed Oct 7 20:55:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 11821801 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DBB1917D1 for ; Wed, 7 Oct 2020 20:55:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B5E202083B for ; Wed, 7 Oct 2020 20:55:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="QEVHlOFL" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728820AbgJGUze (ORCPT ); Wed, 7 Oct 2020 16:55:34 -0400 Received: from aserp2130.oracle.com ([141.146.126.79]:48280 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728810AbgJGUza (ORCPT ); Wed, 7 Oct 2020 16:55:30 -0400 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KoDPg028922; Wed, 7 Oct 2020 20:55:27 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=jlUg/PV7EzRfA2y9oQPeRJqjyI/8yN/GQb1Z5wEkx5w=; b=QEVHlOFLYLETHHB3JiTZAWEICt/jMWj9q9beB6sACi6DWOZP2j8+Uc8o0YVi0U3N3E4r z6sgaN1Gg1Z1Y5zYojz9xDX5sV5ZA/rSS1yKQOjJ9nXnltsAcDtCkQr0DV9oUL8h7ECa az8hqlVdUcCdAVGpY4Jh+6cIdBshsDcv62MfynJoLmw4tX5sfQvoQDRayX/SFFrV7OPs Xa7QnQoYPIg1GFQljTGw6ohNItOkD1cjrvTRYW9bM5ea0fzCmEFQWOdr80Ew4pC0TFi8 7uhRcIeFTFTeLMw7nAamEyXFQA5e9EXDh1GhqK9EKKZPtknwWypvho6hRlx5LMzb7yNC eQ== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2130.oracle.com with ESMTP id 33xetb4efe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 07 Oct 2020 20:55:27 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KpCR3028987; Wed, 7 Oct 2020 20:55:26 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userp3020.oracle.com with ESMTP id 33yyjhpumr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 07 Oct 2020 20:55:26 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 097KtPNM021083; Wed, 7 Oct 2020 20:55:25 GMT Received: from ol2.localdomain (/73.88.28.6) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Oct 2020 13:55:25 -0700 From: Mike Christie To: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, pbonzini@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Subject: [PATCH 15/16] vhost scsi: make completion per vq Date: Wed, 7 Oct 2020 15:55:00 -0500 Message-Id: <1602104101-5592-16-git-send-email-michael.christie@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> References: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 bulkscore=0 phishscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 mlxscore=0 clxscore=1015 priorityscore=1501 adultscore=0 mlxlogscore=999 phishscore=0 impostorscore=0 malwarescore=0 suspectscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org In the last patches we are going to have a worker thread per IO vq. This patch separates the scsi cmd completion code paths so we can complete cmds based on their vq instead of having all cmds complete on the same worker thread. Signed-off-by: Mike Christie --- drivers/vhost/scsi.c | 48 +++++++++++++++++++++++++----------------------- 1 file changed, 25 insertions(+), 23 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index b348e9c..4309f97 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -178,6 +178,7 @@ enum { struct vhost_scsi_virtqueue { struct vhost_virtqueue vq; + struct vhost_scsi *vs; /* * Reference counting for inflight reqs, used for flush operation. At * each time, one reference tracks new commands submitted, while we @@ -192,6 +193,9 @@ struct vhost_scsi_virtqueue { struct vhost_scsi_cmd *scsi_cmds; struct sbitmap scsi_tags; int max_cmds; + + struct vhost_work completion_work; + struct llist_head completion_list; }; struct vhost_scsi { @@ -202,9 +206,6 @@ struct vhost_scsi { struct vhost_dev dev; struct vhost_scsi_virtqueue vqs[VHOST_SCSI_MAX_VQ]; - struct vhost_work vs_completion_work; /* cmd completion work item */ - struct llist_head vs_completion_list; /* cmd completion queue */ - struct vhost_work vs_event_work; /* evt injection work item */ struct llist_head vs_event_list; /* evt injection queue */ @@ -380,10 +381,11 @@ static void vhost_scsi_release_cmd(struct se_cmd *se_cmd) } else { struct vhost_scsi_cmd *cmd = container_of(se_cmd, struct vhost_scsi_cmd, tvc_se_cmd); - struct vhost_scsi *vs = cmd->tvc_vhost; + struct vhost_scsi_virtqueue *svq = container_of(cmd->tvc_vq, + struct vhost_scsi_virtqueue, vq); - llist_add(&cmd->tvc_completion_list, &vs->vs_completion_list); - vhost_work_queue(&vs->dev, &vs->vs_completion_work); + llist_add(&cmd->tvc_completion_list, &svq->completion_list); + vhost_vq_work_queue(cmd->tvc_vq, &svq->completion_work); } } @@ -545,18 +547,17 @@ static void vhost_scsi_evt_work(struct vhost_work *work) */ static void vhost_scsi_complete_cmd_work(struct vhost_work *work) { - struct vhost_scsi *vs = container_of(work, struct vhost_scsi, - vs_completion_work); - DECLARE_BITMAP(signal, VHOST_SCSI_MAX_VQ); + struct vhost_scsi_virtqueue *svq = container_of(work, + struct vhost_scsi_virtqueue, completion_work); struct virtio_scsi_cmd_resp v_rsp; struct vhost_scsi_cmd *cmd, *t; struct llist_node *llnode; struct se_cmd *se_cmd; struct iov_iter iov_iter; - int ret, vq; + bool signal = false; + int ret; - bitmap_zero(signal, VHOST_SCSI_MAX_VQ); - llnode = llist_del_all(&vs->vs_completion_list); + llnode = llist_del_all(&svq->completion_list); llist_for_each_entry_safe(cmd, t, llnode, tvc_completion_list) { se_cmd = &cmd->tvc_se_cmd; @@ -576,21 +577,16 @@ static void vhost_scsi_complete_cmd_work(struct vhost_work *work) cmd->tvc_in_iovs, sizeof(v_rsp)); ret = copy_to_iter(&v_rsp, sizeof(v_rsp), &iov_iter); if (likely(ret == sizeof(v_rsp))) { - struct vhost_scsi_virtqueue *q; + signal = true; vhost_add_used(cmd->tvc_vq, cmd->tvc_vq_desc, 0); - q = container_of(cmd->tvc_vq, struct vhost_scsi_virtqueue, vq); - vq = q - vs->vqs; - __set_bit(vq, signal); } else pr_err("Faulted on virtio_scsi_cmd_resp\n"); vhost_scsi_release_cmd_res(se_cmd); } - vq = -1; - while ((vq = find_next_bit(signal, VHOST_SCSI_MAX_VQ, vq + 1)) - < VHOST_SCSI_MAX_VQ) - vhost_signal(&vs->dev, &vs->vqs[vq].vq); + if (signal) + vhost_signal(&svq->vs->dev, &svq->vq); } static struct vhost_scsi_cmd * @@ -1805,6 +1801,7 @@ static int vhost_scsi_set_features(struct vhost_scsi *vs, u64 features) static int vhost_scsi_open(struct inode *inode, struct file *f) { + struct vhost_scsi_virtqueue *svq; struct vhost_scsi *vs; struct vhost_virtqueue **vqs; int r = -ENOMEM, i; @@ -1820,7 +1817,6 @@ static int vhost_scsi_open(struct inode *inode, struct file *f) if (!vqs) goto err_vqs; - vhost_work_init(&vs->vs_completion_work, vhost_scsi_complete_cmd_work); vhost_work_init(&vs->vs_event_work, vhost_scsi_evt_work); vs->vs_events_nr = 0; @@ -1831,8 +1827,14 @@ static int vhost_scsi_open(struct inode *inode, struct file *f) vs->vqs[VHOST_SCSI_VQ_CTL].vq.handle_kick = vhost_scsi_ctl_handle_kick; vs->vqs[VHOST_SCSI_VQ_EVT].vq.handle_kick = vhost_scsi_evt_handle_kick; for (i = VHOST_SCSI_VQ_IO; i < VHOST_SCSI_MAX_VQ; i++) { - vqs[i] = &vs->vqs[i].vq; - vs->vqs[i].vq.handle_kick = vhost_scsi_handle_kick; + svq = &vs->vqs[i]; + + vqs[i] = &svq->vq; + svq->vs = vs; + init_llist_head(&svq->completion_list); + vhost_work_init(&svq->completion_work, + vhost_scsi_complete_cmd_work); + svq->vq.handle_kick = vhost_scsi_handle_kick; } /* From patchwork Wed Oct 7 20:55:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 11821811 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DF02C17D1 for ; Wed, 7 Oct 2020 20:55:36 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BC5D12083B for ; Wed, 7 Oct 2020 20:55:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="Tl5yKk7X" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728819AbgJGUzd (ORCPT ); Wed, 7 Oct 2020 16:55:33 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:35250 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728818AbgJGUzb (ORCPT ); Wed, 7 Oct 2020 16:55:31 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KoCJr113276; Wed, 7 Oct 2020 20:55:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : in-reply-to : references; s=corp-2020-01-29; bh=TxaIUJd6h4H8huBw9neYsTBcQJ2+geI1a5MAf+wgTwk=; b=Tl5yKk7XcjoaHsGh03RwJyJvP2NwwuHToewkEDB73uUOwXDyzJOekSpE9agfU3DGpDDh UhcZsvb63xe/de1Xqh+2d29XWdmRndV03lcOorKl5JZ4ox8lWO2/205pk8BXKEAk5Onc VrfiKduzWQFPgTm2ei63/I61tdQu5CsdVWPOgfc6XrdFb2Q4L6RGz1QPa6AjyyILogf4 gg+0D+YKzH+lJTeZA7lEJAPFssR8oCzHVhbl2z9/OO7tcxLIA+K/Wj8hM1k8drCTC0qf MVmP/e6d30Rj43L086WhOxOzXT+J9qBRUa2KrGlxlxRVnLaMnCVCdzmBgjvCiGVm/Sr0 7w== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2120.oracle.com with ESMTP id 33ym34scr5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 07 Oct 2020 20:55:27 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 097KpLpV166381; Wed, 7 Oct 2020 20:55:27 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserp3030.oracle.com with ESMTP id 33y2vpyv4j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 07 Oct 2020 20:55:27 +0000 Received: from abhmp0005.oracle.com (abhmp0005.oracle.com [141.146.116.11]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 097KtPNV032127; Wed, 7 Oct 2020 20:55:26 GMT Received: from ol2.localdomain (/73.88.28.6) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 07 Oct 2020 13:55:25 -0700 From: Mike Christie To: martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, pbonzini@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Subject: [PATCH 16/16] vhost scsi: multiple worker support Date: Wed, 7 Oct 2020 15:55:01 -0500 Message-Id: <1602104101-5592-17-git-send-email-michael.christie@oracle.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> References: <1602104101-5592-1-git-send-email-michael.christie@oracle.com> X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 malwarescore=0 suspectscore=0 spamscore=0 phishscore=0 bulkscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 X-Proofpoint-Virus-Version: vendor=nai engine=6000 definitions=9767 signatures=668680 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 impostorscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 clxscore=1015 bulkscore=0 spamscore=0 malwarescore=0 phishscore=0 suspectscore=0 adultscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2006250000 definitions=main-2010070134 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org Create a vhost_worker per IO vq. When using a more than 2 vqs and/or multiple LUNs per vhost-scsi dev, we hit a bottleneck with the single worker. The problem is that we want to start and complete all vqs and all LUNs from the same thread. Combine with the previous patches that allow us to add more than 2 vqs, we see a IOPs workloads (like 50/50 randrw 4K IOs) go from 150K to 400K where the native device is 500K. For the lio rd_mcp backend, we see IOPs go for from 400K to 600K. Signed-off-by: Mike Christie --- drivers/vhost/scsi.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 4309f97..e5f73c1 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -1624,6 +1624,22 @@ static int vhost_scsi_setup_vq_cmds(struct vhost_virtqueue *vq, int max_cmds) memcpy(vs->vs_vhost_wwpn, t->vhost_wwpn, sizeof(vs->vs_vhost_wwpn)); + /* + * For compat, have the evt and ctl vqs share worker0 with + * the first IO vq like is setup as default already. Any + * addition vqs will get their own worker. + * + * Note: if we fail later, then the vhost_dev_cleanup call on + * release() will clean up all the workers. + */ + ret = vhost_workers_create(&vs->dev, + vs->dev.nvqs - VHOST_SCSI_VQ_IO); + if (ret) { + pr_err("Could not create vhost-scsi workers. Error %d.", + ret); + goto undepend; + } + for (i = VHOST_SCSI_VQ_IO; i < VHOST_SCSI_MAX_VQ; i++) { vq = &vs->vqs[i].vq; if (!vq->initialized) @@ -1631,6 +1647,7 @@ static int vhost_scsi_setup_vq_cmds(struct vhost_virtqueue *vq, int max_cmds) if (vhost_scsi_setup_vq_cmds(vq, vq->num)) goto destroy_vq_cmds; + vhost_vq_set_worker(vq, i - VHOST_SCSI_VQ_IO); } for (i = 0; i < VHOST_SCSI_MAX_VQ; i++) {