From patchwork Fri Nov 9 02:39:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Farman X-Patchwork-Id: 10675263 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DD85115A6 for ; Fri, 9 Nov 2018 02:40:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CF85E2EB52 for ; Fri, 9 Nov 2018 02:40:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C3E862EB71; Fri, 9 Nov 2018 02:40:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 49BBF2EB52 for ; Fri, 9 Nov 2018 02:40:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727883AbeKIMSu (ORCPT ); Fri, 9 Nov 2018 07:18:50 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:52002 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727445AbeKIMSl (ORCPT ); Fri, 9 Nov 2018 07:18:41 -0500 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id wA92XUVL005551 for ; Thu, 8 Nov 2018 21:40:09 -0500 Received: from e06smtp03.uk.ibm.com (e06smtp03.uk.ibm.com [195.75.94.99]) by mx0b-001b2d01.pphosted.com with ESMTP id 2nmwmrrawy-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 08 Nov 2018 21:40:09 -0500 Received: from localhost by e06smtp03.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 9 Nov 2018 02:40:07 -0000 Received: from b06cxnps4076.portsmouth.uk.ibm.com (9.149.109.198) by e06smtp03.uk.ibm.com (192.168.101.133) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Fri, 9 Nov 2018 02:40:05 -0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps4076.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id wA92e37B55050368 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Fri, 9 Nov 2018 02:40:03 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 951FFA4057; Fri, 9 Nov 2018 02:40:03 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7BFBCA4051; Fri, 9 Nov 2018 02:40:03 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTPS; Fri, 9 Nov 2018 02:40:03 +0000 (GMT) Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 4958) id 0310E20F64F; Fri, 9 Nov 2018 03:40:03 +0100 (CET) From: Eric Farman To: Cornelia Huck , Pierre Morel Cc: Eric Farman , linux-s390@vger.kernel.org, kvm@vger.kernel.org, Halil Pasic , Farhan Ali , "Jason J . Herne" Subject: [RFC PATCH v1 08/10] s390/cio: Split pfn_array_alloc_pin into pieces Date: Fri, 9 Nov 2018 03:39:35 +0100 X-Mailer: git-send-email 2.16.4 In-Reply-To: <20181109023937.96105-1-farman@linux.ibm.com> References: <20181109023937.96105-1-farman@linux.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18110902-0012-0000-0000-000002C78162 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18110902-0013-0000-0000-000020FBD302 Message-Id: <20181109023937.96105-9-farman@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-11-08_13:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=954 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1811090022 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The pfn_array_alloc_pin routine is doing too much. Today, it does the alloc of the pfn_array struct and its member arrays, builds the iova address lists out of a contiguous piece of guest memory, and asks vfio to pin the resulting pages. While the input iova variable points to a contiguous piece of memory, the pfn_array, uh, array would permit a non-contiguous range of addresses such as those provided by an IDA List. By breaking pfn_array_alloc_pin into its component pieces, we will be in a good place to share this code regardless of what pfn_array contains. Signed-off-by: Eric Farman --- drivers/s390/cio/vfio_ccw_cp.c | 96 +++++++++++++++++++----------------------- 1 file changed, 44 insertions(+), 52 deletions(-) diff --git a/drivers/s390/cio/vfio_ccw_cp.c b/drivers/s390/cio/vfio_ccw_cp.c index 5f16aa88c629..d01e146289fb 100644 --- a/drivers/s390/cio/vfio_ccw_cp.c +++ b/drivers/s390/cio/vfio_ccw_cp.c @@ -49,37 +49,32 @@ struct ccwchain { struct pfn_array_table *ch_pat; }; -/* - * pfn_array_alloc_pin() - alloc memory for PFNs, then pin user pages in memory - * @pa: pfn_array on which to perform the operation - * @mdev: the mediated device to perform pin/unpin operations - * @iova: target guest physical address - * @len: number of pages that should be pinned from @iova - * - * Attempt to allocate memory for PFNs, and pin user pages in memory. - * - * Usage of pfn_array: - * We expect (pa_nr == 0) and (pa_iova_pfn == NULL), any field in - * this structure will be filled in by this function. - * - * Returns: - * Number of pages pinned on success. - * If @pa->pa_nr is not 0, or @pa->pa_iova_pfn is not NULL initially, - * returns -EINVAL. - * If no pages were pinned, returns -errno. - */ -static int pfn_array_alloc_pin(struct pfn_array *pa, struct device *mdev, - u64 iova, unsigned int len) +static int pfn_array_pin(struct pfn_array *pa, struct device *mdev) { - int i, ret = 0; + int ret = 0; + + ret = vfio_pin_pages(mdev, pa->pa_iova_pfn, pa->pa_nr, + IOMMU_READ | IOMMU_WRITE, pa->pa_pfn); + + if (ret > 0 && ret != pa->pa_nr) { + vfio_unpin_pages(mdev, pa->pa_iova_pfn, ret); + ret = -EINVAL; + } + if (ret < 0) + pa->pa_iova = 0; + + return ret; +} + +static int pfn_array_alloc(struct pfn_array *pa, unsigned int len) +{ if (!len) return 0; if (pa->pa_nr || pa->pa_iova_pfn) return -EINVAL; - pa->pa_iova = iova; pa->pa_nr = len; pa->pa_iova_pfn = kcalloc(pa->pa_nr, @@ -90,35 +85,14 @@ static int pfn_array_alloc_pin(struct pfn_array *pa, struct device *mdev, return -ENOMEM; pa->pa_pfn = pa->pa_iova_pfn + pa->pa_nr; - pa->pa_iova_pfn[0] = pa->pa_iova >> PAGE_SHIFT; - for (i = 1; i < pa->pa_nr; i++) - pa->pa_iova_pfn[i] = pa->pa_iova_pfn[i - 1] + 1; - - ret = vfio_pin_pages(mdev, pa->pa_iova_pfn, pa->pa_nr, - IOMMU_READ | IOMMU_WRITE, pa->pa_pfn); - - if (ret < 0) { - goto err_out; - } else if (ret > 0 && ret != pa->pa_nr) { - vfio_unpin_pages(mdev, pa->pa_iova_pfn, ret); - ret = -EINVAL; - goto err_out; - } - - return ret; - -err_out: - pa->pa_nr = 0; - kfree(pa->pa_iova_pfn); - pa->pa_iova_pfn = NULL; - - return ret; + return 0; } /* Unpin the pages before releasing the memory. */ static void pfn_array_unpin_free(struct pfn_array *pa, struct device *mdev) { - vfio_unpin_pages(mdev, pa->pa_iova_pfn, pa->pa_nr); + if (pa->pa_iova) + vfio_unpin_pages(mdev, pa->pa_iova_pfn, pa->pa_nr); pa->pa_nr = 0; kfree(pa->pa_iova_pfn); } @@ -206,11 +180,21 @@ static long copy_from_iova(struct device *mdev, int i, ret; unsigned long l, m; - ret = pfn_array_alloc_pin(&pa, mdev, iova, - idal_nr_words((void *)(iova), n)); - if (ret <= 0) + ret = pfn_array_alloc(&pa, idal_nr_words((void *)(iova), n)); + if (ret < 0) return ret; + pa.pa_iova = iova; + pa.pa_iova_pfn[0] = pa.pa_iova >> PAGE_SHIFT; + for (i = 1; i < pa.pa_nr; i++) + pa.pa_iova_pfn[i] = pa.pa_iova_pfn[i - 1] + 1; + + ret = pfn_array_pin(&pa, mdev); + if (ret <= 0) { + pfn_array_unpin_free(&pa, mdev); + return ret; + } + l = n; for (i = 0; i < pa.pa_nr; i++) { from = pa.pa_pfn[i] << PAGE_SHIFT; @@ -454,6 +438,7 @@ static int ccwchain_fetch_ccw(struct ccwchain *chain, { struct ccw1 *ccw; struct pfn_array_table *pat; + struct pfn_array *pa; unsigned long *idaws; u64 idaw_iova; unsigned int idaw_nr, idaw_len; @@ -515,11 +500,18 @@ static int ccwchain_fetch_ccw(struct ccwchain *chain, for (i = 0; i < idaw_nr; i++) { idaw_iova = *(idaws + i); + pa = pat->pat_pa + i; - ret = pfn_array_alloc_pin(pat->pat_pa + i, cp->mdev, - idaw_iova, 1); + ret = pfn_array_alloc(pa, 1); if (ret < 0) goto out_free_idaws; + + pa->pa_iova = idaws[0]; + pa->pa_iova_pfn[0] = idaws[i] >> PAGE_SHIFT; + + ret = pfn_array_pin(pa, cp->mdev); + if (ret <= 0) + goto out_free_idaws; } ccw->cda = virt_to_phys(idaws);