From patchwork Mon Feb 1 09:26:18 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: vijayak@caviumnetworks.com X-Patchwork-Id: 8177191 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id DF00A9F4DD for ; Mon, 1 Feb 2016 09:33:04 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id B25BC2042A for ; Mon, 1 Feb 2016 09:33:03 +0000 (UTC) Received: from lists.xen.org (lists.xenproject.org [50.57.142.19]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5AB782041E for ; Mon, 1 Feb 2016 09:33:02 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aQAoT-0006pM-0T; Mon, 01 Feb 2016 09:30:25 +0000 Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aQAmj-0006gk-OH for xen-devel@lists.xen.org; Mon, 01 Feb 2016 09:28:37 +0000 Received: from [85.158.143.35] by server-1.bemta-4.messagelabs.com id D1/7A-09708-5452FA65; Mon, 01 Feb 2016 09:28:37 +0000 X-Env-Sender: Vijaya.Kumar@caviumnetworks.com X-Msg-Ref: server-10.tower-21.messagelabs.com!1454318909!13218867!1 X-Originating-IP: [207.46.100.63] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 7.35.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 2755 invoked from network); 1 Feb 2016 09:28:35 -0000 Received: from mail-by2on0063.outbound.protection.outlook.com (HELO na01-by2-obe.outbound.protection.outlook.com) (207.46.100.63) by server-10.tower-21.messagelabs.com with AES256-SHA256 encrypted SMTP; 1 Feb 2016 09:28:35 -0000 Received: from cavium-Vostro-2520.caveonetworks.com (111.93.218.67) by BN3PR0701MB1688.namprd07.prod.outlook.com (10.163.39.154) with Microsoft SMTP Server (TLS) id 15.1.396.15; Mon, 1 Feb 2016 09:28:23 +0000 From: To: , , , , , Date: Mon, 1 Feb 2016 14:56:18 +0530 Message-ID: <1454318798-31913-9-git-send-email-vijayak@caviumnetworks.com> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1454318798-31913-1-git-send-email-vijayak@caviumnetworks.com> References: <1454318798-31913-1-git-send-email-vijayak@caviumnetworks.com> MIME-Version: 1.0 X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: BM1PR01CA0026.INDPRD01.PROD.OUTLOOK.COM (25.163.198.161) To BN3PR0701MB1688.namprd07.prod.outlook.com (25.163.39.154) X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1688; 2:iKNxmDPlnTLrLOx8/GC4kimmuAf+cOs2LnuqYlAd/P18KjcKhevMaj5kvBPL0Sx323eS64qSR6tSyhofCLSuH40BI2yy325LY2rE5oOu6zPmWhVrQcen9UsdX/4Dc+zBowuvWaPy6q37v/0vd4S/Ug==; 3:yPeKWJJE1GLT9uLW4gTannrZNpj1iuNvRv17A7HsFLq+Xyrhw0pAtc0J9XyS3rXnUbg8Gv2uTTn8RFmEs2LBugKK+H+IFBaNP7rMrH6bCg81X3UJkMBI3DFQhJ3MqSV5; 25:mJc+EA3eyDslNhBOgh1lmyFyJhe2LHxTosSJthVnevXlYtGzX4krA3OK5t8KekdO4VXmwqAMcpVS3lcWN8TivZkUXBSIPdi4bTYrBvI3YtawGKsYjYy/gwNWdsEi6DMyqaf7assH8mREXhZsKwFdJAJ6HtcbVV/oNHuzrxMJeYGB3349s/DW77WT6v2oJItWxRqQ+C5nOL7KNFXiZiW4sPjDmadqTOG0FrrcEH/PEyWRFd7JaHFl4l49oO7WTsZj X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN3PR0701MB1688; X-MS-Office365-Filtering-Correlation-Id: fa5e25be-f76c-40b7-e2f6-08d32aea0a69 X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1688; 20:VNiaNIOo1pY7xT38D+ydRjW45z+rVrnEYFb7WN5OsiPOvF2K/1AvxZF4f5E6c2VY4KpjlxpKwCRzmhL8dSLXO/rS2bQO+4SXINFqR3nHQM451JLwONQ+QTvedK54qXDliVzVf1QmmpVHxhEQOYBp7QjTvYtmfeFOuyUdtlmWEKTMX4bcUNkXpmq0eDU/vTIVu21hMVBRVJyE6YOdWJ39SX/SEp3909MfUmJG9qs+mfYMXUNROCLKFAdFIoRKXxXzBqBgOs6sMbLvNLRP/k+gjEPfyZYjmvlWYywcumXWoES7areLdkN/QewxlklMbi9aqH26DrgU3ExEkGTGBmydvwJd6uPtUnXgn2tQJECHgDfsbaioqy2k0jKFv7TOUMqRwJ8BOaDEm6ZgUQV7oajhIIgdMr2lQEAcd/H79EbBgTfkxp1DmO9IHTeOXql5P1jxlhbRyii3pQFRgwikU3BJdI6j5/IwXtWB+FCdHbSTXtGJTGFIpx3+g/HPtbLamW/FRVa67SkVmm0hYwmep939yKogZWgcfqn4fqgAzeKte96iF5zZjmmZbxz3zei51Or+erRlVLTWbpw3yxtJekx33SuSelmsUHuw2bZNHPadfKA= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046); SRVR:BN3PR0701MB1688; BCL:0; PCL:0; RULEID:; SRVR:BN3PR0701MB1688; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1688; 4:KWJhmRgXyugRIemrAnvDVmUX05RVW3HxuOcIFiRR+TPhawEDg1EsqM4NNt9r4HcYXzsp0YAFXDbyusKpMG/vIv1yjzxVLiWpAd6RzArwrE1om4EiugSuWv+9CCw4LvqfinxwKBLW5XpFSMiERGwyINwqda2UiQgS6/HNG1UNGcMc3UFrGNHjbWEtpThIJuuzu0xNDvrX08cZGnnaAPQfJQPlgI/Q+cwDbdHeggXHrGk5dHcV+nIUczRYhd5w1KUM7kCxT+lifoxF3TzBD/UrmyngSOZyQsOnVxgoq8VTu197LG7MtaJnlXyfQDc0Y7ctz53e0jjd5OBhJfifTbei0ag2FjOKNzLUC+t8rjwR+OuLA4kln+Txt8hTdQijuCJi X-Forefront-PRVS: 0839D067E7 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(6009001)(3846002)(5009440100003)(4001430100002)(586003)(1096002)(76176999)(50986999)(36756003)(189998001)(92566002)(5004730100002)(2950100001)(229853001)(50226001)(87976001)(86152002)(107886002)(2201001)(5001960100002)(77096005)(53416004)(48376002)(19580395003)(19580405001)(5003940100001)(5001770100001)(42186005)(33646002)(66066001)(47776003)(2906002)(2876002)(4326007)(122386002)(3470700001)(5008740100001)(50466002)(40100003)(7099028)(7059030)(2101003); DIR:OUT; SFP:1101; SCL:1; SRVR:BN3PR0701MB1688; H:cavium-Vostro-2520.caveonetworks.com; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN3PR0701MB1688; 23:Axi/oH3+HflaywslF26hmc+BiWt7PMkT6DvUBQY?= =?us-ascii?Q?DG6AI9Aj+zJB21aVRUoOWMUijb/ztiv9kOk8XbwQJGR1oemspK519AclLdC/?= =?us-ascii?Q?8UuCXOBkdj0kWV6szXf4z50jm1KG1A6UkI8XrZkgxScvlUIJnwPf4mJd2g/V?= =?us-ascii?Q?CNWCd7CeF91pvh2E9kEq6cPXbMBKCjcMP3u5Gl/sNcgmTTZDJuHrgEswf6cd?= =?us-ascii?Q?kBEW9MLIWxWsqgyreDWNYy0WzY65jc11HGdKbeyCOqU+3RMdqEE37LT+Q3bS?= =?us-ascii?Q?zWnxr7P1cyDFWcwtooLq3s89F3vv3LtFh4U5DL4s0etI8CTVdU4wvgd0OzGv?= =?us-ascii?Q?obGfvBGIBVRV8KbvhmDkhgsx5HiFGPDOfCzUhUhTrFU8vBrR3p8Fqjw7rn1f?= =?us-ascii?Q?+1swBMDXBGUvE3VO3Kfrbvo24C4U1UK6zSE/Jz/qbLyTSdF2IqASMn/wl8jF?= =?us-ascii?Q?7YwSH9YnBr8RV9RMO+RPH4TBlrSxKycLDWCCjyqNYGQX/exoFyfn5sgUdFL5?= =?us-ascii?Q?rrvl8J99IK3A2lgiw1iwFuEwyJPM+t0ZH+a6kfpJ/wxkikCi/QFOtYIeGtPB?= =?us-ascii?Q?M1Y3lJDzF5ahgPC/nTZX5LAvGCbb05hSPxwRl9PrdPxs4bHSe8czySv2LQVU?= =?us-ascii?Q?JMwmcoUVvvAP2ea1evgv7WkdoBQM5e7nFjCtcVxzEF15eLtzFy+pwFvkBHp4?= =?us-ascii?Q?znIFCyTxfOchvhMRVfeI840an9dyzQhJ7CnSwGWh8y+i2BNCPjQwXf2FhmDl?= =?us-ascii?Q?MACFc0J9TOU7BkdXvVBT2/3uOwdPEpU7uV+/zculEm/Fu7kXNMAeUASbytW9?= =?us-ascii?Q?bai76yAxVOHCtT3Eia/Xvnuc0QoYiTd+jYWIY1YYW6j0allo7O9Lka/OUDzU?= =?us-ascii?Q?jurKv5CJBhfbsP7cnp/dAsoLUrUcIS5WIifhWaXLNXk/qZYaYqeSaw6ZSFRs?= =?us-ascii?Q?97BCyVVOnrXugddmI+hAU5uijaER12uZMe6EMW8TWGhEPQ8pS0tiTaPAgvZB?= =?us-ascii?Q?EXefnQZoVMGqjLNyxoGWbyZYTbNH2rB2W8qmfJqvBo5j3oMLoYJcac4ebKsr?= =?us-ascii?Q?Ylnoql8/lqDdj6HYxE0s3cB87l9lgXv/5/mG82FE0VK+SstWTTBVdzuxjhgk?= =?us-ascii?Q?SczVbTEgJSLkpwF6lpbyzS9L5B9K0vEWx?= X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1688; 5:2ovwLEMXQrnp0BzHCYK4DMix0aZPP3HP56mQ8KrfsaXruY/HWZQLvwuctjSrEwgLQQaHkI+NMFtHbS3dfVsD/VsHUd2Mfsks7NGG0OadKbt+SyYe2TndAazWBZKLAjsimS4lSmwbzwYzyGqLD1/tyQ==; 24:n91AWyVs2RESZXQVBeqaRWttgKgsUYgS7KBvCqj8BQeZh5sTerX49v/WJSe7aNQmksnvPNv3GPYoaDt/ADFBLVzJWNN75wi2/lfiDSvrRb0= SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Feb 2016 09:28:23.8510 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR0701MB1688 X-Mailman-Approved-At: Mon, 01 Feb 2016 09:30:20 +0000 Cc: Prasun.Kapoor@caviumnetworks.com, manish.jaggi@caviumnetworks.com, Vijaya Kumar K , vijay.kilari@gmail.com Subject: [Xen-devel] [PATCH v8 08/28] xen/arm: ITS: Add APIs to add and assign device X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Spam-Status: No, score=-4.2 required=5.0 tests=BAD_ENC_HEADER,BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Vijaya Kumar K Add APIs to add devices to RB-tree, assign and remove devices to domain. Signed-off-by: Vijaya Kumar K --- v8: - Merged its_free_msi_descs() with its_discard_lpis - Dropped extra spin_unlock() in its_assign_device() - Dropped check on device domain to be dom0 in assign_device() - device is assigned to domain in assign_device() instead of add_device() function - Replaced XENLOG_E_ERR with XENLOG_ERR v7: - Added check for domain in its_assign_device() to avoid assigning device to DomU - Added comments whereever requested - Called its_remove_device to remove from rb-tree when failed to add device - Changed dprintk to printk - Store domain pointer instead of domain id in its_device struct - Free msi_desc and reset irq_desc when lpis are discarded - Add function its_free_msi_descs() to free msi_desc v6: - Moved this patch #19 to patch #8 - Used col_map to store collection id - Use helper functions to update msi_desc members v5: - Removed its_detach_device API - Pass nr_ites as parameter to its_add_device v4: - Introduced helper to populate its_device struct - Fixed freeing of its_device memory - its_device struct holds domain id --- xen/arch/arm/gic-v3-its.c | 252 +++++++++++++++++++++++++++++++++++++++++ xen/include/asm-arm/gic-its.h | 6 + 2 files changed, 258 insertions(+) diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c index 205b610..c233950 100644 --- a/xen/arch/arm/gic-v3-its.c +++ b/xen/arch/arm/gic-v3-its.c @@ -145,6 +145,19 @@ static struct its_collection *dev_event_to_col(struct its_device *dev, return its->collections + dev->event_map.col_map[event]; } +static struct its_node *its_get_phys_node(struct dt_device_node *dt) +{ + struct its_node *its; + + list_for_each_entry(its, &its_nodes, entry) + { + if ( its->dt_node == dt ) + return its; + } + + return NULL; +} + /* RB-tree helpers for its_device */ static struct its_device *its_find_device(u32 devid) { @@ -544,6 +557,245 @@ static void its_lpi_free(struct its_device *dev) xfree(dev->event_map.col_map); } +static inline u32 its_get_plpi(struct its_device *dev, u32 event) +{ + return dev->event_map.lpi_base + event; +} + +static void its_discard_lpis(struct its_device *dev, u32 ids) +{ + u32 i; + struct irq_desc *desc; + + for ( i = 0; i < ids; i++ ) + { + its_send_discard(dev, i); + desc = irq_to_desc(its_get_plpi(dev, i)); + + spin_lock(&desc->lock); + irqdesc_set_lpi_event(desc, 0); + irqdesc_set_its_device(desc, NULL); + /* + * Here only msi_descs are cleaned. + * TODO: Clean irq_descs of this pLPIs. + */ + xfree(irq_get_msi_desc(desc)); + irq_set_msi_desc(desc, NULL); + spin_unlock(&desc->lock); + } +} + +static int its_alloc_device_irq(struct its_device *dev, u32 *hwirq) +{ + int idx; + + idx = find_first_zero_bit(dev->event_map.lpi_map, dev->event_map.nr_lpis); + if ( idx == dev->event_map.nr_lpis ) + return -ENOSPC; + + *hwirq = its_get_plpi(dev, idx); + set_bit(idx, dev->event_map.lpi_map); + + return 0; +} + +static void its_free_device(struct its_device *dev) +{ + xfree(dev->itt); + its_lpi_free(dev); + xfree(dev); +} + +static struct its_device *its_alloc_device(u32 devid, u32 nr_ites, + struct dt_device_node *dt_its) +{ + struct its_device *dev; + unsigned long *lpi_map; + int lpi_base, sz; + u16 *col_map = NULL; + + dev = xzalloc(struct its_device); + if ( dev == NULL ) + return NULL; + + dev->its = its_get_phys_node(dt_its); + if ( dev->its == NULL ) + { + printk(XENLOG_ERR + "ITS: Failed to find ITS node for devid 0x%"PRIx32"\n", devid); + goto err; + } + + /* + * At least one bit of EventID is being used, hence a minimum + * of two entries. No, the architecture doesn't let you + * express an ITT with a single entry. + */ + nr_ites = max(2UL, roundup_pow_of_two(nr_ites)); + sz = nr_ites * dev->its->ite_size; + sz = max(sz, ITS_ITT_ALIGN) + ITS_ITT_ALIGN - 1; + + dev->itt = xzalloc_bytes(sz); + if ( !dev->itt ) + goto err; + + lpi_map = its_lpi_alloc_chunks(nr_ites, &lpi_base); + if ( !lpi_map ) + goto lpi_err; + + col_map = xzalloc_bytes(sizeof(*col_map) * nr_ites); + if ( !col_map ) + goto col_err; + + dev->event_map.lpi_map = lpi_map; + dev->event_map.lpi_base = lpi_base; + dev->event_map.col_map = col_map; + dev->event_map.nr_lpis = nr_ites; + dev->device_id = devid; + + return dev; + +col_err: + its_free_device(dev); + return NULL; +lpi_err: + xfree(dev->itt); +err: + xfree(dev); + + return NULL; +} + +/* Device assignment */ +int its_add_device(u32 devid, u32 nr_ites, struct dt_device_node *dt_its) +{ + struct its_device *dev; + u32 i, plpi = 0; + struct its_collection *col; + struct irq_desc *desc; + struct msi_desc *msi = NULL; + int res = 0; + + spin_lock(&rb_its_dev_lock); + dev = its_find_device(devid); + if ( dev ) + { + printk(XENLOG_ERR "ITS: Device already exists 0x%"PRIx32"\n", + dev->device_id); + res = -EEXIST; + goto err_unlock; + } + + dev = its_alloc_device(devid, nr_ites, dt_its); + if ( !dev ) + { + res = -ENOMEM; + goto err_unlock; + } + + BUG_ON(its_insert_device(dev)); + spin_unlock(&rb_its_dev_lock); + + DPRINTK("ITS:Add Device 0x%"PRIx32" lpis %"PRIu32" base 0x%"PRIx32"\n", + dev->device_id, dev->event_map.nr_lpis, dev->event_map.lpi_base); + + /* Map device to ITS ITT */ + its_send_mapd(dev, 1); + + for ( i = 0; i < dev->event_map.nr_lpis; i++ ) + { + msi = xzalloc(struct msi_desc); + if ( its_alloc_device_irq(dev, &plpi) || !msi ) + { + /* Discard LPIs and free device on failure to allocate pLPI */ + its_discard_lpis(dev, i); + its_send_mapd(dev, 0); + + spin_lock(&rb_its_dev_lock); + its_remove_device(dev); + spin_unlock(&rb_its_dev_lock); + + its_free_device(dev); + + printk(XENLOG_ERR "ITS: Cannot add device 0x%"PRIx32"\n", devid); + res = -ENOSPC; + goto err; + } + + /* + * Each Collection is mapped to one physical CPU and + * each pLPI allocated to this device is mapped one collection + * in a round robin fashion. Hence all pLPIs are distributed + * across all processors in the system. + * With this approach, multiple devices having same eventID + * will be mapped to same cpu. + */ + col = &dev->its->collections[(i % nr_cpu_ids)]; + desc = irq_to_desc(plpi); + + spin_lock(&desc->lock); + dev->event_map.col_map[i] = col->col_id; + irq_set_msi_desc(desc, msi); + irqdesc_set_lpi_event(desc, i); + irqdesc_set_its_device(desc, dev); + spin_unlock(&desc->lock); + + /* For each pLPI send MAPVI command */ + its_send_mapvi(dev, plpi, i); + } + + return 0; + +err_unlock: + spin_unlock(&rb_its_dev_lock); +err: + return res; +} + +int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid) +{ + struct its_device *pdev; + u32 plpi, i; + + DPRINTK("ITS: Assign request for dev 0x%"PRIx32" to domain %"PRIu16"\n", + vdevid, d->domain_id); + + spin_lock(&rb_its_dev_lock); + pdev = its_find_device(pdevid); + spin_unlock(&rb_its_dev_lock); + if ( !pdev ) + return -ENODEV; + + /* + * TODO: For pass-through following has to be implemented + * 1) Allow device to be assigned to other domains (Dom0 -> DomU). + * 2) Allow device to be re-assigned to Dom0 (DomU -> Dom0). + * Implement separate function to handle this or rework this function. + * For now do not allow assigning devices other than Dom0. + */ + if ( !is_hardware_domain(d) ) + { + printk(XENLOG_ERR + "ITS: PCI-Passthrough not supported!! to assign from d%d to d%d", + pdev->domain->domain_id, d->domain_id); + return -ENXIO; + } + + pdev->domain = d; + pdev->virt_device_id = vdevid; + + DPRINTK("ITS: Assign pdevid 0x%"PRIx32" lpis %"PRIu32" for dom %"PRIu16"\n", + pdevid, pdev->event_map.nr_lpis, d->domain_id); + + for ( i = 0; i < pdev->event_map.nr_lpis; i++ ) + { + plpi = its_get_plpi(pdev, i); + /* TODO: Route lpi */ + } + + return 0; +} + /* * We allocate 64kB for PROPBASE. That gives us at most 64K LPIs to * deal with (one configuration byte per interrupt). PENDBASE has to diff --git a/xen/include/asm-arm/gic-its.h b/xen/include/asm-arm/gic-its.h index 70065be..7fd7140 100644 --- a/xen/include/asm-arm/gic-its.h +++ b/xen/include/asm-arm/gic-its.h @@ -273,6 +273,10 @@ struct its_device { struct event_lpi_map event_map; /* Physical Device id */ u32 device_id; + /* Virtual Device id */ + u32 virt_device_id; + /* Domain assigned */ + struct domain *domain; /* RB-tree entry */ struct rb_node node; }; @@ -283,6 +287,8 @@ struct its_device *irqdesc_get_its_device(struct irq_desc *desc); void irqdesc_set_its_device(struct irq_desc *desc, struct its_device *dev); int its_init(struct rdist_prop *rdists); int its_cpu_init(void); +int its_add_device(u32 devid, u32 nr_ites, struct dt_device_node *dt_its); +int its_assign_device(struct domain *d, u32 vdevid, u32 pdevid); #endif /* __ASM_ARM_GIC_ITS_H__ */ /*