From patchwork Mon Sep 11 04:38:19 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haozhong Zhang X-Patchwork-Id: 9946621 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7005B6035D for ; Mon, 11 Sep 2017 04:42:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6526E27F2B for ; Mon, 11 Sep 2017 04:42:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5A13128429; Mon, 11 Sep 2017 04:42:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B106D2837E for ; Mon, 11 Sep 2017 04:42:13 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1drGW4-0003cB-5I; Mon, 11 Sep 2017 04:40:12 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1drGW2-0003YR-Nt for xen-devel@lists.xen.org; Mon, 11 Sep 2017 04:40:10 +0000 Received: from [193.109.254.147] by server-9.bemta-6.messagelabs.com id F1/0D-03422-AA316B95; Mon, 11 Sep 2017 04:40:10 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmpgkeJIrShJLcpLzFFi42Jpa+sQ0V0pvC3 S4Mo9KYslHxezODB6HN39mymAMYo1My8pvyKBNWPGmdeMBa32FWeuPWZvYNyl28XIxSEkMJ1R 4tP+RpYuRk4OCQFeiSPLZrB2MXIA2QESe39FQtT0Mkr82HGUEaSGTUBfYsXjg6wgtoiAtMS1z 5fB4swCfxgl7n2SBbGFBUIlFj5ZB1bDIqAqceX0ZTCbV8BO4uOfuYwQu+QldrVdBItzAsUPvn zHDGILCdhKLDi9gHUCI+8CRoZVjBrFqUVlqUW6RuZ6SUWZ6RkluYmZObqGBmZ6uanFxYnpqTm JScV6yfm5mxiB4cAABDsYF68NPMQoycGkJMr77viWSCG+pPyUyozE4oz4otKc1OJDjDIcHEoS vCpC2yKFBItS01Mr0jJzgIEJk5bg4FES4Y0CSfMWFyTmFmemQ6ROMRpzHJhw5Q8TR8fNu3+Yh Fjy8vNSpcR55UBKBUBKM0rz4AbBIuYSo6yUMC8j0GlCPAWpRbmZJajyrxjFORiVhHkjQKbwZO aVwO17BXQKE9ApPJe2gJxSkoiQkmpgnFzy/825yUp737ez/f1yZe3CK9qr1YxFJj3S+7R1cdZ 6X5YlCxT1fq+4/PC89pF1Cg+5JHV2eYqf9vRkEPvXmxBkGHOH2ZT9ikPnTM57lyVt5i7ZyCfe G3TjfOYko9w9kX1nJ02pL86L98j5y5M2/V1pdaftJfvV0Y/frj+75otJ/acPKXf9LJRYijMSD bWYi4oTAWaV18WTAgAA X-Env-Sender: haozhong.zhang@intel.com X-Msg-Ref: server-2.tower-27.messagelabs.com!1505104735!56506342!33 X-Originating-IP: [134.134.136.20] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTM0LjEzNC4xMzYuMjAgPT4gMzU1MzU4\n X-StarScan-Received: X-StarScan-Version: 9.4.45; banners=-,-,- X-VirusChecked: Checked Received: (qmail 52487 invoked from network); 11 Sep 2017 04:40:08 -0000 Received: from mga02.intel.com (HELO mga02.intel.com) (134.134.136.20) by server-2.tower-27.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 11 Sep 2017 04:40:08 -0000 Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Sep 2017 21:40:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.42,376,1500966000"; d="scan'208"; a="1217078660" Received: from hz-desktop.sh.intel.com (HELO localhost) ([10.239.159.142]) by fmsmga002.fm.intel.com with ESMTP; 10 Sep 2017 21:40:06 -0700 From: Haozhong Zhang To: xen-devel@lists.xen.org Date: Mon, 11 Sep 2017 12:38:19 +0800 Message-Id: <20170911043820.14617-39-haozhong.zhang@intel.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170911043820.14617-1-haozhong.zhang@intel.com> References: <20170911043820.14617-1-haozhong.zhang@intel.com> Cc: Haozhong Zhang , Wei Liu , Ian Jackson , Chao Peng , Dan Williams Subject: [Xen-devel] [RFC XEN PATCH v3 38/39] tools/libxl: initiate PMEM mapping via QMP callback X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP The base guest physical address of each vNVDIMM device is decided by QEMU. Add a QMP callback to get the base address from QEMU and query Xen hypervisor to map host PMEM pages to that address. Signed-off-by: Haozhong Zhang --- Cc: Ian Jackson Cc: Wei Liu --- tools/libxl/libxl_qmp.c | 130 ++++++++++++++++++++++++++++++++++++++++++++ tools/libxl/libxl_vnvdimm.c | 30 ++++++++++ tools/libxl/libxl_vnvdimm.h | 30 ++++++++++ 3 files changed, 190 insertions(+) create mode 100644 tools/libxl/libxl_vnvdimm.h diff --git a/tools/libxl/libxl_qmp.c b/tools/libxl/libxl_qmp.c index e1eb47c1d2..299f9c8260 100644 --- a/tools/libxl/libxl_qmp.c +++ b/tools/libxl/libxl_qmp.c @@ -26,6 +26,7 @@ #include "_libxl_list.h" #include "libxl_internal.h" +#include "libxl_vnvdimm.h" /* #define DEBUG_RECEIVED */ @@ -1170,6 +1171,127 @@ int libxl_qemu_monitor_command(libxl_ctx *ctx, uint32_t domid, return rc; } +#if defined(__linux__) + +static int qmp_register_vnvdimm_callback(libxl__qmp_handler *qmp, + const libxl__json_object *o, + void *arg) +{ + GC_INIT(qmp->ctx); + const libxl_domain_config *guest_config = arg; + const libxl_device_vnvdimm *vnvdimm; + const libxl__json_object *obj, *sub_map, *sub_obj; + const char *id, *expected_id; + unsigned int i, slot; + unsigned long gpa, size, mfn, gpfn, nr_pages; + int rc = 0; + + for (i = 0; (obj = libxl__json_array_get(o, i)); i++) { + if (!libxl__json_object_is_map(obj)) + continue; + + sub_map = libxl__json_map_get("data", obj, JSON_MAP); + if (!sub_map) + continue; + + sub_obj = libxl__json_map_get("slot", sub_map, JSON_INTEGER); + slot = libxl__json_object_get_integer(sub_obj); + if (slot > guest_config->num_vnvdimms) { + LOG(ERROR, + "Invalid QEMU memory device slot %u, expecting less than %u", + slot, guest_config->num_vnvdimms); + rc = -ERROR_INVAL; + goto out; + } + vnvdimm = &guest_config->vnvdimms[slot]; + + /* + * Double check whether it's a NVDIMM memory device, through + * all memory devices in QEMU on Xen are for vNVDIMM. + */ + expected_id = libxl__sprintf(gc, "xen_nvdimm%u", slot + 1); + if (!expected_id) { + LOG(ERROR, "Cannot build device id"); + rc = -ERROR_FAIL; + goto out; + } + sub_obj = libxl__json_map_get("id", sub_map, JSON_STRING); + id = libxl__json_object_get_string(sub_obj); + if (!id || strncmp(id, expected_id, strlen(expected_id))) { + LOG(ERROR, + "Invalid QEMU memory device id %s, expecting %s", + id, expected_id); + rc = -ERROR_FAIL; + goto out; + } + + sub_obj = libxl__json_map_get("addr", sub_map, JSON_INTEGER); + gpa = libxl__json_object_get_integer(sub_obj); + sub_obj = libxl__json_map_get("size", sub_map, JSON_INTEGER); + size = libxl__json_object_get_integer(sub_obj); + if ((gpa | size) & ~XC_PAGE_MASK) { + LOG(ERROR, + "Invalid address 0x%lx or size 0x%lx of QEMU memory device %s, " + "not aligned to 0x%lx", + gpa, size, id, XC_PAGE_SIZE); + rc = -ERROR_INVAL; + goto out; + } + gpfn = gpa >> XC_PAGE_SHIFT; + + nr_pages = size >> XC_PAGE_SHIFT; + if (nr_pages > vnvdimm->nr_pages) { + LOG(ERROR, + "Invalid size 0x%lx of QEMU memory device %s, " + "expecting no larger than 0x%lx", + size, id, vnvdimm->nr_pages << XC_PAGE_SHIFT); + rc = -ERROR_INVAL; + goto out; + } + + switch (vnvdimm->backend_type) { + case LIBXL_VNVDIMM_BACKEND_TYPE_MFN: + mfn = vnvdimm->u.mfn; + break; + + default: + LOG(ERROR, "Invalid NVDIMM backend type %u", vnvdimm->backend_type); + rc = -ERROR_INVAL; + goto out; + } + + rc = libxl_vnvdimm_add_pages(gc, qmp->domid, mfn, gpfn, nr_pages); + if (rc) { + LOG(ERROR, + "Cannot map PMEM pages for QEMU memory device %s, " + "mfn 0x%lx, gpfn 0x%lx, nr 0x%lx, rc %d", + id, mfn, gpfn, nr_pages, rc); + rc = -ERROR_FAIL; + goto out; + } + } + + out: + GC_FREE; + return rc; +} + +static int libxl__qmp_query_vnvdimms(libxl__qmp_handler *qmp, + const libxl_domain_config *guest_config) +{ + int rc; + GC_INIT(qmp->ctx); + + rc = qmp_synchronous_send(qmp, "query-memory-devices", NULL, + qmp_register_vnvdimm_callback, + (void *)guest_config, qmp->timeout); + + GC_FREE; + return rc; +} + +#endif /* __linux__ */ + int libxl__qmp_initializations(libxl__gc *gc, uint32_t domid, const libxl_domain_config *guest_config) { @@ -1189,6 +1311,14 @@ int libxl__qmp_initializations(libxl__gc *gc, uint32_t domid, if (!ret) { ret = qmp_query_vnc(qmp); } + +#if defined(__linux__) + if (!ret && guest_config->num_vnvdimms) { + ignore_error = false; + ret = libxl__qmp_query_vnvdimms(qmp, guest_config); + } +#endif /* __linux__ */ + libxl__qmp_close(qmp); out: diff --git a/tools/libxl/libxl_vnvdimm.c b/tools/libxl/libxl_vnvdimm.c index 4de8f04303..ff786d4177 100644 --- a/tools/libxl/libxl_vnvdimm.c +++ b/tools/libxl/libxl_vnvdimm.c @@ -19,6 +19,7 @@ #include #include "libxl_internal.h" +#include "libxl_vnvdimm.h" int libxl_vnvdimm_copy_config(libxl_ctx *ctx, libxl_domain_config *dst, @@ -47,3 +48,32 @@ int libxl_vnvdimm_copy_config(libxl_ctx *ctx, GC_FREE; return rc; } + +#if defined(__linux__) + +int libxl_vnvdimm_add_pages(libxl__gc *gc, uint32_t domid, + xen_pfn_t mfn, xen_pfn_t gpfn, xen_pfn_t nr_pages) +{ + unsigned int nr; + int ret; + + while (nr_pages) { + nr = min(nr_pages, (unsigned long)UINT_MAX); + + ret = xc_domain_populate_pmem_map(CTX->xch, domid, mfn, gpfn, nr); + if (ret && ret != -ERESTART) { + LOG(ERROR, "failed to map PMEM pages, mfn 0x%" PRI_xen_pfn ", " + "gpfn 0x%" PRI_xen_pfn ", nr_pages %u, err %d", + mfn, gpfn, nr, ret); + break; + } + + nr_pages -= nr; + mfn += nr; + gpfn += nr; + } + + return ret; +} + +#endif /* __linux__ */ diff --git a/tools/libxl/libxl_vnvdimm.h b/tools/libxl/libxl_vnvdimm.h new file mode 100644 index 0000000000..ec63c95088 --- /dev/null +++ b/tools/libxl/libxl_vnvdimm.h @@ -0,0 +1,30 @@ +/* + * tools/libxl/libxl_vnvdimm.h + * + * Copyright (C) 2017, Intel Corporation + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License, version 2.1, as published by the Free Software Foundation. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; If not, see . + */ + +#ifndef LIBXL_VNVDIMM_H +#define LIBXL_VNVDIMM_H + +#include +#include "libxl_internal.h" + +#if defined(__linux__) +int libxl_vnvdimm_add_pages(libxl__gc *gc, uint32_t domid, + xen_pfn_t mfn, xen_pfn_t gpfn, xen_pfn_t nr_pages); +#endif /* __linux__ */ + +#endif /* !LIBXL_VNVDIMM_H */