From patchwork Mon Mar 20 00:09:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haozhong Zhang X-Patchwork-Id: 9632895 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 9573A6020B for ; Mon, 20 Mar 2017 00:16:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 88ED327F60 for ; Mon, 20 Mar 2017 00:16:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7DD8627F9F; Mon, 20 Mar 2017 00:16:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_MED,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E28A62808C for ; Mon, 20 Mar 2017 00:15:55 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cpkx1-0006zM-F2; Mon, 20 Mar 2017 00:13:31 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cpkx0-0006wH-L0 for xen-devel@lists.xen.org; Mon, 20 Mar 2017 00:13:30 +0000 Received: from [85.158.137.68] by server-12.bemta-3.messagelabs.com id 82/8F-12861-AAE1FC85; Mon, 20 Mar 2017 00:13:30 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupgkeJIrShJLcpLzFFi42I5YG5SrLtS7ny EQdN6DoslHxezODB6HN39mymAMYo1My8pvyKBNaN1Vx97wRvPiufbvjE1MDZadDFycbAI3GKS 6Jncwt7FyMkhJDCdUaLpuiGILSHAK3Fk2QxWCNtPYsKpzewgDUICvYwSb2/dAGtgE9CXWPH4I FiRiIC0xLXPlxlBipgFzjJKTF/TClYkLBApMb9vBVgRi4CqxJGDJ9hAbF4BO4lpu14xQmyQl7 hw9RQLiM0JFF9z6TbURbYS+29NY5vAyLeAkWEVo0ZxalFZapGuoYleUlFmekZJbmJmjq6hgbF ebmpxcWJ6ak5iUrFecn7uJkZgqDAAwQ7GFds9DzFKcjApifKW/zgRIcSXlJ9SmZFYnBFfVJqT WnyIUYaDQ0mC94jSyQghwaLU9NSKtMwcYNDCpCU4eJREeHNB0rzFBYm5xZnpEKlTjLocc2bvf sMkxJKXn5cqJc57AaRIAKQoozQPbgQsgi4xykoJ8zICHSXEU5BalJtZgir/ilGcg1FJmHczyB SezLwSuE2vgI5gAjri7YcTIEeUJCKkpBoYNdtYnOXv9uaGv3UtF18dIpixfV2+zJHyB3/4lI2 qw+c6MUcV7592t6Ipvtw4q+PV6+7pfzRlBZRXiWgJtMrJxU2oniXIKN3z+Oq97vV/jDirNZZc Ew4Jynr1+b3k8kvqNf/nl17av7k840b4K5OOyxOsf60yy1JRWLc9+MzeX3F/zvwyDtVXYinOS DTUYi4qTgQA9qLYE5sCAAA= X-Env-Sender: haozhong.zhang@intel.com X-Msg-Ref: server-8.tower-31.messagelabs.com!1489968802!91050058!3 X-Originating-IP: [192.55.52.115] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 9.2.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 57181 invoked from network); 20 Mar 2017 00:13:28 -0000 Received: from mga14.intel.com (HELO mga14.intel.com) (192.55.52.115) by server-8.tower-31.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 20 Mar 2017 00:13:28 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=intel.com; i=@intel.com; q=dns/txt; s=intel; t=1489968808; x=1521504808; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=6tBG1ADU/o0v2Hd/0U7KeVu1L9NDzSJcx2dd3hEjVuc=; b=ZPyerMcoQksWkzswkMiWivJMSTXnXiDcH2DCNRRFGBHKF2baFXlqpP7P Noh3Ss3NXWX1Y97oE2X6aAyukFpZKg==; Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga103.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 19 Mar 2017 17:13:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.36,191,1486454400"; d="scan'208";a="78799229" Received: from hz-desktop.sh.intel.com (HELO localhost) ([10.239.159.153]) by fmsmga006.fm.intel.com with ESMTP; 19 Mar 2017 17:13:26 -0700 From: Haozhong Zhang To: xen-devel@lists.xen.org Date: Mon, 20 Mar 2017 08:09:46 +0800 Message-Id: <20170320000949.24675-13-haozhong.zhang@intel.com> X-Mailer: git-send-email 2.10.1 In-Reply-To: <20170320000949.24675-1-haozhong.zhang@intel.com> References: <20170320000949.24675-1-haozhong.zhang@intel.com> Cc: Konrad Rzeszutek Wilk , Dan Williams , Ian Jackson , Wei Liu , Haozhong Zhang Subject: [Xen-devel] [RFC XEN PATCH v2 12/15] tools/libxl: build qemu options from xl vNVDIMM configs X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP For xl configs vnvdimms = [ '/path/to/pmem0', '/path/to/pmem1', ... ] the following qemu options are built -machine ,nvdimm -m ,slots=$NR_SLOTS,maxmem=$MEM_SIZE -object memory-backend-xen,id=mem1,size=$PMEM0_SIZE,mem-path=/path/to/pmem0 -device nvdimm,id=nvdimm1,memdev=mem1 -object memory-backend-xen,id=mem2,size=$PMEM1_SIZE,mem-path=/path/to/pmem1 -device nvdimm,id=nvdimm2,memdev=mem2 ... where - NR_SLOTS is the number of entries in vnvdimms + 1, - MEM_SIZE is the total size of all RAM and NVDIMM devices, - PMEM#_SIZE is the size of the host pmem device/file '/path/to/pmem#'. Signed-off-by: Haozhong Zhang --- Cc: Ian Jackson Cc: Wei Liu The qemu option "-object memory-backend-xen" is added by the QEMU patch "hostmem: add a host memory backend for Xen". Other qemu options have been implemented since QEMU 2.6.0. Changes in v2: * Update the manpage of xl.cfg for the new option "vnvdimms". --- docs/man/xl.cfg.pod.5.in | 6 +++ tools/libxl/libxl_dm.c | 109 +++++++++++++++++++++++++++++++++++++++++++- tools/libxl/libxl_types.idl | 8 ++++ tools/xl/xl_parse.c | 16 +++++++ 4 files changed, 137 insertions(+), 2 deletions(-) diff --git a/docs/man/xl.cfg.pod.5.in b/docs/man/xl.cfg.pod.5.in index 505c11137f..53296595ff 100644 --- a/docs/man/xl.cfg.pod.5.in +++ b/docs/man/xl.cfg.pod.5.in @@ -1064,6 +1064,12 @@ FIFO-based event channel ABI support up to 131,071 event channels. Other guests are limited to 4095 (64-bit x86 and ARM) or 1023 (32-bit x86). +=item B + +Specify the virtual NVDIMM devices which are provided to the guest. +B, B, ... specify the host NVDIMM pmem devices which are used +as the backend storage of each virtual NVDIMM device. + =back =head2 Paravirtualised (PV) Guest Specific Options diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c index 281058de45..695396da2d 100644 --- a/tools/libxl/libxl_dm.c +++ b/tools/libxl/libxl_dm.c @@ -24,6 +24,10 @@ #include #include +#if defined(__linux__) +#include /* for ioctl(BLKGETSIZE64) */ +#endif + static const char *libxl_tapif_script(libxl__gc *gc) { #if defined(__linux__) || defined(__FreeBSD__) @@ -910,6 +914,82 @@ static char *qemu_disk_ide_drive_string(libxl__gc *gc, const char *target_path, return drive; } +#if defined(__linux__) + +static uint64_t libxl__build_dm_vnvdimm_args(libxl__gc *gc, flexarray_t *dm_args, + struct libxl_device_vnvdimm *dev, + int dev_no) +{ + int fd, rc; + struct stat st; + uint64_t size = 0; + char *arg; + + fd = open(dev->file, O_RDONLY); + if (fd < 0) { + LOG(ERROR, "failed to open file %s: %s", + dev->file, strerror(errno)); + goto out; + } + + if (stat(dev->file, &st)) { + LOG(ERROR, "failed to get status of file %s: %s", + dev->file, strerror(errno)); + goto out_fclose; + } + + switch (st.st_mode & S_IFMT) { + case S_IFBLK: + rc = ioctl(fd, BLKGETSIZE64, &size); + if (rc == -1) { + LOG(ERROR, "failed to get size of block device %s: %s", + dev->file, strerror(errno)); + size = 0; + } + break; + + default: + LOG(ERROR, "%s not block device", dev->file); + break; + } + + if (!size) + goto out_fclose; + + flexarray_append(dm_args, "-object"); + arg = GCSPRINTF("memory-backend-xen,id=mem%d,size=%"PRIu64",mem-path=%s", + dev_no + 1, size, dev->file); + flexarray_append(dm_args, arg); + + flexarray_append(dm_args, "-device"); + arg = GCSPRINTF("nvdimm,id=nvdimm%d,memdev=mem%d", dev_no + 1, dev_no + 1); + flexarray_append(dm_args, arg); + + out_fclose: + close(fd); + out: + return size; +} + +static uint64_t libxl__build_dm_vnvdimms_args( + libxl__gc *gc, flexarray_t *dm_args, + struct libxl_device_vnvdimm *vnvdimms, int num_vnvdimms) +{ + uint64_t total_size = 0, size; + unsigned int i; + + for (i = 0; i < num_vnvdimms; i++) { + size = libxl__build_dm_vnvdimm_args(gc, dm_args, &vnvdimms[i], i); + if (!size) + break; + total_size += size; + } + + return total_size; +} + +#endif /* __linux__ */ + static int libxl__build_device_model_args_new(libxl__gc *gc, const char *dm, int guest_domid, const libxl_domain_config *guest_config, @@ -923,13 +1003,18 @@ static int libxl__build_device_model_args_new(libxl__gc *gc, const libxl_device_nic *nics = guest_config->nics; const int num_disks = guest_config->num_disks; const int num_nics = guest_config->num_nics; +#if defined(__linux__) + const int num_vnvdimms = guest_config->num_vnvdimms; +#else + const int num_vnvdimms = 0; +#endif const libxl_vnc_info *vnc = libxl__dm_vnc(guest_config); const libxl_sdl_info *sdl = dm_sdl(guest_config); const char *keymap = dm_keymap(guest_config); char *machinearg; flexarray_t *dm_args, *dm_envs; int i, connection, devid, ret; - uint64_t ram_size; + uint64_t ram_size, ram_size_in_byte, vnvdimms_size = 0; const char *path, *chardev; char *user = NULL; @@ -1313,6 +1398,9 @@ static int libxl__build_device_model_args_new(libxl__gc *gc, } } + if (num_vnvdimms) + machinearg = libxl__sprintf(gc, "%s,nvdimm", machinearg); + flexarray_append(dm_args, machinearg); for (i = 0; b_info->extra_hvm && b_info->extra_hvm[i] != NULL; i++) flexarray_append(dm_args, b_info->extra_hvm[i]); @@ -1322,8 +1410,25 @@ static int libxl__build_device_model_args_new(libxl__gc *gc, } ram_size = libxl__sizekb_to_mb(b_info->max_memkb - b_info->video_memkb); + ram_size_in_byte = ram_size * 1024 * 1024; + if (num_vnvdimms) { + vnvdimms_size = libxl__build_dm_vnvdimms_args(gc, dm_args, + guest_config->vnvdimms, + num_vnvdimms); + if (ram_size_in_byte + vnvdimms_size < ram_size_in_byte) { + LOG(ERROR, + "total size of RAM (%"PRIu64") and NVDIMM (%"PRIu64") overflow", + ram_size_in_byte, vnvdimms_size); + return ERROR_INVAL; + } + } flexarray_append(dm_args, "-m"); - flexarray_append(dm_args, GCSPRINTF("%"PRId64, ram_size)); + flexarray_append(dm_args, + vnvdimms_size ? + GCSPRINTF("%"PRId64",slots=%d,maxmem=%"PRId64, + ram_size, num_vnvdimms + 1, + ROUNDUP(ram_size_in_byte + vnvdimms_size, 12)) : + GCSPRINTF("%"PRId64, ram_size)); if (b_info->type == LIBXL_DOMAIN_TYPE_HVM) { if (b_info->u.hvm.hdtype == LIBXL_HDTYPE_AHCI) diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl index a612d1f4ff..e1a3fd9279 100644 --- a/tools/libxl/libxl_types.idl +++ b/tools/libxl/libxl_types.idl @@ -704,6 +704,13 @@ libxl_device_channel = Struct("device_channel", [ ])), ]) +libxl_device_vnvdimm = Struct("device_vnvdimm", [ + ("backend_domid", libxl_domid), + ("backend_domname", string), + ("devid", libxl_devid), + ("file", string), +]) + libxl_domain_config = Struct("domain_config", [ ("c_info", libxl_domain_create_info), ("b_info", libxl_domain_build_info), @@ -721,6 +728,7 @@ libxl_domain_config = Struct("domain_config", [ ("channels", Array(libxl_device_channel, "num_channels")), ("usbctrls", Array(libxl_device_usbctrl, "num_usbctrls")), ("usbdevs", Array(libxl_device_usbdev, "num_usbdevs")), + ("vnvdimms", Array(libxl_device_vnvdimm, "num_vnvdimms")), ("on_poweroff", libxl_action_on_shutdown), ("on_reboot", libxl_action_on_shutdown), diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c index 1ef0c272a8..2cf8c9756b 100644 --- a/tools/xl/xl_parse.c +++ b/tools/xl/xl_parse.c @@ -718,6 +718,7 @@ void parse_config_data(const char *config_source, XLU_ConfigList *cpus, *vbds, *nics, *pcis, *cvfbs, *cpuids, *vtpms, *usbctrls, *usbdevs; XLU_ConfigList *channels, *ioports, *irqs, *iomem, *viridian, *dtdevs; + XLU_ConfigList *vnvdimms; int num_ioports, num_irqs, num_iomem, num_cpus, num_viridian; int pci_power_mgmt = 0; int pci_msitranslate = 0; @@ -1902,6 +1903,21 @@ skip_usbdev: } } + if (!xlu_cfg_get_list (config, "vnvdimms", &vnvdimms, 0, 0)) { +#if defined(__linux__) + while ((buf = xlu_cfg_get_listitem(vnvdimms, + d_config->num_vnvdimms)) != NULL) { + libxl_device_vnvdimm *vnvdimm = + ARRAY_EXTEND_INIT(d_config->vnvdimms, d_config->num_vnvdimms, + libxl_device_vnvdimm_init); + vnvdimm->file = strdup(buf); + } +#else + fprintf(stderr, "ERROR: vnvdimms is only supported on Linux\n"); + exit(-ERROR_FAIL); +#endif /* __linux__ */ + } + xlu_cfg_destroy(config); }