From patchwork Thu Feb 1 21:40:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Andryuk X-Patchwork-Id: 13541833 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AA975C4828D for ; Thu, 1 Feb 2024 21:40:40 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.674716.1049841 (Exim 4.92) (envelope-from ) id 1rVenP-0005Z8-Nk; Thu, 01 Feb 2024 21:40:31 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 674716.1049841; Thu, 01 Feb 2024 21:40:31 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rVenP-0005Yx-L1; Thu, 01 Feb 2024 21:40:31 +0000 Received: by outflank-mailman (input) for mailman id 674716; Thu, 01 Feb 2024 21:40:29 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rVenN-0005YC-PJ for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 21:40:29 +0000 Received: from mail-qk1-x731.google.com (mail-qk1-x731.google.com [2607:f8b0:4864:20::731]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 83c67fe8-c14a-11ee-8a43-1f161083a0e0; Thu, 01 Feb 2024 22:40:28 +0100 (CET) Received: by mail-qk1-x731.google.com with SMTP id af79cd13be357-783d4b3a00bso100070885a.3 for ; Thu, 01 Feb 2024 13:40:28 -0800 (PST) Received: from pm2-ws13.praxislan02.com (207-172-141-204.s8906.c3-0.slvr-cbr1.lnh-slvr.md.cable.rcncustomer.com. [207.172.141.204]) by smtp.gmail.com with ESMTPSA id h7-20020a05620a21c700b007853eb72c49sm154978qka.46.2024.02.01.13.40.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Feb 2024 13:40:26 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 83c67fe8-c14a-11ee-8a43-1f161083a0e0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706823627; x=1707428427; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vkexTyneUo8yDBZMkq46yZq63xLJ/JfR3uJQ589GQF4=; b=aMbHaPRdMZxgH0MmI0cZagUUrvBf+uCDs2Li3M/7v/JjuaHESdDmjwNSkHTCyS0vu9 U46JZzZTN8Ka0BfP0AcGJata3CNGz/UXm9OBTtNKOXj3VuRh22DYCAeA/Sch57mV2lyZ pvlz6vjBn/2UlmP32a8QedqyP8+i+9upuz1g6IXfTjstnmtnm9CE8ki7tm6KbNvmEw6a 6R5oEgwnOeJlPQvZA0HFJPOsEIAnVPc0fkyZaApjR1cu2i9FiGUBWKO/IhZUgPDJw8Br /esNCGEtVJT+IFQHiMuAdl7WQE0gmc3H5D+m5k5LmsBNZXi6JnvlrJXHNOVntrh8qSV1 pR3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706823627; x=1707428427; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vkexTyneUo8yDBZMkq46yZq63xLJ/JfR3uJQ589GQF4=; b=qfM7BGu1yZWOloU9QFdHtCU3pktqt/la2gJPztA1AKCZGJHgQq0ez4yxYiQrIKQSZW H0Eku/Nd8jUackv8sNvRafuAxq2jpIslv4p61o290nQfwGvACe/bIisjXzsuU6UFfuoi aHzx/f6ukh8v+9Z5EzLADOnLJN8DlWG06bxDIn5gZsreup6K5ln2vKIoPViDK8XT09N3 6kNg6mzU9wF1Fg5uLiGTtoybVhe9kK2F8KJj4GEq8+GyIbMz2vN+TFxWKid3mnc0KNR4 1f3i+l9+97oYVYyCbsCIOVKddZ6gHuIA3t2c75nIaIqC6Yw1LcUt+GIspQsyg5qXqC/2 LJnw== X-Gm-Message-State: AOJu0YzueLxubb2/RA9J1UQ+6MczHB+NMSnAMa+CNPNRjINqmhylaf9g ho4lX/r7vrqdSjmvdRqloLw4lhMG9mXzuNbqqAi3LJ5wKgar/vLVSOVXKkBT X-Google-Smtp-Source: AGHT+IGLBrIpsduzyOBzC7LSkPVe7PuJfrOhZUbgPczkeBMlBbhnOdd0NAToewP/ECbnKyOeZmEAYw== X-Received: by 2002:a05:6214:dcc:b0:685:c584:6e8b with SMTP id 12-20020a0562140dcc00b00685c5846e8bmr7595618qvt.0.1706823626707; Thu, 01 Feb 2024 13:40:26 -0800 (PST) X-Forwarded-Encrypted: i=0; AJvYcCX34J5GhBe54FX3RBJqs7ZIWuDZVmL3wYyHWNMhm4yVwBjP4NDDQcS4ST2AaUoq/EH7He+c6OGqS3wkKJlg5Rhh3uN4e+jlCVP/jJCzcXOuBXqDeRJMNoCkC4IN3lwXH6Vjdym+ADGppGppFifok3YOG4DAbHttMLfKsw== From: Jason Andryuk To: xen-devel@lists.xenproject.org Cc: marmarek@invisiblethingslab.com, Jason Andryuk , Wei Liu , Anthony PERARD , Juergen Gross Subject: [PATCH v2 1/3] libxl: Create empty file for Phy cdrom Date: Thu, 1 Feb 2024 16:40:02 -0500 Message-ID: <20240201214004.238858-2-jandryuk@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240201214004.238858-1-jandryuk@gmail.com> References: <20240201214004.238858-1-jandryuk@gmail.com> MIME-Version: 1.0 With a device model stubdom, dom0 exports a PV disk to the stubdom. Inside the stubdom, QEMU emulates a cdrom to the guest with a host_device pointing at the PV frontend (/dev/xvdc) An empty cdrom drive causes problems booting the stubdom. The PV disk protocol isn't designed to support no media. That can be partially hacked around, but the stubdom kernel waits for all block devices to transition to Connected. Since the backend never connects empty media, stubdom launch times out and it is destroyed. Empty media and the PV disks not connecting is fine at runtime since the stubdom keeps running irrespective of the disk state. Empty media can be worked around my providing an empty file to the stubdom for the PV disk source. This works as the disk is exposed as a zero-size disk. Dynamically create the empty file as needed and remove in the stubdom cleanup. libxl__device_disk_set_backend() needs to allow through these "empty" disks with a pdev_path. Fixup the params writing since scripts have trouble with an empty params field. This works for non-stubdom HVMs as well. Signed-off-by: Jason Andryuk Reviewed-by: Anthony PERARD --- v2: New to support "empty" cdroms --- tools/libs/light/libxl_device.c | 5 ++++- tools/libs/light/libxl_disk.c | 36 +++++++++++++++++++++++++++---- tools/libs/light/libxl_domain.c | 4 ++++ tools/libs/light/libxl_internal.h | 1 + 4 files changed, 41 insertions(+), 5 deletions(-) diff --git a/tools/libs/light/libxl_device.c b/tools/libs/light/libxl_device.c index 13da6e0573..09d85928d7 100644 --- a/tools/libs/light/libxl_device.c +++ b/tools/libs/light/libxl_device.c @@ -421,7 +421,10 @@ int libxl__device_disk_set_backend(libxl__gc *gc, libxl_device_disk *disk) { LOG(ERROR, "Disk vdev=%s is empty but not cdrom", disk->vdev); return ERROR_INVAL; } - if (disk->pdev_path != NULL && strcmp(disk->pdev_path, "")) { + if (disk->pdev_path != NULL && + (strcmp(disk->pdev_path, "") && + strncmp(disk->pdev_path, LIBXL_STUBDOM_EMPTY_CDROM, + strlen(LIBXL_STUBDOM_EMPTY_CDROM)))) { LOG(ERROR, "Disk vdev=%s is empty but an image has been provided: %s", disk->vdev, disk->pdev_path); diff --git a/tools/libs/light/libxl_disk.c b/tools/libs/light/libxl_disk.c index ea3623dd6f..c48e1de659 100644 --- a/tools/libs/light/libxl_disk.c +++ b/tools/libs/light/libxl_disk.c @@ -199,6 +199,32 @@ static int libxl__device_disk_setdefault(libxl__gc *gc, uint32_t domid, disk->backend = LIBXL_DISK_BACKEND_QDISK; } + if (disk->is_cdrom && + disk->format == LIBXL_DISK_FORMAT_EMPTY && + disk->backend == LIBXL_DISK_BACKEND_PHY && + disk->backend_domid == LIBXL_TOOLSTACK_DOMID) { + uint32_t target_domid; + int fd; + + if (libxl_is_stubdom(CTX, domid, &target_domid)) { + LOGED(DEBUG, domid, "Using target_domid %u", target_domid); + } else { + target_domid = domid; + } + free(disk->pdev_path); + disk->pdev_path = + libxl__sprintf(NOGC, LIBXL_STUBDOM_EMPTY_CDROM ".%u", + target_domid); + fd = creat(disk->pdev_path, 0400); + if (fd < 0) { + LOGED(ERROR, domid, "Failed to create empty cdrom \"%s\"", + disk->pdev_path); + return ERROR_FAIL; + } + + close(fd); + } + rc = libxl__device_disk_set_backend(gc, disk); return rc; } @@ -988,7 +1014,7 @@ static void cdrom_insert_ejected(libxl__egc *egc, empty = flexarray_make(gc, 4, 1); flexarray_append_pair(empty, "type", libxl__device_disk_string_of_backend(disk->backend)); - flexarray_append_pair(empty, "params", ""); + flexarray_append_pair(empty, "params", disk->pdev_path ?: ""); for (;;) { rc = libxl__xs_transaction_start(gc, &t); @@ -1164,13 +1190,15 @@ static void cdrom_insert_inserted(libxl__egc *egc, insert = flexarray_make(gc, 4, 1); flexarray_append_pair(insert, "type", libxl__device_disk_string_of_backend(disk->backend)); - if (disk->format != LIBXL_DISK_FORMAT_EMPTY) + if (disk->backend == LIBXL_DISK_BACKEND_QDISK && + disk->format != LIBXL_DISK_FORMAT_EMPTY) { flexarray_append_pair(insert, "params", GCSPRINTF("%s:%s", libxl__device_disk_string_of_format(disk->format), disk->pdev_path)); - else - flexarray_append_pair(insert, "params", ""); + } else { + flexarray_append_pair(insert, "params", disk->pdev_path ?: ""); + } for (;;) { rc = libxl__xs_transaction_start(gc, &t); diff --git a/tools/libs/light/libxl_domain.c b/tools/libs/light/libxl_domain.c index 5ee1544d9c..6751fc785f 100644 --- a/tools/libs/light/libxl_domain.c +++ b/tools/libs/light/libxl_domain.c @@ -1525,6 +1525,10 @@ static void devices_destroy_cb(libxl__egc *egc, if (rc < 0) LOGD(ERROR, domid, "libxl__devices_destroy failed"); + /* Remove the file after the hotplug scripts have run. The scripts won't + * run if the file doesn't exist when they are run. */ + libxl__remove_file(gc, GCSPRINTF(LIBXL_STUBDOM_EMPTY_CDROM ".%u", domid)); + vm_path = libxl__xs_read(gc, XBT_NULL, GCSPRINTF("%s/vm", dom_path)); if (vm_path) if (!xs_rm(ctx->xsh, XBT_NULL, vm_path)) diff --git a/tools/libs/light/libxl_internal.h b/tools/libs/light/libxl_internal.h index d5732d1c37..094d0df9b1 100644 --- a/tools/libs/light/libxl_internal.h +++ b/tools/libs/light/libxl_internal.h @@ -120,6 +120,7 @@ #define STUBDOM_CONSOLE_SERIAL 3 #define STUBDOM_SPECIAL_CONSOLES 3 #define LIBXL_LINUX_STUBDOM_MEM 128 +#define LIBXL_STUBDOM_EMPTY_CDROM XEN_RUN_DIR "/empty-cdrom" /* .$domid */ #define TAP_DEVICE_SUFFIX "-emu" #define DOMID_XS_PATH "domid" #define PVSHIM_BASENAME "xen-shim" From patchwork Thu Feb 1 21:40:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Andryuk X-Patchwork-Id: 13541835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 69028C48292 for ; Thu, 1 Feb 2024 21:40:43 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.674717.1049851 (Exim 4.92) (envelope-from ) id 1rVenT-0005ps-36; Thu, 01 Feb 2024 21:40:35 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 674717.1049851; Thu, 01 Feb 2024 21:40:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rVenS-0005ph-UT; Thu, 01 Feb 2024 21:40:34 +0000 Received: by outflank-mailman (input) for mailman id 674717; Thu, 01 Feb 2024 21:40:33 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rVenR-0005Jd-Qq for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 21:40:33 +0000 Received: from mail-yw1-x1130.google.com (mail-yw1-x1130.google.com [2607:f8b0:4864:20::1130]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 85fb3eb9-c14a-11ee-98f5-efadbce2ee36; Thu, 01 Feb 2024 22:40:32 +0100 (CET) Received: by mail-yw1-x1130.google.com with SMTP id 00721157ae682-6040d9e52b9so14766197b3.0 for ; Thu, 01 Feb 2024 13:40:32 -0800 (PST) Received: from pm2-ws13.praxislan02.com (207-172-141-204.s8906.c3-0.slvr-cbr1.lnh-slvr.md.cable.rcncustomer.com. [207.172.141.204]) by smtp.gmail.com with ESMTPSA id h7-20020a05620a21c700b007853eb72c49sm154978qka.46.2024.02.01.13.40.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Feb 2024 13:40:29 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 85fb3eb9-c14a-11ee-98f5-efadbce2ee36 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706823630; x=1707428430; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ucbmMMAbbZ+Pz4AdE+09SKz62vvqmDMnyT9AshCA9EQ=; b=Qsibb857iTEPS1EEvQDMv1Wz+TmQkg/EwlPqWKozxR3UH1IYEDkQvSAIaJKCyMrJQc R8ZNpsoTXmjJ/zwkGZJBApYJRhgqAJEiHA6+u2s9EafRGxZQ2BwesEGeTe4FovUnbqkc 2awQVWK59lBmHQbDj7g+TBHgc1yGOW1sbkkltym96DOqMpg25cbjevZjSu9lancgoChk 7y4e6fkwhDt4NSm9uXFLxOMrkXD2l+Z5rVuMc0YqpDLbnKoAmpR4v8yNFhA8W87pyk08 Pxv3kTKcSnAFd61f5C7payadPlUYWeIDrgss3DUhKY0M8CwQCajdd9z+woCsXnrPXqXa d5Qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706823630; x=1707428430; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ucbmMMAbbZ+Pz4AdE+09SKz62vvqmDMnyT9AshCA9EQ=; b=VIpoGrOzpy7g5nNLGCaMOiBrW52SYDp6wKKB2TPHkmtHTMzdiLOeDO5oNbN6NfpJJs aZMDiiTH3ylYltn7WOWka6SHgiRiV45fkPWE1KqNe/doLKRDObSWDlPyITgjeKHSOuBB FCHrvgAG7FT61GNNJi5vMA0ORLU3ET8v2W/M04//DdG1ON6ZOItGiAzuYKEox2N9Evw2 Xb/CVflISCg1inRMGOyLAi9LwAWJbNlxQ0I3yxMCzkr6T7kUsnF850/hsQ88PUivtt2Z 91mX44L3nR9GDuk8z/PqcS3E0RAIDbO+hPhPwfdpNrc2hji4sDFZURWODamSfrqMxCnN I5iw== X-Gm-Message-State: AOJu0YxBNaTiG1IM5z8MAjpiO/9Vr1vcHkLIHFL/s9L0jKFKErgQcJGd Njs9Vbovx1FQGs6Tm5lCHoOc1N0yugTdK4Whm+tSU10FukK8J3AnFhjBrPit X-Google-Smtp-Source: AGHT+IGyZ3ROrgxWgwd9VmfOUiLAllEP63hV3T4NdJde0vqe3kbDuDiHRW5UFWDUI28W5/SabNCBWw== X-Received: by 2002:a0d:cc53:0:b0:5ff:6b9e:cdd9 with SMTP id o80-20020a0dcc53000000b005ff6b9ecdd9mr463708ywd.46.1706823630026; Thu, 01 Feb 2024 13:40:30 -0800 (PST) X-Forwarded-Encrypted: i=0; AJvYcCWt/eemjeAB0YsJRR+bss0JwPO4dcbAfWs7OXwO60kt3BD3y1I2eGHOBZL3aMxPpKLx8YYVZeNUEUmAMMNLGe99eaiJiz5jrUGQwORoA/L3/EG66/nsdnT8Yd4yV+zKju6qm0LYgtGFDem4UWXV9C1+PwFIa2cW4mOc4w== From: Jason Andryuk To: xen-devel@lists.xenproject.org Cc: marmarek@invisiblethingslab.com, Jason Andryuk , Wei Liu , Anthony PERARD , Juergen Gross Subject: [PATCH v2 2/3] libxl: Allow Phy backend for CDROM devices Date: Thu, 1 Feb 2024 16:40:03 -0500 Message-ID: <20240201214004.238858-3-jandryuk@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240201214004.238858-1-jandryuk@gmail.com> References: <20240201214004.238858-1-jandryuk@gmail.com> MIME-Version: 1.0 A Linux HVM domain ignores PV block devices with type cdrom. The Windows PV drivers also ignore device-type != "disk". Therefore QEMU's emulated CD-ROM support is used. This allows ejection and other CD-ROM features to work. With a stubdom, QEMU is running in the stubdom. A PV disk is still connected into the stubdom, and then QEMU can emulate the CD-ROM into the guest. Phy support has been enhanced to provide a placeholder file forempty disks, so it is usable as a CDROM backend as well. Allow Phy to pass the check as well. (Bypassing just for a linux-based stubdom doesn't work because libxl__device_disk_setdefault() gets called early in domain creation before xenstore is populated with relevant information for the stubdom type. The build information isn't readily available and won't exist in some call trees, so it isn't usable either.) Let disk_try_backend() allow format empty for Phy cdrom drives. Signed-off-by: Jason Andryuk Reviewed-by: Anthony PERARD --- v2: Different approach to pass QDISK requirement check. --- tools/libs/light/libxl_device.c | 12 ++++++++---- tools/libs/light/libxl_disk.c | 11 +++++++---- 2 files changed, 15 insertions(+), 8 deletions(-) diff --git a/tools/libs/light/libxl_device.c b/tools/libs/light/libxl_device.c index 09d85928d7..f73c6705d4 100644 --- a/tools/libs/light/libxl_device.c +++ b/tools/libs/light/libxl_device.c @@ -301,13 +301,17 @@ static int disk_try_backend(disk_try_backend_args *a, switch (backend) { case LIBXL_DISK_BACKEND_PHY: - if (a->disk->format != LIBXL_DISK_FORMAT_RAW) { - goto bad_format; - } - if (libxl_defbool_val(a->disk->colo_enable)) goto bad_colo; + if (a->disk->is_cdrom && a->disk->format == LIBXL_DISK_FORMAT_EMPTY) { + LOG(DEBUG, "Disk vdev=%s is an empty cdrom", a->disk->vdev); + return backend; + } + + if (a->disk->format != LIBXL_DISK_FORMAT_RAW) + goto bad_format; + if (a->disk->backend_domid != LIBXL_TOOLSTACK_DOMID) { LOG(DEBUG, "Disk vdev=%s, is using a storage driver domain, " "skipping physical device check", a->disk->vdev); diff --git a/tools/libs/light/libxl_disk.c b/tools/libs/light/libxl_disk.c index c48e1de659..09082ffb58 100644 --- a/tools/libs/light/libxl_disk.c +++ b/tools/libs/light/libxl_disk.c @@ -188,15 +188,18 @@ static int libxl__device_disk_setdefault(libxl__gc *gc, uint32_t domid, return ERROR_FAIL; } - /* Force Qdisk backend for CDROM devices of guests with a device model. */ + /* Only allow Qdisk or Phy for CDROM devices. */ if (disk->is_cdrom != 0 && libxl__domain_type(gc, domid) == LIBXL_DOMAIN_TYPE_HVM) { + if (disk->backend == LIBXL_DISK_BACKEND_UNKNOWN) + disk->backend = LIBXL_DISK_BACKEND_QDISK; + if (!(disk->backend == LIBXL_DISK_BACKEND_QDISK || - disk->backend == LIBXL_DISK_BACKEND_UNKNOWN)) { - LOGD(ERROR, domid, "Backend for CD devices on HVM guests must be Qdisk"); + disk->backend == LIBXL_DISK_BACKEND_PHY)) { + LOGD(ERROR, domid, + "Backend for CD devices on HVM guests must be Qdisk or Phy"); return ERROR_FAIL; } - disk->backend = LIBXL_DISK_BACKEND_QDISK; } if (disk->is_cdrom && From patchwork Thu Feb 1 21:40:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Andryuk X-Patchwork-Id: 13541836 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6525DC4828D for ; Thu, 1 Feb 2024 21:40:47 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.674718.1049861 (Exim 4.92) (envelope-from ) id 1rVenW-00067j-9d; Thu, 01 Feb 2024 21:40:38 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 674718.1049861; Thu, 01 Feb 2024 21:40:38 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rVenW-00067a-64; Thu, 01 Feb 2024 21:40:38 +0000 Received: by outflank-mailman (input) for mailman id 674718; Thu, 01 Feb 2024 21:40:36 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1rVenU-0005YC-HZ for xen-devel@lists.xenproject.org; Thu, 01 Feb 2024 21:40:36 +0000 Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com [2607:f8b0:4864:20::735]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 87c20d68-c14a-11ee-8a43-1f161083a0e0; Thu, 01 Feb 2024 22:40:35 +0100 (CET) Received: by mail-qk1-x735.google.com with SMTP id af79cd13be357-78407987510so70292385a.3 for ; Thu, 01 Feb 2024 13:40:35 -0800 (PST) Received: from pm2-ws13.praxislan02.com (207-172-141-204.s8906.c3-0.slvr-cbr1.lnh-slvr.md.cable.rcncustomer.com. [207.172.141.204]) by smtp.gmail.com with ESMTPSA id h7-20020a05620a21c700b007853eb72c49sm154978qka.46.2024.02.01.13.40.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Feb 2024 13:40:33 -0800 (PST) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 87c20d68-c14a-11ee-8a43-1f161083a0e0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706823634; x=1707428434; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wX9UDy1UxiRe7Hnzf1ly24vLwl89/WjFYev/11/D2g4=; b=lmwXYq9LM8MDYhc6H6NeZ9DkM9l0o/2eeS8k5YUUgL8cbHEiUOnTzoQjlFf0+IEBMq JmHkaiFJ7Sl8IUXmKKNwsIiLxvG5PuFLqfCPTB9TxeYEVNvtHP4ZPM8XJWh0QzwQPC6B /cjRhFXvv1XF3dvRUt0OvXI6XZL00NRxpJExoo1wu4YTG2CiJmdlOE6Pk5fjpMw/9JoC lBWbsV2ffldjX0vFS5oSjOWTFVrF62EY2f1HFI3NwOqPR9ngjH2HVEM2adjIxBRlZMyt IeixCdBnR3rbJyiOMas062WoHg+ixhHFZ+32g2uts4T4GhGLPcVL+0gwUHFU/Rf1C36W hXNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706823634; x=1707428434; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wX9UDy1UxiRe7Hnzf1ly24vLwl89/WjFYev/11/D2g4=; b=vyLGAKyXASIwaYXbeOfDpEUNJ6BbiZXRolNAZU4DThH0a1QxRDwe1Iw81WSN4vKBVC H6Az96ABnOC0tXWxhTIJuB6bpt0WJxw9fWC4RaqIUPnZl01JpJb/p/11IOKYP9zrZcmT jAKOhFFiUc6utbfJYAYND8x3r6Q+VR+GaTWR7BJ7oqEYzNnsMaQnxTi0BnIDyRDcYdvy cplW068nFw1BbnG169fu2Y6uuRCaCnaedZAdDbe6gyymTI65W2ueABPpRHrneawotNkh wzcZi5kYKVQU9eQuX6aWZ2MwIvRjEpAs7TTNC0cHqcbG5gtIScJ0hO051nxyMD9/bd8N pItA== X-Gm-Message-State: AOJu0YyKwciZ/efvCYxYDgHlAeRCdR6RMQVkvv/4MRlCqlm59tBnBQAi UXI4I24YO3EXdx36nnx87hF6/tWRClAM/jf1Zrn7kSFyy5BHlF2vfwfXINge X-Google-Smtp-Source: AGHT+IEX178TypQ98U+CByn1kT4kVETUvyWOs9cB5bGAMhTVzG7Y7WLPgidUFDvGOT4uGCSHRj2/1g== X-Received: by 2002:a05:620a:248b:b0:783:9ab7:374 with SMTP id i11-20020a05620a248b00b007839ab70374mr4495234qkn.8.1706823633759; Thu, 01 Feb 2024 13:40:33 -0800 (PST) X-Forwarded-Encrypted: i=0; AJvYcCXgOnxKl8woBEDI2950HAGp1QbE3WzPvs7NrYFP6KZwAFr/Z3wtM10l01aHKQCriAJNUMBtPgK9eG5IGrATujMSueXD3gSuQKWYSKR9Td5DtC6cwhkdr+2Sbg27/1XQCtiq+cfNj0sLuSjnNKQHz73oesTxlZDozfKoZX2VjWfzq+GGpGwwKRvn1QwmRWl1zqexnlcZfT652TxoPLlhutevKomA3YfcMp61fC7guhh6brcBdCKLSuMgqKdIEgj+0vjVcy/ayyIuMW1PAIW7jehMB1Qgz4WqtfXmJXHZdrrc43BiK6SX4feM7k4= From: Jason Andryuk To: xen-devel@lists.xenproject.org Cc: marmarek@invisiblethingslab.com, Jason Andryuk , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu , Anthony PERARD , Juergen Gross Subject: [PATCH v2 3/3] libxl: Enable stubdom cdrom changing Date: Thu, 1 Feb 2024 16:40:04 -0500 Message-ID: <20240201214004.238858-4-jandryuk@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240201214004.238858-1-jandryuk@gmail.com> References: <20240201214004.238858-1-jandryuk@gmail.com> MIME-Version: 1.0 To change the cd-rom medium, libxl will: - QMP eject the medium from QEMU - block-detach the old PV disk - block-attach the new PV disk - QMP change the medium to the new PV disk by fdset-id The QMP code is reused, and remove and attach are implemented here. The stubdom must internally handle adding /dev/xvdc to the appropriate fdset. libxl in dom0 doesn't see the result of adding to the fdset as that is internal to the stubdom, so a delay and retries are added to around calling cdrom_insert_addfd_cb(). For cd-eject, we still need to attach the empty vbd. This is necessary since xenstore is used to determine that hdc exists. Otherwise after eject, hdc would be gone and the cd-insert would fail to find the drive to insert new media. The ERROR_JSON_CONFIG_EMPTY check in cdrom_insert_inserted() is because a stubdom don't have a json config. Signed-off-by: Jason Andryuk --- v2: Only allow for Linux stubdoms (Marek) Fix cd-eject Fix errant hard tabs Move the debug print and into the special case. --- docs/misc/stubdom.txt | 16 ++ tools/libs/light/libxl_disk.c | 298 +++++++++++++++++++++++++++++++--- 2 files changed, 288 insertions(+), 26 deletions(-) diff --git a/docs/misc/stubdom.txt b/docs/misc/stubdom.txt index c717a95d17..1b2380ae8f 100644 --- a/docs/misc/stubdom.txt +++ b/docs/misc/stubdom.txt @@ -127,6 +127,22 @@ Limitations: - at most 26 emulated disks are supported (more are still available as PV disks) - graphics output (VNC/SDL/Spice) not supported +CD-ROM changing: + +To change the CD-ROM medium, libxl will: + - QMP eject the medium from QEMU + - block-detach the old PV disk + - block-attach the new PV disk + - QMP change the medium to the new PV disk by fdset-id + +The QMP change insert uses fdset-id STUBDOM_FDSET_CD + $disk - 'a'. +That is, hda -> 'a', so +STUBDOM_FDSET_CD + 'a' - 'a' = STUBDOM_FDSET_CD. +For hdc: +STUBDOM_FDSET_CD + 'c' - 'a' = STUBDOM_FDSET_CD + 2. + +The stubdom must internally handle adding /dev/xvdc to the appropriate +fdset. PV-GRUB ======= diff --git a/tools/libs/light/libxl_disk.c b/tools/libs/light/libxl_disk.c index 09082ffb58..6354982c05 100644 --- a/tools/libs/light/libxl_disk.c +++ b/tools/libs/light/libxl_disk.c @@ -808,25 +808,46 @@ int libxl_device_disk_getinfo(libxl_ctx *ctx, uint32_t domid, typedef struct { libxl__ao *ao; + libxl__ao_device aodev; + libxl__ao_device aodev_del; libxl_domid domid; + libxl_domid disk_domid; libxl_device_disk *disk; libxl_device_disk disk_saved; libxl__ev_slowlock qmp_lock; int dm_ver; libxl__ev_time time; + libxl__ev_time timeout_retry; libxl__ev_qmp qmp; + int retries; + int stubdom_fdset; } libxl__cdrom_insert_state; static void cdrom_insert_lock_acquired(libxl__egc *, libxl__ev_slowlock *, int rc); static void cdrom_insert_qmp_connected(libxl__egc *, libxl__ev_qmp *, const libxl__json_object *, int rc); +static void cdrom_insert_stubdom_removefd(libxl__egc *egc, libxl__ev_qmp *qmp, + const libxl__json_object *response, + int rc); +static void cdrom_insert_stubdom_ejected(libxl__egc *egc, libxl__ev_qmp *, + const libxl__json_object *, int rc); +static void cdrom_insert_stubdom_disk_ejected_aocomplete(libxl__egc *egc, + libxl__ao_device *aodev); +static void cdrom_insert_stubdom_disk_ejected(libxl__egc *egc, libxl__ev_qmp *, + const libxl__json_object *, + int rc); +static void cdrom_insert_ejected_aodevcb(libxl__egc *egc, + libxl__ao_device *aodev); static void cdrom_insert_ejected(libxl__egc *egc, libxl__ev_qmp *, const libxl__json_object *, int rc); static void cdrom_insert_addfd_cb(libxl__egc *egc, libxl__ev_qmp *, const libxl__json_object *, int rc); static void cdrom_insert_inserted(libxl__egc *egc, libxl__ev_qmp *, const libxl__json_object *, int rc); +static void cdrom_insert_addfd_retry(libxl__egc *egc, libxl__ev_time *ev, + const struct timeval *requested_abs, + int rc); static void cdrom_insert_timout(libxl__egc *egc, libxl__ev_time *ev, const struct timeval *requested_abs, int rc); @@ -842,6 +863,7 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk, libxl_device_disk *disks = NULL; int rc; libxl__cdrom_insert_state *cis; + libxl_domid stubdomid; GCNEW(cis); cis->ao = ao; @@ -853,6 +875,8 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk, cis->qmp_lock.ao = ao; cis->qmp_lock.domid = domid; libxl__ev_time_init(&cis->time); + libxl__ev_time_init(&cis->timeout_retry); + cis->retries = 0; libxl__ev_qmp_init(&cis->qmp); cis->qmp.ao = ao; cis->qmp.domid = domid; @@ -869,12 +893,6 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk, goto out; } - if (libxl_get_stubdom_id(ctx, domid) != 0) { - LOGD(ERROR, domid, "cdrom-insert doesn't work for stub domains"); - rc = ERROR_INVAL; - goto out; - } - cis->dm_ver = libxl__device_model_version_running(gc, domid); if (cis->dm_ver == -1) { LOGD(ERROR, domid, "Cannot determine device model version"); @@ -882,7 +900,22 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk, goto out; } - disks = libxl__device_list(gc, &libxl__disk_devtype, domid, &num); + stubdomid = libxl_get_stubdom_id(CTX, cis->domid); + if (stubdomid == 0) { + cis->disk_domid = domid; + } else { + cis->disk_domid = stubdomid; + disk->backend = LIBXL_DISK_BACKEND_PHY; + } + + if (cis->dm_ver == LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL && + stubdomid) { + LOGD(ERROR, domid, "cdrom-insert doesn't work for Mini-OS stubdoms"); + rc = ERROR_INVAL; + goto out; + } + + disks = libxl__device_list(gc, &libxl__disk_devtype, cis->disk_domid, &num); for (i = 0; i < num; i++) { if (disks[i].is_cdrom && !strcmp(disk->vdev, disks[i].vdev)) { @@ -897,7 +930,7 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk, goto out; } - rc = libxl__device_disk_setdefault(gc, domid, disk, false); + rc = libxl__device_disk_setdefault(gc, cis->disk_domid, disk, false); if (rc) goto out; if (!disk->pdev_path) { @@ -905,6 +938,18 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk, disk->format = LIBXL_DISK_FORMAT_EMPTY; } +#define STUBDOM_FDSET_CD 8000 + if (strncmp(disk->vdev, "hd", 2) == 0) { + cis->stubdom_fdset = STUBDOM_FDSET_CD + disk->vdev[2] - 'a'; + } else if (strncmp(disk->vdev, "xvd", 3) == 0) { + cis->stubdom_fdset = STUBDOM_FDSET_CD + disk->vdev[3] - 'a'; + } else { + LOGD(ERROR, cis->domid, "disk->vdev \"%s\" isn't hdX or xvdY", + disk->vdev); + rc = ERROR_FAIL; + goto out; + } + out: libxl__device_list_free(&libxl__disk_devtype, disks, num); if (rc) { @@ -923,6 +968,7 @@ static void cdrom_insert_lock_acquired(libxl__egc *egc, libxl__cdrom_insert_state *cis = CONTAINER_OF(lock, *cis, qmp_lock); STATE_AO_GC(cis->ao); + LOGD(DEBUG, cis->domid, "rc=%d", rc); if (rc) goto out; rc = libxl__ev_time_register_rel(ao, &cis->time, @@ -971,7 +1017,12 @@ static void cdrom_insert_qmp_connected(libxl__egc *egc, libxl__ev_qmp *qmp, QMP_PARAMETERS_SPRINTF(&args, "id", "ide-%i", devid); else QMP_PARAMETERS_SPRINTF(&args, "device", "ide-%i", devid); - qmp->callback = cdrom_insert_ejected; + + if (libxl_get_stubdom_id(CTX, cis->domid)) + qmp->callback = cdrom_insert_stubdom_removefd; + else + qmp->callback = cdrom_insert_ejected; + rc = libxl__ev_qmp_send(egc, qmp, "eject", args); if (rc) goto out; return; @@ -979,6 +1030,148 @@ out: cdrom_insert_done(egc, cis, rc); /* must be last */ } +static void cdrom_insert_stubdom_removefd(libxl__egc *egc, libxl__ev_qmp *qmp, + const libxl__json_object *response, + int rc) +{ + libxl__cdrom_insert_state *cis = CONTAINER_OF(qmp, *cis, qmp); + STATE_AO_GC(cis->ao); + + if (rc) goto out; + + /* Only called for qemu-xen/linux stubdom. */ + assert(cis->dm_ver == LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN); + libxl__json_object *args = NULL; + + libxl__qmp_param_add_integer(gc, &args, "fdset-id", cis->stubdom_fdset); + + cis->qmp.callback = cdrom_insert_stubdom_ejected; + + rc = libxl__ev_qmp_send(egc, &cis->qmp, "remove-fd", args); + if (rc) goto out; + + return; + +out: + cdrom_insert_done(egc, cis, rc); /* must be last */ +} + +static void cdrom_insert_stubdom_ejected(libxl__egc *egc, libxl__ev_qmp *qmp, + const libxl__json_object *response, + int rc) +{ + libxl__cdrom_insert_state *cis = CONTAINER_OF(qmp, *cis, qmp); + libxl__device *device; + STATE_AO_GC(cis->ao); + domid_t stubdomid = libxl_get_stubdom_id(CTX, cis->domid); + + LOGD(DEBUG, cis->domid, "rc=%d", rc); + /* cis->stubdom_fdset is initially empty, so remove-fd fails the first + * call with: + * {"error": {"class": "GenericError", + * "desc": "File descriptor named 'fdset-id:8675' not found"}} + * Carry on in that case. */ + if (rc && rc != ERROR_QMP_GENERIC_ERROR) goto out; + + GCNEW(device); + rc = libxl__device_from_disk(gc, stubdomid, cis->disk, device); + if (rc != 0) goto out; + + /* block dev eject */ + /* Below is basically an open coding of: + * libxl_device_disk__remove(CTX, cis->domid, cis->disk, 0); + * ...since we can't call it from within libxl. + */ + libxl__prepare_ao_device(ao, &cis->aodev_del); + cis->aodev_del.action = LIBXL__DEVICE_ACTION_REMOVE; + cis->aodev_del.dev = device; + cis->aodev_del.callback = cdrom_insert_stubdom_disk_ejected_aocomplete; + cis->aodev_del.force.flag = LIBXL__FORCE_OFF; + libxl__initiate_device_generic_remove(egc, &cis->aodev_del); + return; + + out: + cdrom_insert_done(egc, cis, rc); /* must be last */ +} + +static void cdrom_insert_stubdom_disk_ejected_aocomplete(libxl__egc *egc, + libxl__ao_device *aodev) +{ + STATE_AO_GC(aodev->ao); + libxl__cdrom_insert_state *cis = CONTAINER_OF(aodev, *cis, aodev_del); + + LOGD(DEBUG, cis->domid, "rc=%d", aodev->rc); + if (aodev->rc) { + if (aodev->dev) { + LOGD(ERROR, aodev->dev->domid, "Unable to %s %s with id %u", + libxl__device_action_to_string(aodev->action), + libxl__device_kind_to_string(aodev->dev->kind), + aodev->dev->devid); + } else { + LOG(ERROR, "unable to %s device", + libxl__device_action_to_string(aodev->action)); + } + goto out; + } + + cdrom_insert_stubdom_disk_ejected(egc, &cis->qmp, NULL, aodev->rc); + return; + + out: + cdrom_insert_done(egc, cis, aodev->rc); +} + +static void cdrom_insert_stubdom_disk_ejected(libxl__egc *egc, + libxl__ev_qmp *qmp, + const libxl__json_object *response, + int rc) +{ + libxl__cdrom_insert_state *cis = CONTAINER_OF(qmp, *cis, qmp); + STATE_AO_GC(cis->ao); + domid_t stubdomid = libxl_get_stubdom_id(CTX, cis->domid); + + LOGD(DEBUG, cis->domid, "rc=%d", rc); + if (rc) goto out; + + /* block dev insert - this may be inserting an empty disk for eject. */ + libxl__prepare_ao_device(ao, &cis->aodev); + /* set an ao callback to end up in cdrom_insert_ejected */ + cis->aodev.callback = cdrom_insert_ejected_aodevcb; + libxl__device_disk_add(egc, stubdomid, cis->disk, &cis->aodev); + + return; + + out: + cdrom_insert_done(egc, cis, rc); /* must be last */ +} + +static void cdrom_insert_ejected_aodevcb(libxl__egc *egc, + libxl__ao_device *aodev) +{ + STATE_AO_GC(aodev->ao); + libxl__cdrom_insert_state *cis = CONTAINER_OF(aodev, *cis, aodev); + + LOGD(DEBUG, cis->domid, "rc=%d", aodev->rc); + if (aodev->rc) { + if (aodev->dev) { + LOGD(ERROR, aodev->dev->domid, "Unable to %s %s with id %u", + libxl__device_action_to_string(aodev->action), + libxl__device_kind_to_string(aodev->dev->kind), + aodev->dev->devid); + } else { + LOG(ERROR, "unable to %s device", + libxl__device_action_to_string(aodev->action)); + } + goto out; + } + + cdrom_insert_ejected(egc, &cis->qmp, NULL, aodev->rc); + return; + + out: + cdrom_insert_done(egc, cis, aodev->rc); +} + static void cdrom_insert_ejected(libxl__egc *egc, libxl__ev_qmp *qmp, const libxl__json_object *response, @@ -1001,9 +1194,10 @@ static void cdrom_insert_ejected(libxl__egc *egc, libxl_domain_config_init(&d_config); + LOGD(DEBUG, cis->domid, "rc=%d", rc); if (rc) goto out; - rc = libxl__device_from_disk(gc, domid, disk, &device); + rc = libxl__device_from_disk(gc, cis->disk_domid, disk, &device); if (rc) goto out; be_path = libxl__device_backend_path(gc, &device); libxl_path = libxl__device_libxl_path(gc, &device); @@ -1050,7 +1244,7 @@ static void cdrom_insert_ejected(libxl__egc *egc, */ rc = libxl__get_domain_configuration(gc, domid, &d_config); - if (rc) goto out; + if (rc && rc != ERROR_JSON_CONFIG_EMPTY) goto out; device_add_domain_config(gc, &d_config, &libxl__disk_devtype, &cis->disk_saved); @@ -1058,10 +1252,15 @@ static void cdrom_insert_ejected(libxl__egc *egc, rc = libxl__dm_check_start(gc, &d_config, domid); if (rc) goto out; + LOGD(DEBUG, cis->domid, "stubdom_id=%d", + libxl_get_stubdom_id(CTX, cis->domid)); + /* A linux stubdom will perform add-fd with calculated stubdom_fdset. */ if (cis->dm_ver == LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN && + libxl_get_stubdom_id(CTX, cis->domid) == 0 && disk->format != LIBXL_DISK_FORMAT_EMPTY) { libxl__json_object *args = NULL; + LOGD(DEBUG, cis->domid, "Doing qmp add-fd path"); assert(qmp->payload_fd == -1); qmp->payload_fd = open(disk->pdev_path, O_RDONLY); if (qmp->payload_fd < 0) { @@ -1080,20 +1279,30 @@ static void cdrom_insert_ejected(libxl__egc *egc, if (rc) goto out; has_callback = true; } else { + LOGD(DEBUG, cis->domid, "Skipping qmp add-fd path"); has_callback = false; } rc = 0; out: + LOGD(DEBUG, cis->domid, "out label rc=%d", rc); libxl__xs_transaction_abort(gc, &t); libxl_domain_config_dispose(&d_config); if (data_lock) libxl__unlock_file(data_lock); if (rc) { cdrom_insert_done(egc, cis, rc); /* must be last */ } else if (!has_callback) { - /* Only called if no asynchronous callback are set. */ - cdrom_insert_inserted(egc, qmp, NULL, 0); /* must be last */ + if (libxl_get_stubdom_id(CTX, cis->domid) && + disk->format != LIBXL_DISK_FORMAT_EMPTY) { + LOGD(DEBUG, cis->domid, + "stubdom %d needs to perform add-fd internally", + libxl_get_stubdom_id(CTX, cis->domid)); + cdrom_insert_addfd_cb(egc, qmp, NULL, 0); /* must be last */ + } else { + /* Only called if no asynchronous callback are set. */ + cdrom_insert_inserted(egc, qmp, NULL, 0); /* must be last */ + } } } @@ -1112,17 +1321,24 @@ static void cdrom_insert_addfd_cb(libxl__egc *egc, /* convenience aliases */ libxl_device_disk *disk = cis->disk; - close(qmp->payload_fd); - qmp->payload_fd = -1; + LOGD(DEBUG, cis->domid, "rc=%d", rc); if (rc) goto out; - o = libxl__json_map_get("fdset-id", response, JSON_INTEGER); - if (!o) { - rc = ERROR_FAIL; - goto out; + /* response non-NULL only for non-stubdom */ + if (response) { + close(qmp->payload_fd); + qmp->payload_fd = -1; + + o = libxl__json_map_get("fdset-id", response, JSON_INTEGER); + if (!o) { + rc = ERROR_FAIL; + goto out; + } + fdset = libxl__json_object_get_integer(o); + } else { + fdset = cis->stubdom_fdset; } - fdset = libxl__json_object_get_integer(o); devid = libxl__device_disk_dev_number(disk->vdev, NULL, NULL); qmp->callback = cdrom_insert_inserted; @@ -1135,8 +1351,13 @@ static void cdrom_insert_addfd_cb(libxl__egc *egc, if (libxl__qmp_ev_qemu_compare_version(qmp, 2, 8, 0) >= 0) { QMP_PARAMETERS_SPRINTF(&args, "id", "ide-%i", devid); QMP_PARAMETERS_SPRINTF(&args, "filename", "/dev/fdset/%d", fdset); - libxl__qmp_param_add_string(gc, &args, "format", - libxl__qemu_disk_format_string(disk->format)); + if (response) { + libxl__qmp_param_add_string(gc, &args, "format", + libxl__qemu_disk_format_string(disk->format)); + } else { + /* Stubdom is using blockdev /dev/xvd* */ + libxl__qmp_param_add_string(gc, &args, "format", "host_device"); + } rc = libxl__ev_qmp_send(egc, qmp, "blockdev-change-medium", args); } else { QMP_PARAMETERS_SPRINTF(&args, "device", "ide-%i", devid); @@ -1150,13 +1371,22 @@ out: cdrom_insert_done(egc, cis, rc); /* must be last */ } +static void cdrom_insert_addfd_retry(libxl__egc *egc, libxl__ev_time *ev, + const struct timeval *requested_abs, + int rc) +{ + libxl__cdrom_insert_state *cis = CONTAINER_OF(ev, *cis, timeout_retry); + + cdrom_insert_addfd_cb(egc, &cis->qmp, NULL, 0); +} + static void cdrom_insert_inserted(libxl__egc *egc, libxl__ev_qmp *qmp, const libxl__json_object *response, int rc) { - EGC_GC; libxl__cdrom_insert_state *cis = CONTAINER_OF(qmp, *cis, qmp); + STATE_AO_GC(cis->ao); libxl__flock *data_lock = NULL; libxl_domain_config d_config; flexarray_t *insert = NULL; @@ -1171,9 +1401,22 @@ static void cdrom_insert_inserted(libxl__egc *egc, libxl_domain_config_init(&d_config); - if (rc) goto out; + LOGD(DEBUG, cis->domid, "rc=%d", rc); + + if (rc) { + if (cis->retries++ < 10 ) { + LOGD(DEBUG, qmp->domid, "Retrying QMP cdrom change\n"); + rc = libxl__ev_time_register_rel(ao, &cis->timeout_retry, + cdrom_insert_addfd_retry, 100); + if (rc) goto out; - rc = libxl__device_from_disk(gc, domid, disk, &device); + return; + } else { + goto out; + } + } + + rc = libxl__device_from_disk(gc, cis->disk_domid, disk, &device); if (rc) goto out; be_path = libxl__device_backend_path(gc, &device); libxl_path = libxl__device_libxl_path(gc, &device); @@ -1185,7 +1428,7 @@ static void cdrom_insert_inserted(libxl__egc *egc, } rc = libxl__get_domain_configuration(gc, domid, &d_config); - if (rc) goto out; + if (rc && rc != ERROR_JSON_CONFIG_EMPTY) goto out; device_add_domain_config(gc, &d_config, &libxl__disk_devtype, &cis->disk_saved); @@ -1257,7 +1500,10 @@ static void cdrom_insert_done(libxl__egc *egc, { EGC_GC; + LOGD(DEBUG, cis->domid, "rc=%d", rc); + libxl__ev_time_deregister(gc, &cis->time); + libxl__ev_time_deregister(gc, &cis->timeout_retry); libxl__ev_qmp_dispose(gc, &cis->qmp); if (cis->qmp.payload_fd >= 0) close(cis->qmp.payload_fd); libxl__ev_slowlock_unlock(gc, &cis->qmp_lock);