From patchwork Thu Sep 19 11:04:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anthony PERARD X-Patchwork-Id: 11152191 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 539C314ED for ; Thu, 19 Sep 2019 11:06:42 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2483D21907 for ; Thu, 19 Sep 2019 11:06:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="LDqYBLmM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2483D21907 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iAuF8-0003B1-I5; Thu, 19 Sep 2019 11:04:58 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iAuF7-0003AJ-Mx for xen-devel@lists.xenproject.org; Thu, 19 Sep 2019 11:04:57 +0000 X-Inumbo-ID: 4a8b99ea-dacd-11e9-9656-12813bfff9fa Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 4a8b99ea-dacd-11e9-9656-12813bfff9fa; Thu, 19 Sep 2019 11:04:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1568891088; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=c/X1kAtunOTd0Rp883uPK6Xt10M+dQzHralPJlY9zws=; b=LDqYBLmMqWOdwy5s5wXXHnm7wIYDFzXUMf9APUYJc5DVE87f514/TVmY PWKY3jITL61IST00UyDRklENRRtvZhBxwPQZPL38bB3b/Y8R2wLjxjGJr TZPV+NSGMsvMoJeZDeruPK57On9ygc1gBJNetef9CT0v1ueZV9CUJISvG g=; Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=anthony.perard@citrix.com; spf=Pass smtp.mailfrom=anthony.perard@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of anthony.perard@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="anthony.perard@citrix.com"; x-sender="anthony.perard@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa2.hc3370-68.iphmx.com: domain of anthony.perard@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="anthony.perard@citrix.com"; x-sender="anthony.perard@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ~all" Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="anthony.perard@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: ysXiLdwkv5v76d55sl3LorW8/e3qU83mlkaLYiOkyK/eKnXbtUDlOefHsNWDCdp597II/b94Z2 KBwu0riujcTWXZw6w7Y1c5BEircF6JaZwFznry6ZYCzxp9zMxUZS5Q6xUBYQye+wtKmtWyPRmK 5SINhAIIll6aegyFBu23p26YuEophLuDpsp0fkvG6qV1GGyQhwNYvh9ENIKhKzPPyt4aP37a0/ 4zYBM6bU7Oc8Cz7Vt3NysGlnWfTUM+NZrgPf2mXoWOsas4bCcOrO8rt6V+7i0s8cs7cnCxDb0G UWM= X-SBRS: 2.7 X-MesageID: 5777958 X-Ironport-Server: esa2.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.64,523,1559534400"; d="scan'208";a="5777958" From: Anthony PERARD To: Date: Thu, 19 Sep 2019 12:04:40 +0100 Message-ID: <20190919110443.817594-7-anthony.perard@citrix.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190919110443.817594-1-anthony.perard@citrix.com> References: <20190919110443.817594-1-anthony.perard@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v3 6/9] libxl_disk: Cut libxl_cdrom_insert into steps .. X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Anthony PERARD , Ian Jackson , Wei Liu Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" .. and use a new "slow" lock to avoid holding the userdata lock across several functions. This patch cuts libxl_cdrom_insert into different step/function but there are still called synchronously. (Taking the ev_lock is the only step that might be asynchronous.) A later patch will call them asynchronously when QMP is involved. Thee userdata lock (json_lock) use to protect against concurrent change of cdrom is replaced by an ev_lock which can be held across different CTX_LOCK sections. The json_lock is still used when reading/modifying the domain userdata (mandatory) and update xenstore (mostly because it's updated as the same time as the userdata). Signed-off-by: Anthony PERARD Acked-by: Ian Jackson --- Notes: v2: - rewrite patch description - rework use of the new lock tools/libxl/libxl_disk.c | 196 +++++++++++++++++++++++++++++++-------- 1 file changed, 155 insertions(+), 41 deletions(-) diff --git a/tools/libxl/libxl_disk.c b/tools/libxl/libxl_disk.c index 45bf555061df..98023f169ce6 100644 --- a/tools/libxl/libxl_disk.c +++ b/tools/libxl/libxl_disk.c @@ -642,24 +642,43 @@ int libxl_device_disk_getinfo(libxl_ctx *ctx, uint32_t domid, return rc; } +typedef struct { + libxl__ao *ao; + libxl_domid domid; + libxl_device_disk *disk; + libxl_device_disk disk_saved; + libxl__ev_devlock qmp_lock; + int dm_ver; +} libxl__cdrom_insert_state; + +static void cdrom_insert_lock_acquired(libxl__egc *, libxl__ev_devlock *, + int rc); +static void cdrom_insert_ejected(libxl__egc *egc, + libxl__cdrom_insert_state *cis); +static void cdrom_insert_inserted(libxl__egc *egc, + libxl__cdrom_insert_state *cis); +static void cdrom_insert_done(libxl__egc *egc, + libxl__cdrom_insert_state *cis, + int rc); + int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk, const libxl_asyncop_how *ao_how) { AO_CREATE(ctx, domid, ao_how); int num = 0, i; - libxl_device_disk *disks = NULL, disk_saved; - libxl_domain_config d_config; - int rc, dm_ver; - libxl__device device; - const char *be_path, *libxl_path; - char * tmp; - libxl__domain_userdata_lock *lock = NULL; - xs_transaction_t t = XBT_NULL; - flexarray_t *insert = NULL, *empty = NULL; - - libxl_domain_config_init(&d_config); - libxl_device_disk_init(&disk_saved); - libxl_device_disk_copy(ctx, &disk_saved, disk); + libxl_device_disk *disks = NULL; + int rc; + libxl__cdrom_insert_state *cis; + + GCNEW(cis); + cis->ao = ao; + cis->domid = domid; + cis->disk = disk; + libxl_device_disk_init(&cis->disk_saved); + libxl_device_disk_copy(ctx, &cis->disk_saved, disk); + libxl__ev_devlock_init(&cis->qmp_lock); + cis->qmp_lock.ao = ao; + cis->qmp_lock.domid = domid; libxl_domain_type type = libxl__domain_type(gc, domid); if (type == LIBXL_DOMAIN_TYPE_INVALID) { @@ -678,8 +697,8 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk, goto out; } - dm_ver = libxl__device_model_version_running(gc, domid); - if (dm_ver == -1) { + cis->dm_ver = libxl__device_model_version_running(gc, domid); + if (cis->dm_ver == -1) { LOGD(ERROR, domid, "Cannot determine device model version"); rc = ERROR_FAIL; goto out; @@ -708,40 +727,82 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk, disk->format = LIBXL_DISK_FORMAT_EMPTY; } - rc = libxl__device_from_disk(gc, domid, disk, &device); - if (rc) goto out; +out: + libxl__device_list_free(&libxl__disk_devtype, disks, num); + if (rc) { + cdrom_insert_done(egc, cis, rc); /* must be last */ + } else { + cis->qmp_lock.callback = cdrom_insert_lock_acquired; + libxl__ev_devlock_lock(egc, &cis->qmp_lock); /* must be last */ + } + return AO_INPROGRESS; +} - be_path = libxl__device_backend_path(gc, &device); - libxl_path = libxl__device_libxl_path(gc, &device); +static void cdrom_insert_lock_acquired(libxl__egc *egc, + libxl__ev_devlock *lock, + int rc) +{ + libxl__cdrom_insert_state *cis = CONTAINER_OF(lock, *cis, qmp_lock); + STATE_AO_GC(cis->ao); - /* Note: CTX lock is already held at this point so lock hierarchy - * is maintained. - */ - lock = libxl__lock_domain_userdata(gc, domid); - if (!lock) { - rc = ERROR_LOCK_FAIL; - goto out; - } + if (rc) goto out; /* We need to eject the original image first. This is implemented * by inserting empty media. JSON is not updated. */ - if (dm_ver == LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN) { + if (cis->dm_ver == LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN) { libxl_device_disk disk_empty; libxl_device_disk_init(&disk_empty); disk_empty.format = LIBXL_DISK_FORMAT_EMPTY; - disk_empty.vdev = libxl__strdup(NOGC, disk->vdev); + disk_empty.vdev = libxl__strdup(NOGC, cis->disk->vdev); disk_empty.pdev_path = libxl__strdup(NOGC, ""); disk_empty.is_cdrom = 1; - libxl__device_disk_setdefault(gc, domid, &disk_empty, false); + libxl__device_disk_setdefault(gc, cis->domid, &disk_empty, false); - rc = libxl__qmp_insert_cdrom(gc, domid, &disk_empty); + rc = libxl__qmp_insert_cdrom(gc, cis->domid, &disk_empty); libxl_device_disk_dispose(&disk_empty); if (rc) goto out; } + cdrom_insert_ejected(egc, cis); /* must be last */ + return; + +out: + cdrom_insert_done(egc, cis, rc); /* must be last */ +} + +static void cdrom_insert_ejected(libxl__egc *egc, + libxl__cdrom_insert_state *cis) +{ + EGC_GC; + int rc; + libxl__domain_userdata_lock *data_lock = NULL; + libxl__device device; + const char *be_path, *libxl_path; + flexarray_t *empty = NULL; + xs_transaction_t t = XBT_NULL; + char *tmp; + libxl_domain_config d_config; + + /* convenience aliases */ + libxl_domid domid = cis->domid; + libxl_device_disk *disk = cis->disk; + + libxl_domain_config_init(&d_config); + + rc = libxl__device_from_disk(gc, domid, disk, &device); + if (rc) goto out; + be_path = libxl__device_backend_path(gc, &device); + libxl_path = libxl__device_libxl_path(gc, &device); + + data_lock = libxl__lock_domain_userdata(gc, domid); + if (!data_lock) { + rc = ERROR_LOCK_FAIL; + goto out; + } + empty = flexarray_make(gc, 4, 1); flexarray_append_pair(empty, "type", libxl__device_disk_string_of_backend(disk->backend)); @@ -780,16 +841,66 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk, rc = libxl__get_domain_configuration(gc, domid, &d_config); if (rc) goto out; - device_add_domain_config(gc, &d_config, &libxl__disk_devtype, &disk_saved); + device_add_domain_config(gc, &d_config, &libxl__disk_devtype, + &cis->disk_saved); rc = libxl__dm_check_start(gc, &d_config, domid); if (rc) goto out; - if (dm_ver == LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN) { + if (cis->dm_ver == LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN) { rc = libxl__qmp_insert_cdrom(gc, domid, disk); if (rc) goto out; } + rc = 0; + +out: + libxl__xs_transaction_abort(gc, &t); + libxl_domain_config_dispose(&d_config); + if (data_lock) libxl__unlock_domain_userdata(data_lock); + if (rc) { + cdrom_insert_done(egc, cis, rc); /* must be last */ + } else { + cdrom_insert_inserted(egc, cis); /* must be last */ + } +} + +static void cdrom_insert_inserted(libxl__egc *egc, + libxl__cdrom_insert_state *cis) +{ + EGC_GC; + int rc; + libxl__domain_userdata_lock *data_lock = NULL; + libxl_domain_config d_config; + flexarray_t *insert = NULL; + xs_transaction_t t = XBT_NULL; + libxl__device device; + const char *be_path, *libxl_path; + char *tmp; + + /* convenience aliases */ + libxl_domid domid = cis->domid; + libxl_device_disk *disk = cis->disk; + + libxl_domain_config_init(&d_config); + + rc = libxl__device_from_disk(gc, domid, disk, &device); + if (rc) goto out; + be_path = libxl__device_backend_path(gc, &device); + libxl_path = libxl__device_libxl_path(gc, &device); + + data_lock = libxl__lock_domain_userdata(gc, domid); + if (!data_lock) { + rc = ERROR_LOCK_FAIL; + goto out; + } + + rc = libxl__get_domain_configuration(gc, domid, &d_config); + if (rc) goto out; + + device_add_domain_config(gc, &d_config, &libxl__disk_devtype, + &cis->disk_saved); + insert = flexarray_make(gc, 4, 1); flexarray_append_pair(insert, "type", libxl__device_disk_string_of_backend(disk->backend)); @@ -830,21 +941,24 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk, if (rc < 0) goto out; } - /* success, no actual async */ - libxl__ao_complete(egc, ao, 0); - rc = 0; out: libxl__xs_transaction_abort(gc, &t); - libxl__device_list_free(&libxl__disk_devtype, disks, num); - libxl_device_disk_dispose(&disk_saved); libxl_domain_config_dispose(&d_config); + if (data_lock) libxl__unlock_domain_userdata(data_lock); + cdrom_insert_done(egc, cis, rc); /* must be last */ +} - if (lock) libxl__unlock_domain_userdata(lock); +static void cdrom_insert_done(libxl__egc *egc, + libxl__cdrom_insert_state *cis, + int rc) +{ + EGC_GC; - if (rc) return AO_CREATE_FAIL(rc); - return AO_INPROGRESS; + libxl__ev_devlock_unlock(gc, &cis->qmp_lock); + libxl_device_disk_dispose(&cis->disk_saved); + libxl__ao_complete(egc, cis->ao, rc); } /* libxl__alloc_vdev only works on the local domain, that is the domain