From patchwork Fri Feb 21 11:20:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Durrant X-Patchwork-Id: 11396287 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BC70314E3 for ; Fri, 21 Feb 2020 11:22:13 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8390A24672 for ; Fri, 21 Feb 2020 11:22:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="vy0We7ja" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8390A24672 Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1j56Mj-000530-Jy; Fri, 21 Feb 2020 11:21:05 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1j56Mi-00052u-D7 for xen-devel@lists.xenproject.org; Fri, 21 Feb 2020 11:21:04 +0000 X-Inumbo-ID: 3ed701c0-549c-11ea-bc8e-bc764e2007e4 Received: from smtp-fw-9101.amazon.com (unknown [207.171.184.25]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 3ed701c0-549c-11ea-bc8e-bc764e2007e4; Fri, 21 Feb 2020 11:21:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1582284064; x=1613820064; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=OmblxcrSY+biuTqXXgXi966byZ/MlmescutJ3zjWh1s=; b=vy0We7jacXfWwA+qSGhgosZyEhqNPbm7PaMRy0qNsoFV+8kU26ngiVeH 3humJjlPX93W3krCu24eHCHlXhYeMBxToPj908NusurhrtyISkiO13CUV 9JNHBkB8AjPwIKOcLcaLBuPhk0rgOzmJuM4l8N7SUql/r6JGdlH8un/Nn w=; IronPort-SDR: JQL63GKOBQ3VmYd5odiI9HyVJk4xEaIMutKLuWANoEENWBel13ud58rRjrHdmNfuSHQ1toUjb8 cAyXXCCMQCQg== X-IronPort-AV: E=Sophos;i="5.70,468,1574121600"; d="scan'208";a="18290019" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-1d-74cf8b49.us-east-1.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-9101.sea19.amazon.com with ESMTP; 21 Feb 2020 11:21:01 +0000 Received: from EX13MTAUEA002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan2.iad.amazon.com [10.40.159.162]) by email-inbound-relay-1d-74cf8b49.us-east-1.amazon.com (Postfix) with ESMTPS id 54F21C1E5A; Fri, 21 Feb 2020 11:20:59 +0000 (UTC) Received: from EX13D32EUB002.ant.amazon.com (10.43.166.114) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1236.3; Fri, 21 Feb 2020 11:20:59 +0000 Received: from EX13MTAUEB002.ant.amazon.com (10.43.60.12) by EX13D32EUB002.ant.amazon.com (10.43.166.114) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 21 Feb 2020 11:20:58 +0000 Received: from u2f063a87eabd5f.cbg10.amazon.com (10.125.106.135) by mail-relay.amazon.com (10.43.60.234) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Fri, 21 Feb 2020 11:20:57 +0000 From: Paul Durrant To: Date: Fri, 21 Feb 2020 11:20:44 +0000 Message-ID: <20200221112049.3077-2-pdurrant@amazon.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200221112049.3077-1-pdurrant@amazon.com> References: <20200221112049.3077-1-pdurrant@amazon.com> MIME-Version: 1.0 Precedence: Bulk Subject: [Xen-devel] [PATCH v7 1/6] libxl: add infrastructure to track and query 'recent' domids X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Anthony PERARD , Paul Durrant , Ian Jackson , Wei Liu Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" A domid is considered recent if the domain it represents was destroyed less than a specified number of seconds ago. For debugging and/or testing purposes the number can be set using the environment variable LIBXL_DOMID_REUSE_TIMEOUT. If the variable does not exist then a default value of 60s is used. Whenever a domain is destroyed, a time-stamped record will be written into a history file (/var/run/xen/domid-history). To avoid the history file growing too large, any records with time-stamps that indicate that the age of a domid has exceeded the re-use timeout will also be purged. A new utility function, libxl__is_recent_domid(), has been added. This function reads the same history file checking whether a specified domid has a record that does not exceed the re-use timeout. Since this utility function does not write to the file, no records are actually purged by it. NOTE: The history file is purged on boot to it is safe to use CLOCK_MONOTONIC as a time source. Signed-off-by: Paul Durrant Reviewed-by: Ian Jackson --- Cc: Ian Jackson Cc: Wei Liu Cc: Anthony PERARD v7: - Addressed further comments from Ian - Introduced 'struct libxl__domid_history' to hold context v6: _ Addressed further comments from Ian v5: - Re-work file manipulation some more - Add more error checks v4: - Use new generalised libxl__flock - Don't read and write the same file - Use 'recent' rather than 'retired' - Add code into xen-init-dom0 to delete an old history file at boot v2: - New in v2 --- tools/helpers/xen-init-dom0.c | 30 +++++ tools/libxl/libxl.h | 7 ++ tools/libxl/libxl_domain.c | 226 ++++++++++++++++++++++++++++++++++ tools/libxl/libxl_internal.c | 10 ++ tools/libxl/libxl_internal.h | 14 +++ 5 files changed, 287 insertions(+) diff --git a/tools/helpers/xen-init-dom0.c b/tools/helpers/xen-init-dom0.c index a1e5729458..56f69ab66f 100644 --- a/tools/helpers/xen-init-dom0.c +++ b/tools/helpers/xen-init-dom0.c @@ -12,6 +12,32 @@ #define DOMNAME_PATH "/local/domain/0/name" #define DOMID_PATH "/local/domain/0/domid" +int clear_domid_history(void) +{ + int rc = 1; + xentoollog_logger_stdiostream *logger; + libxl_ctx *ctx; + + logger = xtl_createlogger_stdiostream(stderr, XTL_ERROR, 0); + if (!logger) + return 1; + + if (libxl_ctx_alloc(&ctx, LIBXL_VERSION, 0, + (xentoollog_logger *)logger)) { + fprintf(stderr, "cannot init libxl context\n"); + goto outlog; + } + + if (!libxl_clear_domid_history(ctx)) + rc = 0; + + libxl_ctx_free(ctx); + +outlog: + xtl_logger_destroy((xentoollog_logger *)logger); + return rc; +} + int main(int argc, char **argv) { int rc; @@ -70,6 +96,10 @@ int main(int argc, char **argv) if (rc) goto out; + rc = clear_domid_history(); + if (rc) + goto out; + /* Write xenstore entries. */ if (!xs_write(xsh, XBT_NULL, DOMID_PATH, "0", strlen("0"))) { fprintf(stderr, "cannot set domid for Dom0\n"); diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h index fde8548847..80ae110a52 100644 --- a/tools/libxl/libxl.h +++ b/tools/libxl/libxl.h @@ -2679,6 +2679,13 @@ static inline int libxl_qemu_monitor_command_0x041200(libxl_ctx *ctx, #include +/* + * This function is for use only during host initialisation. If it is + * invoked on a host with running domains, or concurrent libxl + * processes then the system may malfuntion. + */ +int libxl_clear_domid_history(libxl_ctx *ctx); + #endif /* LIBXL_H */ /* diff --git a/tools/libxl/libxl_domain.c b/tools/libxl/libxl_domain.c index 973fc1434d..8937aeb260 100644 --- a/tools/libxl/libxl_domain.c +++ b/tools/libxl/libxl_domain.c @@ -1268,6 +1268,230 @@ static void dm_destroy_cb(libxl__egc *egc, libxl__devices_destroy(egc, &dis->drs); } +static unsigned int libxl__get_domid_reuse_timeout(void) +{ + const char *env_timeout = getenv("LIBXL_DOMID_REUSE_TIMEOUT"); + + return env_timeout ? strtol(env_timeout, NULL, 0) : + LIBXL_DOMID_REUSE_TIMEOUT; +} + +char *libxl__domid_history_path(libxl__gc *gc, const char *suffix) +{ + return GCSPRINTF("%s/domid-history%s", libxl__run_dir_path(), + suffix ?: ""); +} + +int libxl_clear_domid_history(libxl_ctx *ctx) +{ + GC_INIT(ctx); + char *path; + int rc = ERROR_FAIL; + + path = libxl__domid_history_path(gc, NULL); + if (!path) + goto out; + + if (unlink(path) < 0 && errno != ENOENT) { + LOGE(ERROR, "failed to remove '%s'\n", path); + goto out; + } + + rc = 0; + +out: + GC_FREE; + return rc; +} + +struct libxl__domid_history { + long timeout; + char *path; + FILE *f; + struct timespec ts; +}; + +static void libxl__domid_history_dispose( + struct libxl__domid_history *ctxt) +{ + if (ctxt->f) { + fclose(ctxt->f); + ctxt->f = NULL; + } +} + +static int libxl__open_domid_history(libxl__gc *gc, + struct libxl__domid_history *ctxt) +{ + ctxt->timeout = libxl__get_domid_reuse_timeout(); + ctxt->path = libxl__domid_history_path(gc, NULL); + + ctxt->f = fopen(ctxt->path, "r"); + if (!ctxt->f && errno != ENOENT) { + LOGE(ERROR, "failed to open '%s'", ctxt->path); + return ERROR_FAIL; + } + + if (clock_gettime(CLOCK_MONOTONIC, &ctxt->ts)) { + LOGE(ERROR, "failed to get time"); + libxl__domid_history_dispose(ctxt); + return ERROR_FAIL; + } + + return 0; +} + +static int libxl__close_domid_history(libxl__gc *gc, + struct libxl__domid_history *ctxt) +{ + int r; + + if (!ctxt->f) return 0; + + r = fclose(ctxt->f); + ctxt->f = NULL; + if (r == EOF) { + LOGE(ERROR, "failed to close '%s'", ctxt->path); + return ERROR_FAIL; + } + + return 0; +} + +static int libxl__read_recent(libxl__gc *gc, + struct libxl__domid_history *ctxt, + unsigned long *sec, unsigned int *domid) +{ + if (!ctxt->f) { + *domid = INVALID_DOMID; + return 0; + } + + for (;;) { + int r = fscanf(ctxt->f, "%lu %u", sec, domid); + + if (r == EOF) { + if (ferror(ctxt->f)) { + LOGE(ERROR, "failed to read from '%s'", ctxt->path); + return ERROR_FAIL; + } + + *domid = INVALID_DOMID; + break; + } else if (r == 2 && libxl_domid_valid_guest(*domid) && + ctxt->ts.tv_sec - *sec <= ctxt->timeout) { + break; + } + } + + return 0; +} + +static int libxl__mark_domid_recent(libxl__gc *gc, uint32_t domid) +{ + libxl__flock *lock; + struct libxl__domid_history ctxt; + char *new; + FILE *nf = NULL; + int r, rc; + + lock = libxl__lock_domid_history(gc); + if (!lock) { + LOGED(ERROR, domid, "failed to acquire lock"); + rc = ERROR_FAIL; + goto out; + } + + rc = libxl__open_domid_history(gc, &ctxt); + if (rc) goto out; + + new = libxl__domid_history_path(gc, ".new"); + nf = fopen(new, "a"); + if (!nf) { + LOGED(ERROR, domid, "failed to open '%s'", new); + goto out; + } + + for (;;) { + unsigned long sec; + unsigned int val; + + rc = libxl__read_recent(gc, &ctxt, &sec, &val); + if (rc) goto out; + + if (val == INVALID_DOMID) /* EOF */ + break; + + r = fprintf(nf, "%lu %u\n", sec, val); + if (r < 0) { + LOGED(ERROR, domid, "failed to write to '%s'", new); + goto out; + } + } + + r = fprintf(nf, "%lu %u\n", ctxt.ts.tv_sec, domid); + if (r < 0) { + LOGED(ERROR, domid, "failed to write to '%s'", new); + goto out; + } + + r = fclose(nf); + nf = NULL; + if (r == EOF) { + LOGED(ERROR, domid, "failed to close '%s'", new); + goto out; + } + + rc = libxl__close_domid_history(gc, &ctxt); + if (rc) goto out; + + r = rename(new, ctxt.path); + if (r) { + LOGE(ERROR, "failed to rename '%s' -> '%s'", new, ctxt.path); + return ERROR_FAIL; + } + +out: + if (nf) fclose(nf); + libxl__domid_history_dispose(&ctxt); + if (lock) libxl__unlock_file(lock); + + return rc; +} + +int libxl__is_domid_recent(libxl__gc *gc, uint32_t domid, bool *recent) +{ + struct libxl__domid_history ctxt; + int rc; + + rc = libxl__open_domid_history(gc, &ctxt); + if (rc) goto out; + + *recent = false; + for (;;) { + unsigned long sec; + unsigned int val; + + rc = libxl__read_recent(gc, &ctxt, &sec, &val); + if (rc) goto out; + + if (val == INVALID_DOMID) /* EOF */ + break; + + if (val == domid && ctxt.ts.tv_sec - sec <= ctxt.timeout) { + *recent = true; + break; + } + } + + rc = libxl__close_domid_history(gc, &ctxt); + +out: + libxl__domid_history_dispose(&ctxt); + + return rc; +} + static void devices_destroy_cb(libxl__egc *egc, libxl__devices_remove_state *drs, int rc) @@ -1331,6 +1555,8 @@ static void devices_destroy_cb(libxl__egc *egc, if (!ctx->xch) goto badchild; if (!dis->soft_reset) { + rc = libxl__mark_domid_recent(gc, domid); + if (rc) goto badchild; rc = xc_domain_destroy(ctx->xch, domid); } else { rc = xc_domain_pause(ctx->xch, domid); diff --git a/tools/libxl/libxl_internal.c b/tools/libxl/libxl_internal.c index 211236dc99..bbd4c6cba9 100644 --- a/tools/libxl/libxl_internal.c +++ b/tools/libxl/libxl_internal.c @@ -504,6 +504,16 @@ libxl__flock *libxl__lock_domain_userdata(libxl__gc *gc, uint32_t domid) return lock; } +libxl__flock *libxl__lock_domid_history(libxl__gc *gc) +{ + const char *lockfile; + + lockfile = libxl__domid_history_path(gc, ".lock"); + if (!lockfile) return NULL; + + return libxl__lock_file(gc, lockfile); +} + int libxl__get_domain_configuration(libxl__gc *gc, uint32_t domid, libxl_domain_config *d_config) { diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h index 4936446069..43e5885d1e 100644 --- a/tools/libxl/libxl_internal.h +++ b/tools/libxl/libxl_internal.h @@ -4263,6 +4263,8 @@ _hidden void libxl__remus_teardown(libxl__egc *egc, _hidden void libxl__remus_restore_setup(libxl__egc *egc, libxl__domain_create_state *dcs); +_hidden char *libxl__domid_history_path(libxl__gc *gc, + const char *suffix); /* * Convenience macros. @@ -4661,6 +4663,7 @@ libxl__flock *libxl__lock_file(libxl__gc *gc, const char *filename); void libxl__unlock_file(libxl__flock *lock); libxl__flock *libxl__lock_domain_userdata(libxl__gc *gc, uint32_t domid); +libxl__flock *libxl__lock_domid_history(libxl__gc *gc); /* * Retrieve / store domain configuration from / to libxl private @@ -4799,6 +4802,17 @@ _hidden int libxl__domain_pvcontrol(libxl__egc *egc, libxl__xswait_state *pvcontrol, domid_t domid, const char *cmd); +/* + * Maximum number of seconds after desctruction then a domid remains + * 'recent'. Recent domids are not allowed to be re-used. This can be + * overidden, for debugging purposes, by the environment variable of the + * same name. + */ +#define LIBXL_DOMID_REUSE_TIMEOUT 60 + +/* Check whether a domid is recent */ +int libxl__is_domid_recent(libxl__gc *gc, uint32_t domid, bool *recent); + #endif /*