From patchwork Mon Mar 16 12:37:04 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hannes Reinecke X-Patchwork-Id: 6018371 X-Patchwork-Delegate: christophe.varoqui@free.fr Return-Path: X-Original-To: patchwork-dm-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id BE8BDBF90F for ; Mon, 16 Mar 2015 12:45:00 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id BCCBB204D8 for ; Mon, 16 Mar 2015 12:44:59 +0000 (UTC) Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8F66F204D1 for ; Mon, 16 Mar 2015 12:44:58 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t2GCcadf005409; Mon, 16 Mar 2015 08:38:36 -0400 Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id t2GCbMal029009 for ; Mon, 16 Mar 2015 08:37:22 -0400 Received: from mx1.redhat.com (ext-mx02.extmail.prod.ext.phx2.redhat.com [10.5.110.26]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t2GCbM0N004298 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 16 Mar 2015 08:37:22 -0400 Received: from mx2.suse.de (cantor2.suse.de [195.135.220.15]) by mx1.redhat.com (Postfix) with ESMTPS id 05EBB19B761; Mon, 16 Mar 2015 12:37:21 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay1.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 97A6BAE01; Mon, 16 Mar 2015 12:37:11 +0000 (UTC) From: Hannes Reinecke To: Christophe Varoqui Date: Mon, 16 Mar 2015 13:37:04 +0100 Message-Id: <1426509425-15978-78-git-send-email-hare@suse.de> In-Reply-To: <1426509425-15978-1-git-send-email-hare@suse.de> References: <1426509425-15978-1-git-send-email-hare@suse.de> X-RedHat-Spam-Score: -7.309 (BAYES_00, DCC_REPUT_00_12, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, URIBL_BLOCKED) 195.135.220.15 cantor2.suse.de 195.135.220.15 cantor2.suse.de X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 X-Scanned-By: MIMEDefang 2.75 on 10.5.110.26 X-loop: dm-devel@redhat.com Cc: dm-devel@redhat.com Subject: [dm-devel] [PATCH 77/78] multipathd: asynchronous configuration X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk Reply-To: device-mapper development List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP For initial configuration multipathd waits it has synchronized with the existing setup. On larger systems this takes up quite some time (I've measured 80 seconds on a system with 1024 paths) causing systemd to stall and the system to fail booting. This patch makes the initial configuration asynchronous, and using the same codepath as the existing 'reconfigure' CLI command. Signed-off-by: Hannes Reinecke --- multipathd/cli_handlers.c | 5 ++- multipathd/main.c | 79 ++++++++++++++++++++++++++++------------------- multipathd/main.h | 1 + 3 files changed, 51 insertions(+), 34 deletions(-) diff --git a/multipathd/cli_handlers.c b/multipathd/cli_handlers.c index dc96c45..e51537e 100644 --- a/multipathd/cli_handlers.c +++ b/multipathd/cli_handlers.c @@ -830,11 +830,10 @@ cli_switch_group(void * v, char ** reply, int * len, void * data) int cli_reconfigure(void * v, char ** reply, int * len, void * data) { - struct vectors * vecs = (struct vectors *)data; - condlog(2, "reconfigure (operator)"); - return reconfigure(vecs); + post_config_state(DAEMON_CONFIGURE); + return 0; } int diff --git a/multipathd/main.c b/multipathd/main.c index 6c98686..d9f2435 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -93,10 +93,11 @@ struct mpath_event_param unsigned int mpath_mx_alloc_len; int logsink; -enum daemon_status running_state; +enum daemon_status running_state = DAEMON_INIT; pid_t daemon_pid; +pthread_mutex_t config_lock = PTHREAD_MUTEX_INITIALIZER; +pthread_cond_t config_cond = PTHREAD_COND_INITIALIZER; -static sem_t exit_sem; /* * global copy of vecs for use in sig handlers */ @@ -104,6 +105,21 @@ struct vectors * gvecs; struct udev * udev; +static void config_cleanup(void *arg) +{ + pthread_mutex_unlock(&config_lock); +} + +void post_config_state(enum daemon_status state) +{ + pthread_mutex_lock(&config_lock); + if (state != running_state) { + running_state = state; + pthread_cond_broadcast(&config_cond); + } + pthread_mutex_unlock(&config_lock); +} + static int need_switch_pathgroup (struct multipath * mpp, int refresh) { @@ -860,6 +876,15 @@ uev_trigger (struct uevent * uev, void * trigger_data) if (uev_discard(uev->devpath)) return 0; + pthread_cleanup_push(config_cleanup, NULL); + pthread_mutex_lock(&config_lock); + if (running_state != DAEMON_RUNNING) + pthread_cond_wait(&config_cond, &config_lock); + pthread_cleanup_pop(1); + + if (running_state == DAEMON_SHUTDOWN) + return 0; + /* * device map event * Add events are ignored here as the tables @@ -948,7 +973,7 @@ uxlsnrloop (void * ap) set_handler_callback(ADD+MAP, cli_add_map); set_handler_callback(DEL+MAP, cli_del_map); set_handler_callback(SWITCH+MAP+GROUP, cli_switch_group); - set_handler_callback(RECONFIGURE, cli_reconfigure); + set_unlocked_handler_callback(RECONFIGURE, cli_reconfigure); set_handler_callback(SUSPEND+MAP, cli_suspend); set_handler_callback(RESUME+MAP, cli_resume); set_handler_callback(RESIZE+MAP, cli_resize); @@ -977,7 +1002,7 @@ uxlsnrloop (void * ap) void exit_daemon (void) { - sem_post(&exit_sem); + post_config_state(DAEMON_SHUTDOWN); } const char * @@ -1561,8 +1586,6 @@ reconfigure (struct vectors * vecs) struct config * old = conf; int retval = 1; - running_state = DAEMON_CONFIGURE; - /* * free old map and path vectors ... they use old conf state */ @@ -1587,7 +1610,7 @@ reconfigure (struct vectors * vecs) retval = 0; } - running_state = DAEMON_RUNNING; + post_config_state(DAEMON_RUNNING); return retval; } @@ -1643,12 +1666,7 @@ handle_signals(void) { if (reconfig_sig && running_state == DAEMON_RUNNING) { condlog(2, "reconfigure (signal)"); - pthread_cleanup_push(cleanup_lock, - &gvecs->lock); - lock(gvecs->lock); - pthread_testcancel(); - reconfigure(gvecs); - lock_cleanup_pop(gvecs->lock); + post_config_state(DAEMON_CONFIGURE); } if (log_reset_sig) { condlog(2, "reset log (signal)"); @@ -1781,7 +1799,6 @@ child (void * param) char *envp; mlockall(MCL_CURRENT | MCL_FUTURE); - sem_init(&exit_sem, 0, 0); signal_init(); udev = udev_new(); @@ -1796,7 +1813,7 @@ child (void * param) pthread_attr_destroy(&log_attr); } - running_state = DAEMON_START; + post_config_state(DAEMON_START); #ifdef USE_SYSTEMD sd_notify(0, "STATUS=startup"); @@ -1891,15 +1908,7 @@ child (void * param) #ifdef USE_SYSTEMD sd_notify(0, "STATUS=configure"); #endif - running_state = DAEMON_CONFIGURE; - - lock(vecs->lock); - if (configure(vecs, 1)) { - unlock(vecs->lock); - condlog(0, "failure during configuration"); - goto failed; - } - unlock(vecs->lock); + post_config_state(DAEMON_CONFIGURE); /* * start threads @@ -1918,20 +1927,29 @@ child (void * param) pid_rc = pidfile_create(DEFAULT_PIDFILE, daemon_pid); /* Ignore errors, we can live without */ - running_state = DAEMON_RUNNING; #ifdef USE_SYSTEMD sd_notify(0, "READY=1\nSTATUS=running"); #endif - /* - * exit path - */ - while(sem_wait(&exit_sem) != 0); /* Do nothing */ + while (running_state != DAEMON_SHUTDOWN) { + pthread_cleanup_push(config_cleanup, NULL); + pthread_mutex_lock(&config_lock); + if (running_state == DAEMON_RUNNING) { + pthread_cond_wait(&config_cond, &config_lock); + } + pthread_cleanup_pop(1); + if (running_state == DAEMON_CONFIGURE) { + pthread_cleanup_push(cleanup_lock, &vecs->lock); + lock(vecs->lock); + pthread_testcancel(); + reconfigure(vecs); + lock_cleanup_pop(vecs->lock); + } + } #ifdef USE_SYSTEMD sd_notify(0, "STATUS=shutdown"); #endif - running_state = DAEMON_SHUTDOWN; lock(vecs->lock); if (conf->queue_without_daemon == QUE_NO_DAEMON_OFF) vector_foreach_slot(vecs->mpvec, mpp, i) @@ -2066,7 +2084,6 @@ main (int argc, char *argv[]) int foreground = 0; logsink = 1; - running_state = DAEMON_INIT; dm_init(); if (getuid() != 0) { diff --git a/multipathd/main.h b/multipathd/main.h index 1813633..6edf4ab 100644 --- a/multipathd/main.h +++ b/multipathd/main.h @@ -25,6 +25,7 @@ int ev_remove_path (struct path *, struct vectors *); int ev_add_map (char *, char *, struct vectors *); int ev_remove_map (char *, char *, int, struct vectors *); void sync_map_state (struct multipath *); +void post_config_state(enum daemon_status); void * mpath_alloc_prin_response(int prin_sa); int prin_do_scsi_ioctl(char *, int rq_servact, struct prin_resp * resp, int noisy);