From patchwork Tue Jun 6 19:33:11 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martin Wilck X-Patchwork-Id: 9769897 X-Patchwork-Delegate: christophe.varoqui@free.fr Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 32BAA60353 for ; Tue, 6 Jun 2017 19:34:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 216382621B for ; Tue, 6 Jun 2017 19:34:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1611426E76; Tue, 6 Jun 2017 19:34:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 2EBFB2621B for ; Tue, 6 Jun 2017 19:34:03 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 93F19811A7; Tue, 6 Jun 2017 19:34:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 93F19811A7 Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=dm-devel-bounces@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 93F19811A7 Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3C3625DD71; Tue, 6 Jun 2017 19:34:00 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 5DCAC1800C8B; Tue, 6 Jun 2017 19:33:57 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id v56JXtAk006426 for ; Tue, 6 Jun 2017 15:33:55 -0400 Received: by smtp.corp.redhat.com (Postfix) id AE19B60F8D; Tue, 6 Jun 2017 19:33:55 +0000 (UTC) Delivered-To: dm-devel@redhat.com Received: from mx1.redhat.com (ext-mx06.extmail.prod.ext.phx2.redhat.com [10.5.110.30]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AAE4D756C2; Tue, 6 Jun 2017 19:33:53 +0000 (UTC) Received: from prv3-mh.provo.novell.com (prv3-mh.provo.novell.com [137.65.250.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 66A0312B2D; Tue, 6 Jun 2017 19:33:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 66A0312B2D Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: ext-mx06.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=mwilck@suse.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 66A0312B2D Received: from apollon.suse.de.de (prv-ext-foundry1int.gns.novell.com [137.65.251.240]) by prv3-mh.provo.novell.com with ESMTP (TLS encrypted); Tue, 06 Jun 2017 13:33:47 -0600 From: Martin Wilck To: Christophe Varoqui , Hannes Reinecke , Benjamin Marzinski Date: Tue, 6 Jun 2017 21:33:11 +0200 Message-Id: <20170606193311.5096-1-mwilck@suse.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 203 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 06 Jun 2017 19:33:52 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 06 Jun 2017 19:33:52 +0000 (UTC) for IP:'137.65.250.26' DOMAIN:'prv3-mh.provo.novell.com' HELO:'prv3-mh.provo.novell.com' FROM:'mwilck@suse.com' RCPT:'' X-RedHat-Spam-Score: -1.501 (BAYES_50, RCVD_IN_DNSWL_MED, SPF_PASS) 137.65.250.26 prv3-mh.provo.novell.com 137.65.250.26 prv3-mh.provo.novell.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.30 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-loop: dm-devel@redhat.com Cc: dm-devel@redhat.com Subject: [dm-devel] [PATCH] libmultipath: lazy device-mapper initialization X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Tue, 06 Jun 2017 19:34:03 +0000 (UTC) X-Virus-Scanned: ClamAV using ClamSMTP The multipath command fails early if libdevmapper and/or the dm_multipath driver are not found. But some multipath operations don't require device mapper, notably the path checking operations called from udev rules during early boot. This patch delays dm initialization until the first device-mapper call, allowing such operations to succeed even if the dm_multipath driver is not loaded yet. This fixes the following problem: during system boot, the dm_multipath driver is usually loaded via "ExecStartPre" from the multipathd service file. But with commit d7188fcd, this service is started only after udev settle and thus the modules are loaded too late for udev rule processing, causing "multipath" invocations from udev rules to fail and paths to be wrongly classified as non-multipath. Fixes: d7188fcd "multipathd: start daemon after udev trigger" Signed-off-by: Martin Wilck --- libmultipath/devmapper.c | 79 ++++++++++++++++++++++++++++++++++-------------- libmultipath/devmapper.h | 4 ++- libmultipath/waiter.c | 2 +- multipath/main.c | 8 ++--- multipathd/main.c | 4 +-- 5 files changed, 63 insertions(+), 34 deletions(-) diff --git a/libmultipath/devmapper.c b/libmultipath/devmapper.c index 69b634bf..4319b381 100644 --- a/libmultipath/devmapper.c +++ b/libmultipath/devmapper.c @@ -21,6 +21,7 @@ #include "memory.h" #include "devmapper.h" #include "sysfs.h" +#include "config.h" #include "log_pthread.h" #include @@ -178,7 +179,7 @@ out: } static int -dm_drv_prereq (void) +dm_drv_prereq (unsigned int *ver) { unsigned int minv[3] = {1, 0, 3}; unsigned int version[3] = {0, 0, 0}; @@ -193,19 +194,51 @@ dm_drv_prereq (void) condlog(3, "DM multipath kernel driver v%u.%u.%u", v[0], v[1], v[2]); - if VERSION_GE(v, minv) + if (VERSION_GE(v, minv)) { + ver[0] = v[0]; + ver[1] = v[1]; + ver[2] = v[2]; return 0; + } condlog(0, "DM multipath kernel driver must be >= v%u.%u.%u", minv[0], minv[1], minv[2]); return 1; } -int dm_prereq(void) +static int dm_prereq(unsigned int *v) { if (dm_lib_prereq()) return 1; - return dm_drv_prereq(); + return dm_drv_prereq(v); +} + +static int libmp_dm_udev_sync = 0; + +void libmp_udev_set_sync_support(int on) +{ + libmp_dm_udev_sync = !!on; +} + +void libmp_dm_init(void) +{ + struct config *conf; + + conf = get_multipath_config(); + dm_init(conf->verbosity); + if (dm_prereq(conf->version)) + exit(1); + put_multipath_config(conf); + dm_udev_set_sync_support(libmp_dm_udev_sync); +} + +struct dm_task* +libmp_dm_task_create(int task) +{ + static pthread_once_t dm_initialized = PTHREAD_ONCE_INIT; + + pthread_once(&dm_initialized, libmp_dm_init); + return dm_task_create(task); } #define do_deferred(x) ((x) == DEFERRED_REMOVE_ON || (x) == DEFERRED_REMOVE_IN_PROGRESS) @@ -219,7 +252,7 @@ dm_simplecmd (int task, const char *name, int no_flush, int need_sync, uint16_t uint32_t cookie = 0; struct dm_task *dmt; - if (!(dmt = dm_task_create (task))) + if (!(dmt = libmp_dm_task_create (task))) return 0; if (!dm_task_set_name (dmt, name)) @@ -274,7 +307,7 @@ dm_addmap (int task, const char *target, struct multipath *mpp, uint32_t cookie = 0; uint16_t udev_flags = DM_UDEV_DISABLE_LIBRARY_FALLBACK | ((skip_kpartx == SKIP_KPARTX_ON)? MPATH_UDEV_NO_KPARTX_FLAG : 0); - if (!(dmt = dm_task_create (task))) + if (!(dmt = libmp_dm_task_create (task))) return 0; if (!dm_task_set_name (dmt, mpp->alias)) @@ -411,7 +444,7 @@ do_get_info(const char *name, struct dm_info *info) int r = -1; struct dm_task *dmt; - if (!(dmt = dm_task_create(DM_DEVICE_INFO))) + if (!(dmt = libmp_dm_task_create(DM_DEVICE_INFO))) return r; if (!dm_task_set_name(dmt, name)) @@ -449,7 +482,7 @@ int dm_get_map(const char *name, unsigned long long *size, char *outparams) char *target_type = NULL; char *params = NULL; - if (!(dmt = dm_task_create(DM_DEVICE_TABLE))) + if (!(dmt = libmp_dm_task_create(DM_DEVICE_TABLE))) return 1; if (!dm_task_set_name(dmt, name)) @@ -485,7 +518,7 @@ dm_get_prefixed_uuid(const char *name, char *uuid) const char *uuidtmp; int r = 1; - dmt = dm_task_create(DM_DEVICE_INFO); + dmt = libmp_dm_task_create(DM_DEVICE_INFO); if (!dmt) return 1; @@ -548,7 +581,7 @@ int dm_get_status(char *name, char *outstatus) char *target_type = NULL; char *status = NULL; - if (!(dmt = dm_task_create(DM_DEVICE_STATUS))) + if (!(dmt = libmp_dm_task_create(DM_DEVICE_STATUS))) return 1; if (!dm_task_set_name(dmt, name)) @@ -591,7 +624,7 @@ int dm_type(const char *name, char *type) char *target_type = NULL; char *params; - if (!(dmt = dm_task_create(DM_DEVICE_TABLE))) + if (!(dmt = libmp_dm_task_create(DM_DEVICE_TABLE))) return 0; if (!dm_task_set_name(dmt, name)) @@ -627,7 +660,7 @@ int dm_is_mpath(const char *name) char *params; const char *uuid; - if (!(dmt = dm_task_create(DM_DEVICE_TABLE))) + if (!(dmt = libmp_dm_task_create(DM_DEVICE_TABLE))) return 0; if (!dm_task_set_name(dmt, name)) @@ -679,7 +712,7 @@ dm_get_opencount (const char * mapname) struct dm_task *dmt; struct dm_info info; - if (!(dmt = dm_task_create(DM_DEVICE_INFO))) + if (!(dmt = libmp_dm_task_create(DM_DEVICE_INFO))) return 0; if (!dm_task_set_name(dmt, mapname)) @@ -837,7 +870,7 @@ int dm_flush_maps (int retries) struct dm_names *names; unsigned next = 0; - if (!(dmt = dm_task_create (DM_DEVICE_LIST))) + if (!(dmt = libmp_dm_task_create (DM_DEVICE_LIST))) return 0; dm_task_no_open_count(dmt); @@ -868,7 +901,7 @@ dm_message(const char * mapname, char * message) int r = 1; struct dm_task *dmt; - if (!(dmt = dm_task_create(DM_DEVICE_TARGET_MSG))) + if (!(dmt = libmp_dm_task_create(DM_DEVICE_TARGET_MSG))) return 1; if (!dm_task_set_name(dmt, mapname)) @@ -970,7 +1003,7 @@ dm_get_maps (vector mp) if (!mp) return 1; - if (!(dmt = dm_task_create(DM_DEVICE_LIST))) + if (!(dmt = libmp_dm_task_create(DM_DEVICE_LIST))) return 1; dm_task_no_open_count(dmt); @@ -1056,7 +1089,7 @@ dm_mapname(int major, int minor) int r; int loop = MAX_WAIT * LOOPS_PER_SEC; - if (!(dmt = dm_task_create(DM_DEVICE_STATUS))) + if (!(dmt = libmp_dm_task_create(DM_DEVICE_STATUS))) return NULL; if (!dm_task_set_major(dmt, major) || @@ -1109,7 +1142,7 @@ do_foreach_partmaps (const char * mapname, int r = 1; char *p; - if (!(dmt = dm_task_create(DM_DEVICE_LIST))) + if (!(dmt = libmp_dm_task_create(DM_DEVICE_LIST))) return 1; dm_task_no_open_count(dmt); @@ -1333,7 +1366,7 @@ dm_rename (const char * old, char * new, char *delim, int skip_kpartx) if (dm_rename_partmaps(old, new, delim)) return r; - if (!(dmt = dm_task_create(DM_DEVICE_RENAME))) + if (!(dmt = libmp_dm_task_create(DM_DEVICE_RENAME))) return r; if (!dm_task_set_name(dmt, old)) @@ -1382,7 +1415,7 @@ int dm_reassign_table(const char *name, char *old, char *new) char *buff; void *next = NULL; - if (!(dmt = dm_task_create(DM_DEVICE_TABLE))) + if (!(dmt = libmp_dm_task_create(DM_DEVICE_TABLE))) return 0; if (!dm_task_set_name(dmt, name)) @@ -1392,7 +1425,7 @@ int dm_reassign_table(const char *name, char *old, char *new) if (!dm_task_run(dmt)) goto out; - if (!(reload_dmt = dm_task_create(DM_DEVICE_RELOAD))) + if (!(reload_dmt = libmp_dm_task_create(DM_DEVICE_RELOAD))) goto out; if (!dm_task_set_name(reload_dmt, name)) goto out_reload; @@ -1456,7 +1489,7 @@ int dm_reassign(const char *mapname) return 1; } - if (!(dmt = dm_task_create(DM_DEVICE_DEPS))) { + if (!(dmt = libmp_dm_task_create(DM_DEVICE_DEPS))) { condlog(3, "%s: couldn't make dm task", mapname); return 0; } @@ -1514,7 +1547,7 @@ int dm_setgeometry(struct multipath *mpp) return 1; } - if (!(dmt = dm_task_create(DM_DEVICE_SET_GEOMETRY))) + if (!(dmt = libmp_dm_task_create(DM_DEVICE_SET_GEOMETRY))) return 0; if (!dm_task_set_name(dmt, mpp->alias)) diff --git a/libmultipath/devmapper.h b/libmultipath/devmapper.h index fa739bcc..99a554b5 100644 --- a/libmultipath/devmapper.h +++ b/libmultipath/devmapper.h @@ -25,7 +25,9 @@ #endif void dm_init(int verbosity); -int dm_prereq (void); +void libmp_dm_init(void); +void libmp_udev_set_sync_support(int on); +struct dm_task *libmp_dm_task_create(int task); int dm_drv_version (unsigned int * version, char * str); int dm_simplecmd_flush (int, const char *, uint16_t); int dm_simplecmd_noflush (int, const char *, uint16_t); diff --git a/libmultipath/waiter.c b/libmultipath/waiter.c index 995ea1ad..cb9708b2 100644 --- a/libmultipath/waiter.c +++ b/libmultipath/waiter.c @@ -76,7 +76,7 @@ static int waiteventloop (struct event_thread *waiter) if (!waiter->event_nr) waiter->event_nr = dm_geteventnr(waiter->mapname); - if (!(waiter->dmt = dm_task_create(DM_DEVICE_WAITEVENT))) { + if (!(waiter->dmt = libmp_dm_task_create(DM_DEVICE_WAITEVENT))) { condlog(0, "%s: devmap event #%i dm_task_create error", waiter->mapname, waiter->event_nr); return 1; diff --git a/multipath/main.c b/multipath/main.c index 4174d43d..dede017e 100644 --- a/multipath/main.c +++ b/multipath/main.c @@ -634,12 +634,6 @@ main (int argc, char *argv[]) exit(1); } - dm_init(conf->verbosity); - if (dm_prereq()) - exit(1); - dm_drv_version(conf->version, TGT_MPATH); - dm_udev_set_sync_support(1); - if (optind < argc) { dev = MALLOC(FILE_NAME_SIZE); @@ -670,6 +664,8 @@ main (int argc, char *argv[]) conf->max_fds, strerror(errno)); } + libmp_udev_set_sync_support(1); + if (init_checkers(conf->multipath_dir)) { condlog(0, "failed to initialize checkers"); goto out; diff --git a/multipathd/main.c b/multipathd/main.c index 81c76cab..4be2c579 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -2408,8 +2408,6 @@ child (void * param) conf->ignore_new_devs = ignore_new_devs; uxsock_timeout = conf->uxsock_timeout; rcu_assign_pointer(multipath_conf, conf); - dm_init(conf->verbosity); - dm_drv_version(conf->version, TGT_MPATH); if (init_checkers(conf->multipath_dir)) { condlog(0, "failed to initialize checkers"); goto failed; @@ -2458,7 +2456,6 @@ child (void * param) setscheduler(); set_oom_adj(); - dm_udev_set_sync_support(0); #ifdef USE_SYSTEMD envp = getenv("WATCHDOG_USEC"); if (envp && sscanf(envp, "%lu", &checkint) == 1) { @@ -2700,6 +2697,7 @@ main (int argc, char *argv[]) pthread_cond_init_mono(&config_cond); udev = udev_new(); + libmp_udev_set_sync_support(0); while ((arg = getopt(argc, argv, ":dsv:k::Bn")) != EOF ) { switch(arg) {