From patchwork Mon Jun 20 08:09:12 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hannes Reinecke X-Patchwork-Id: 9186827 X-Patchwork-Delegate: christophe.varoqui@free.fr Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id BDDF860756 for ; Mon, 20 Jun 2016 08:13:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A89AB24B5B for ; Mon, 20 Jun 2016 08:13:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9D66525227; Mon, 20 Jun 2016 08:13:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 2086C24B5B for ; Mon, 20 Jun 2016 08:13:20 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id u5K89qbj001074; Mon, 20 Jun 2016 04:09:52 -0400 Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id u5K89Yuw031169 for ; Mon, 20 Jun 2016 04:09:34 -0400 Received: from mx1.redhat.com (ext-mx02.extmail.prod.ext.phx2.redhat.com [10.5.110.26]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u5K89Y4i002933 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 20 Jun 2016 04:09:34 -0400 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) (using TLSv1 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2B7D97F08F; Mon, 20 Jun 2016 08:09:33 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay1.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id BCE43ADC3; Mon, 20 Jun 2016 08:09:27 +0000 (UTC) From: Hannes Reinecke To: Christophe Varoqui Date: Mon, 20 Jun 2016 10:09:12 +0200 Message-Id: <1466410153-23896-26-git-send-email-hare@suse.de> In-Reply-To: <1466410153-23896-1-git-send-email-hare@suse.de> References: <1466410153-23896-1-git-send-email-hare@suse.de> X-Greylist: Sender IP whitelisted by DNSRBL, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Mon, 20 Jun 2016 08:09:33 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Mon, 20 Jun 2016 08:09:33 +0000 (UTC) for IP:'195.135.220.15' DOMAIN:'mx2.suse.de' HELO:'mx2.suse.de' FROM:'hare@suse.de' RCPT:'' X-RedHat-Spam-Score: -3.327 (BAYES_50, DCC_REPUT_00_12, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD, SPF_PASS) 195.135.220.15 mx2.suse.de 195.135.220.15 mx2.suse.de X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 X-Scanned-By: MIMEDefang 2.75 on 10.5.110.26 X-loop: dm-devel@redhat.com Cc: Hannes Reinecke , dm-devel@redhat.com Subject: [dm-devel] [PATCH 25/26] multipathd: use userspace RCU to access configuration X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Virus-Scanned: ClamAV using ClamSMTP As the configuration is accessed from various threads at various points in time any configuration change is tricky. To avoid any race conditions this patch encapsulates any configuration accesses via RCU, which will avoid any races during reconfiguration. Signed-off-by: Hannes Reinecke --- libmultipath/config.h | 2 ++ libmultipath/waiter.c | 3 +++ mpathpersist/main.c | 4 ++++ multipath/main.c | 4 ++++ multipathd/Makefile | 2 +- multipathd/main.c | 42 +++++++++++++++++++++++++++++++++--------- 6 files changed, 47 insertions(+), 10 deletions(-) diff --git a/libmultipath/config.h b/libmultipath/config.h index 289403e..92f2be9 100644 --- a/libmultipath/config.h +++ b/libmultipath/config.h @@ -3,6 +3,7 @@ #include #include +#include #define ORIGIN_DEFAULT 0 #define ORIGIN_CONFIG 1 @@ -96,6 +97,7 @@ struct mpentry { }; struct config { + struct rcu_head rcu; int verbosity; int pgpolicy_flag; int pgpolicy; diff --git a/libmultipath/waiter.c b/libmultipath/waiter.c index 219876b..34c0110 100644 --- a/libmultipath/waiter.c +++ b/libmultipath/waiter.c @@ -9,6 +9,7 @@ #include #include #include +#include #include "vector.h" #include "memory.h" @@ -41,6 +42,7 @@ void free_waiter (void *data) if (wp->dmt) dm_task_destroy(wp->dmt); + rcu_unregister_thread(); FREE(wp); } @@ -167,6 +169,7 @@ void *waitevent (void *et) waiter = (struct event_thread *)et; pthread_cleanup_push(free_waiter, et); + rcu_register_thread(); while (1) { r = waiteventloop(waiter); diff --git a/mpathpersist/main.c b/mpathpersist/main.c index e4fb39c..5fb831e 100644 --- a/mpathpersist/main.c +++ b/mpathpersist/main.c @@ -53,6 +53,10 @@ void put_multipath_config(struct config *conf) /* Noop for now */ } +void rcu_register_thread_memb(void) {} + +void rcu_unregister_thread_memb(void) {} + int main (int argc, char * argv[]) { int fd, c, res; diff --git a/multipath/main.c b/multipath/main.c index 2ed3003..719d935 100644 --- a/multipath/main.c +++ b/multipath/main.c @@ -73,6 +73,10 @@ void put_multipath_config(struct config *conf) /* Noop for now */ } +void rcu_register_thread_memb(void) {} + +void rcu_unregister_thread_memb(void) {} + static int filter_pathvec (vector pathvec, char * refwwid) { diff --git a/multipathd/Makefile b/multipathd/Makefile index 9b0210f..ec977f3 100644 --- a/multipathd/Makefile +++ b/multipathd/Makefile @@ -9,7 +9,7 @@ CFLAGS += -I$(multipathdir) -I$(mpathpersistdir) -I$(mpathcmddir) ifdef SYSTEMD CFLAGS += -DUSE_SYSTEMD=$(SYSTEMD) endif -LDFLAGS += -lpthread -ldevmapper -lreadline +LDFLAGS += -lurcu -lpthread -ldevmapper -lreadline ifdef SYSTEMD ifeq ($(shell test $(SYSTEMD) -gt 209 && echo 1), 1) LDFLAGS += -lsystemd diff --git a/multipathd/main.c b/multipathd/main.c index d2b57cf..9682b3e 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -17,6 +17,7 @@ #include #include #include +#include #ifdef USE_SYSTEMD #include #endif @@ -206,12 +207,13 @@ int set_config_state(enum daemon_status state) struct config *get_multipath_config(void) { - return multipath_conf; + rcu_read_lock(); + return rcu_dereference(multipath_conf); } void put_multipath_config(struct config *conf) { - /* Noop for now */ + rcu_read_unlock(); } static int @@ -1124,23 +1126,33 @@ out: return r; } +static void *rcu_unregister(void *param) +{ + rcu_unregister_thread(); + return NULL; +} + static void * ueventloop (void * ap) { struct udev *udev = ap; + pthread_cleanup_push(rcu_unregister, NULL); + rcu_register_thread(); if (uevent_listen(udev)) condlog(0, "error starting uevent listener"); - + pthread_cleanup_pop(1); return NULL; } static void * uevqloop (void * ap) { + pthread_cleanup_push(rcu_unregister, NULL); + rcu_register_thread(); if (uevent_dispatch(&uev_trigger, ap)) condlog(0, "error starting uevent dispatcher"); - + pthread_cleanup_pop(1); return NULL; } static void * @@ -1150,7 +1162,8 @@ uxlsnrloop (void * ap) condlog(1, "Failed to init uxsock listener"); return NULL; } - + pthread_cleanup_push(rcu_unregister, NULL); + rcu_register_thread(); set_handler_callback(LIST+PATHS, cli_list_paths); set_handler_callback(LIST+PATHS+FMT, cli_list_paths_fmt); set_handler_callback(LIST+PATHS+RAW+FMT, cli_list_paths_raw); @@ -1200,7 +1213,7 @@ uxlsnrloop (void * ap) umask(077); uxsock_listen(&uxsock_trigger, ap); - + pthread_cleanup_pop(1); return NULL; } @@ -1713,6 +1726,8 @@ checkerloop (void *ap) struct timeval last_time; struct config *conf; + pthread_cleanup_push(rcu_unregister, NULL); + rcu_register_thread(); mlockall(MCL_CURRENT | MCL_FUTURE); vecs = (struct vectors *)ap; condlog(2, "path checkers start up"); @@ -1844,6 +1859,7 @@ checkerloop (void *ap) } } } + pthread_cleanup_pop(1); return NULL; } @@ -1947,6 +1963,13 @@ need_to_delay_reconfig(struct vectors * vecs) return 0; } +void rcu_free_config(struct rcu_head *head) +{ + struct config *conf = container_of(head, struct config, rcu); + + free_config(conf); +} + int reconfigure (struct vectors * vecs) { @@ -1979,12 +2002,12 @@ reconfigure (struct vectors * vecs) conf->ignore_new_devs = ignore_new_devs; uxsock_timeout = conf->uxsock_timeout; - old = multipath_conf; - multipath_conf = conf; + old = rcu_dereference(multipath_conf); + rcu_assign_pointer(multipath_conf, conf); + call_rcu(&old->rcu, rcu_free_config); configure(vecs, 1); - free_config(old); return 0; } @@ -2177,6 +2200,7 @@ child (void * param) mlockall(MCL_CURRENT | MCL_FUTURE); signal_init(); + rcu_init(); setup_thread_attr(&misc_attr, 64 * 1024, 1); setup_thread_attr(&uevent_attr, DEFAULT_UEVENT_STACKSIZE * 1024, 1);