From patchwork Mon Apr 2 19:50:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martin Wilck X-Patchwork-Id: 10320269 X-Patchwork-Delegate: christophe.varoqui@free.fr Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id CA76060247 for ; Mon, 2 Apr 2018 19:52:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B923F20856 for ; Mon, 2 Apr 2018 19:52:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AD08725E13; Mon, 2 Apr 2018 19:52:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id F2ABC20856 for ; Mon, 2 Apr 2018 19:52:40 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 50AC649028; Mon, 2 Apr 2018 19:52:40 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2D5A37A238; Mon, 2 Apr 2018 19:52:40 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 008A61805971; Mon, 2 Apr 2018 19:52:39 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w32Jqc4E027607 for ; Mon, 2 Apr 2018 15:52:38 -0400 Received: by smtp.corp.redhat.com (Postfix) id 33E4C12A79; Mon, 2 Apr 2018 19:52:38 +0000 (UTC) Delivered-To: dm-devel@redhat.com Received: from mx1.redhat.com (ext-mx09.extmail.prod.ext.phx2.redhat.com [10.5.110.38]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 2B14660637; Mon, 2 Apr 2018 19:52:38 +0000 (UTC) Received: from prv3-mh.provo.novell.com (victor.provo.novell.com [137.65.250.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DA9104E35B; Mon, 2 Apr 2018 19:52:36 +0000 (UTC) Received: from apollon.suse.de.de (prv-ext-foundry1int.gns.novell.com [137.65.251.240]) by prv3-mh.provo.novell.com with ESMTP (TLS encrypted); Mon, 02 Apr 2018 13:52:24 -0600 From: Martin Wilck To: Christophe Varoqui , Benjamin Marzinski Date: Mon, 2 Apr 2018 21:50:47 +0200 Message-Id: <20180402195051.26854-17-mwilck@suse.com> In-Reply-To: <20180402195051.26854-1-mwilck@suse.com> References: <20180402195051.26854-1-mwilck@suse.com> X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 207 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Mon, 02 Apr 2018 19:52:37 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Mon, 02 Apr 2018 19:52:37 +0000 (UTC) for IP:'137.65.250.26' DOMAIN:'victor.provo.novell.com' HELO:'prv3-mh.provo.novell.com' FROM:'mwilck@suse.com' RCPT:'' X-RedHat-Spam-Score: -2.301 (RCVD_IN_DNSWL_MED, SPF_PASS) 137.65.250.26 victor.provo.novell.com 137.65.250.26 victor.provo.novell.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.38 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-loop: dm-devel@redhat.com Cc: Julian Andres Klode , dm-devel@redhat.com, Martin Wilck Subject: [dm-devel] [PATCH v3 16/20] multipath -u : set FIND_MULTIPATHS_WAIT_UNTIL from /dev/shm X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Mon, 02 Apr 2018 19:52:40 +0000 (UTC) X-Virus-Scanned: ClamAV using ClamSMTP In "find_multipaths smart" mode, use time stamps under /dev/shm/multipath/find_multipaths to track waiting for multipath siblings. When a path is first encountered and is "maybe" multipath, create a file under /dev/shm, set its modification time to the expiry time of the timer, and set the FIND_MULTIPATHS_WAIT_UNTIL variable. On later calls, also set FIND_MULTIPATHS_WAIT_UNTIL to the expiry time (but don't change the time stamp) if it's not expired yet, or 0 if it is expired. Set FIND_MULTIPATHS_WAIT_UNTIL even if enough evidence becomes available to decide if the path needs to be multipathed - this enables the udev rules to detect that this is a device a timer has been started for, and stop it. By using /dev/shm, we share information about "smart" timers between initrd and root file system, and thus only calculate the timeout once. Signed-off-by: Martin Wilck --- libmultipath/file.c | 2 +- libmultipath/file.h | 1 + multipath/main.c | 133 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 135 insertions(+), 1 deletion(-) diff --git a/libmultipath/file.c b/libmultipath/file.c index d5165ec..8727f16 100644 --- a/libmultipath/file.c +++ b/libmultipath/file.c @@ -36,7 +36,7 @@ * See the file COPYING included with this distribution for more details. */ -static int +int ensure_directories_exist(const char *str, mode_t dir_mode) { char *pathname; diff --git a/libmultipath/file.h b/libmultipath/file.h index 70bffa5..29520c7 100644 --- a/libmultipath/file.h +++ b/libmultipath/file.h @@ -6,6 +6,7 @@ #define _FILE_H #define FILE_TIMEOUT 30 +int ensure_directories_exist(const char *str, mode_t dir_mode); int open_file(const char *file, int *can_write, const char *header); #endif /* _FILE_H */ diff --git a/multipath/main.c b/multipath/main.c index 650e2bb..d09f117 100644 --- a/multipath/main.c +++ b/multipath/main.c @@ -29,6 +29,7 @@ #include #include #include +#include #include "checkers.h" #include "prio.h" @@ -60,6 +61,9 @@ #include "uxsock.h" #include "mpath_cmd.h" #include "foreign.h" +#include "propsel.h" +#include "time-util.h" +#include "file.h" int logsink; struct udev *udev; @@ -350,14 +354,143 @@ out: return r; } +enum { + FIND_MULTIPATHS_WAIT_DONE = 0, + FIND_MULTIPATHS_WAITING = 1, + FIND_MULTIPATHS_ERROR = -1, + FIND_MULTIPATHS_NEVER = -2, +}; + +static const char shm_find_mp_dir[] = MULTIPATH_SHM_BASE "find_multipaths"; +static void close_fd(void* arg) +{ + close((long)arg); +} + +/** + * find_multipaths_check_timeout(wwid, tmo) + * Helper for "find_multipaths smart" + * + * @param[in] pp: path to check / record + * @param[in] tmo: configured timeout for this WWID, or value <= 0 for checking + * @param[out] until: timestamp until we must wait, CLOCK_REALTIME, if return + * value is FIND_MULTIPATHS_WAITING + * @returns: FIND_MULTIPATHS_WAIT_DONE, if waiting has finished + * @returns: FIND_MULTIPATHS_ERROR, if internal error occured + * @returns: FIND_MULTIPATHS_NEVER, if tmo is 0 and we didn't wait for this + * device + * @returns: FIND_MULTIPATHS_WAITING, if timeout hasn't expired + */ +static int find_multipaths_check_timeout(const struct path *pp, long tmo, + struct timespec *until) +{ + char path[PATH_MAX]; + struct timespec now, ftimes[2], tdiff; + struct stat st; + long fd; + int r, err, retries = 0; + + clock_gettime(CLOCK_REALTIME, &now); + + if (snprintf(path, sizeof(path), "%s/%s", shm_find_mp_dir, pp->dev_t) + >= sizeof(path)) { + condlog(1, "%s: path name overflow", __func__); + return -1; + } + + if (ensure_directories_exist(path, 0700)) { + condlog(1, "%s: error creating directories", __func__); + return FIND_MULTIPATHS_ERROR; + } + +retry: + fd = open(path, O_RDONLY); + if (fd != -1) { + pthread_cleanup_push(close_fd, (void*)fd); + r = fstat(fd, &st); + if (r != 0) + err = errno; + pthread_cleanup_pop(1); + + } else if (tmo > 0) { + if (errno == ENOENT) + fd = open(path, O_RDWR|O_EXCL|O_CREAT, 0644); + if (fd == -1) { + if (errno == EEXIST && !retries++) + /* We could have raced with another process */ + goto retry; + condlog(1, "%s: error opening %s: %s", + __func__, path, strerror(errno)); + return FIND_MULTIPATHS_ERROR; + }; + + pthread_cleanup_push(close_fd, (void*)fd); + /* + * We just created the file. Set st_mtim to our desired + * expiry time. + */ + ftimes[0].tv_sec = 0; + ftimes[0].tv_nsec = UTIME_OMIT; + ftimes[1].tv_sec = now.tv_sec + tmo; + ftimes[1].tv_nsec = now.tv_nsec; + if (futimens(fd, ftimes) != 0) { + condlog(1, "%s: error in futimens(%s): %s", __func__, + path, strerror(errno)); + } + r = fstat(fd, &st); + if (r != 0) + err = errno; + pthread_cleanup_pop(1); + } else + return FIND_MULTIPATHS_NEVER; + + if (r != 0) { + condlog(1, "%s: error in fstat for %s: %s", __func__, + path, strerror(err)); + return FIND_MULTIPATHS_ERROR; + } + + timespecsub(&st.st_mtim, &now, &tdiff); + + if (tdiff.tv_sec <= 0) + return FIND_MULTIPATHS_WAIT_DONE; + else { + *until = tdiff; + return FIND_MULTIPATHS_WAITING; + } +} + static int print_cmd_valid(int k, const vector pathvec, struct config *conf) { static const int vals[] = { 1, 0, 2 }; + int wait = FIND_MULTIPATHS_NEVER; + struct timespec until; + struct path *pp; if (k < 0 || k >= sizeof(vals)) return 1; + if (k == 2) { + /* + * Caller ensures that pathvec[0] is the path to + * examine. + */ + pp = VECTOR_SLOT(pathvec, 0); + select_find_multipaths_timeout(conf, pp); + wait = find_multipaths_check_timeout( + pp, pp->find_multipaths_timeout, &until); + if (wait != FIND_MULTIPATHS_WAITING) + k = 1; + } else if (pathvec != NULL) { + pp = VECTOR_SLOT(pathvec, 0); + wait = find_multipaths_check_timeout(pp, 0, &until); + } + if (wait == FIND_MULTIPATHS_WAITING) + printf("FIND_MULTIPATHS_WAIT_UNTIL=\"%ld.%06ld\"\n", + until.tv_sec, until.tv_nsec/1000); + else if (wait == FIND_MULTIPATHS_WAIT_DONE) + printf("FIND_MULTIPATHS_WAIT_UNTIL=\"0\"\n"); printf("DM_MULTIPATH_DEVICE_PATH=\"%d\"\n", vals[k]); return k == 1; }