From patchwork Tue Jan 16 11:48:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chongyun Wu X-Patchwork-Id: 10166575 X-Patchwork-Delegate: christophe.varoqui@free.fr Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7FA75600CA for ; Tue, 16 Jan 2018 11:50:07 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6930F2847E for ; Tue, 16 Jan 2018 11:50:07 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5677128488; Tue, 16 Jan 2018 11:50:07 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 5D7DD2847E for ; Tue, 16 Jan 2018 11:50:05 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4EED149008; Tue, 16 Jan 2018 11:50:02 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 564B8600D2; Tue, 16 Jan 2018 11:49:58 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 6D2434EBDE; Tue, 16 Jan 2018 11:49:37 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id w0GBnZQo032199 for ; Tue, 16 Jan 2018 06:49:35 -0500 Received: by smtp.corp.redhat.com (Postfix) id 2276E60C9E; Tue, 16 Jan 2018 11:49:35 +0000 (UTC) Delivered-To: dm-devel@redhat.com Received: from mx1.redhat.com (ext-mx02.extmail.prod.ext.phx2.redhat.com [10.5.110.26]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1BD7460C97 for ; Tue, 16 Jan 2018 11:49:32 +0000 (UTC) Received: from h3cmg01-ex.h3c.com (smtp.h3c.com [60.191.123.56]) by mx1.redhat.com (Postfix) with ESMTP id 0369E76540 for ; Tue, 16 Jan 2018 11:49:17 +0000 (UTC) Received: from BJHUB01-EX.srv.huawei-3com.com (unknown [10.63.20.169]) by h3cmg01-ex.h3c.com with smtp id 3179_0749_904dddce_f5b8_493e_8180_5207e160aff2; Tue, 16 Jan 2018 19:48:36 +0800 Received: from H3CMLB14-EX.srv.huawei-3com.com ([fe80::f804:6772:bd71:f07f]) by BJHUB01-EX.srv.huawei-3com.com ([::1]) with mapi id 14.03.0248.002; Tue, 16 Jan 2018 19:48:29 +0800 From: Wuchongyun To: Martin Wilck , "dm-devel@redhat.com" Thread-Topic: [PATCH V3] multipathd: release uxsocket and resource when cancel thread Thread-Index: AdOObleUO6MeTqMFQdShlVxVKHsaDQ== Date: Tue, 16 Jan 2018 11:48:28 +0000 Message-ID: Accept-Language: zh-CN, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.125.136.149] MIME-Version: 1.0 X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 207 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Tue, 16 Jan 2018 11:49:19 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Tue, 16 Jan 2018 11:49:19 +0000 (UTC) for IP:'60.191.123.56' DOMAIN:'smtp.h3c.com' HELO:'h3cmg01-ex.h3c.com' FROM:'wu.chongyun@h3c.com' RCPT:'' X-RedHat-Spam-Score: 0.82 (HELO_DYNAMIC_DHCP, RCVD_IN_DNSWL_NONE, SPF_PASS, T_RP_MATCHES_RCVD) 60.191.123.56 smtp.h3c.com 60.191.123.56 smtp.h3c.com X-Scanned-By: MIMEDefang 2.78 on 10.5.110.26 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MIME-Autoconverted: from base64 to 8bit by lists01.pubmisc.prod.ext.phx2.redhat.com id w0GBnZQo032199 X-loop: dm-devel@redhat.com Cc: Guozhonghua , Changwei Ge Subject: Re: [dm-devel] [PATCH V3] multipathd: release uxsocket and resource when cancel thread X-BeenThere: dm-devel@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: device-mapper development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Tue, 16 Jan 2018 11:50:04 +0000 (UTC) X-Virus-Scanned: ClamAV using ClamSMTP Hi Martin, Sorry to forget that, actually I found that dead_client() will not be interrupt by thread cancle, because after all dead_client() calling point be done then handle_signals() have chance to be called by uxsock_listen() which will call exit_daemon() and send cancel threads signal to all child process include uxlsnr. But your comments is good can make code more safer. Below is the new patch, please have a look, thanks. Issue description: we meet this issue: when multipathd initilaze and call uxsock_listen to create unix domain socket, but return -1 and the errno is 98 and then the uxsock_listen return null. After multipathd startup we can't receive any user's multipathd commands to finish the new multipath creation or any operations any more! We found that uxlsnr thread's cleanup function not close the sockets also not release the clients when cancel thread, the domain socket will be release by the system. In any special environment like the machine's load is very heavy or any situations, the system may not close the old domain socket when we try to create and bind the new domain socket may return errno:98(Address already in use). And also we make some experiments: in uxsock_cleanup if we close the ux_sock first and then immdediately call ux_socket_listen to create new ux_sock and initialization will be OK; if we don't close the ux_sock and call ux_socket_listen will return -1 and errno = 98. So we believe that close uxsocket and release clients when cancel thread can make sure of that new starting multipathd thread can create new uxsocket successfully, also can receive multipathd commands properly. And this path can fix clients' memory leak too. Signed-off-by: Chongyun Wu Reviewed-by: Martin Wilck --- multipathd/uxlsnr.c | 29 ++++++++++++++++++++++++----- 1 file changed, 24 insertions(+), 5 deletions(-) diff --git a/multipathd/uxlsnr.c b/multipathd/uxlsnr.c index 98ac25a..c8065ea 100644 --- a/multipathd/uxlsnr.c +++ b/multipathd/uxlsnr.c @@ -102,14 +102,21 @@ static void new_client(int ux_sock) /* * kill off a dead client */ -static void dead_client(struct client *c) +static void _dead_client(struct client *c) { - pthread_mutex_lock(&client_lock); + int fd = c->fd; list_del_init(&c->node); - pthread_mutex_unlock(&client_lock); - close(c->fd); c->fd = -1; FREE(c); + close(fd); +} + +static void dead_client(struct client *c) +{ + pthread_cleanup_push(cleanup_lock, &client_lock); + pthread_mutex_lock(&client_lock); + _dead_client(c); + pthread_cleanup_pop(1); } void free_polls (void) @@ -139,6 +146,18 @@ void check_timeout(struct timespec start_time, char *inbuf, void uxsock_cleanup(void *arg) { + struct client *client_loop; + struct client *client_tmp; + int ux_sock = (int)arg; + + pthread_mutex_lock(&client_lock); + list_for_each_entry_safe(client_loop, client_tmp, &clients, node) { + _dead_client(client_loop); + } + pthread_mutex_unlock(&client_lock); + + close(ux_sock); + cli_exit(); free_polls(); } @@ -162,7 +181,7 @@ void * uxsock_listen(uxsock_trigger_fn uxsock_trigger, void * trigger_data) return NULL; } - pthread_cleanup_push(uxsock_cleanup, NULL); + pthread_cleanup_push(uxsock_cleanup, (void *)ux_sock); condlog(3, "uxsock: startup listener"); polls = (struct pollfd *)MALLOC((MIN_POLLS + 1) * sizeof(struct pollfd));