From patchwork Thu Sep 5 19:48:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11133879 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4EF1714B4 for ; Thu, 5 Sep 2019 19:50:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 079C820870 for ; Thu, 5 Sep 2019 19:50:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2403982AbfIETuo (ORCPT ); Thu, 5 Sep 2019 15:50:44 -0400 Received: from mx1.redhat.com ([209.132.183.28]:9778 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732721AbfIETt0 (ORCPT ); Thu, 5 Sep 2019 15:49:26 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 581453082E03; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3BC1360C18; Thu, 5 Sep 2019 19:49:18 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id BD853220CEE; Thu, 5 Sep 2019 15:49:17 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, miklos@szeredi.hu Cc: linux-kernel@vger.kernel.org, virtio-fs@redhat.com, vgoyal@redhat.com, stefanha@redhat.com, dgilbert@redhat.com, mst@redhat.com Subject: [PATCH 01/18] virtiofs: Remove request from processing list before calling end Date: Thu, 5 Sep 2019 15:48:42 -0400 Message-Id: <20190905194859.16219-2-vgoyal@redhat.com> In-Reply-To: <20190905194859.16219-1-vgoyal@redhat.com> References: <20190905194859.16219-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Thu, 05 Sep 2019 19:49:26 +0000 (UTC) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org In error path we are calling fuse_request_end() but we need to clear FR_SENT bit as well as remove request from processing queue. Otherwise fuse_request_end() triggers a warning as well as other issues show up. Signed-off-by: Vivek Goyal Reviewed-by: Stefan Hajnoczi --- fs/fuse/virtio_fs.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index 197e79e536f9..a708ccb65662 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -826,6 +826,10 @@ __releases(fiq->waitq.lock) } req->out.h.error = ret; pr_err("virtio-fs: virtio_fs_enqueue_req() failed %d\n", ret); + spin_lock(&fpq->lock); + clear_bit(FR_SENT, &req->flags); + list_del_init(&req->list); + spin_unlock(&fpq->lock); fuse_request_end(fc, req); return; } From patchwork Thu Sep 5 19:48:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11133883 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D46B6112B for ; Thu, 5 Sep 2019 19:50:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 95B1E206A3 for ; Thu, 5 Sep 2019 19:50:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732238AbfIETt0 (ORCPT ); Thu, 5 Sep 2019 15:49:26 -0400 Received: from mx1.redhat.com ([209.132.183.28]:59138 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732060AbfIETt0 (ORCPT ); Thu, 5 Sep 2019 15:49:26 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id ED7973082DDD; Thu, 5 Sep 2019 19:49:25 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3DAFA5C1D4; Thu, 5 Sep 2019 19:49:18 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id C154F22539A; Thu, 5 Sep 2019 15:49:17 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, miklos@szeredi.hu Cc: linux-kernel@vger.kernel.org, virtio-fs@redhat.com, vgoyal@redhat.com, stefanha@redhat.com, dgilbert@redhat.com, mst@redhat.com Subject: [PATCH 02/18] virtiofs: Check whether hiprio queue is connected at submission time Date: Thu, 5 Sep 2019 15:48:43 -0400 Message-Id: <20190905194859.16219-3-vgoyal@redhat.com> In-Reply-To: <20190905194859.16219-1-vgoyal@redhat.com> References: <20190905194859.16219-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Thu, 05 Sep 2019 19:49:26 +0000 (UTC) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org For hiprio queue (forget requests), we are keeping a state in queue whether queue is connected or not. If queue is not connected, do not try to submit request and return error instead. As of now, we are checking for this state only in path where worker tries to submit forget after first attempt failed. Check this state even in the path when request is being submitted first time. Signed-off-by: Vivek Goyal Reviewed-by: Stefan Hajnoczi --- fs/fuse/virtio_fs.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index a708ccb65662..e9497b565dd8 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -577,9 +577,16 @@ __releases(fiq->waitq.lock) sg_init_one(&sg, forget, sizeof(*forget)); /* Enqueue the request */ + spin_lock(&fsvq->lock); + + if (!fsvq->connected) { + kfree(forget); + spin_unlock(&fsvq->lock); + goto out; + } + vq = fsvq->vq; dev_dbg(&vq->vdev->dev, "%s\n", __func__); - spin_lock(&fsvq->lock); ret = virtqueue_add_sgs(vq, sgs, 1, 0, forget, GFP_ATOMIC); if (ret < 0) { From patchwork Thu Sep 5 19:48:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11133867 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DF14B112B for ; Thu, 5 Sep 2019 19:50:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9FB4A20825 for ; Thu, 5 Sep 2019 19:50:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388314AbfIETt1 (ORCPT ); Thu, 5 Sep 2019 15:49:27 -0400 Received: from mx1.redhat.com ([209.132.183.28]:45520 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732717AbfIETt0 (ORCPT ); Thu, 5 Sep 2019 15:49:26 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 554507EB88; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id 400FF5D6A3; Thu, 5 Sep 2019 19:49:18 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id C87AA22539B; Thu, 5 Sep 2019 15:49:17 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, miklos@szeredi.hu Cc: linux-kernel@vger.kernel.org, virtio-fs@redhat.com, vgoyal@redhat.com, stefanha@redhat.com, dgilbert@redhat.com, mst@redhat.com Subject: [PATCH 03/18] virtiofs: Pass fsvq instead of vq as parameter to virtio_fs_enqueue_req Date: Thu, 5 Sep 2019 15:48:44 -0400 Message-Id: <20190905194859.16219-4-vgoyal@redhat.com> In-Reply-To: <20190905194859.16219-1-vgoyal@redhat.com> References: <20190905194859.16219-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.71]); Thu, 05 Sep 2019 19:49:26 +0000 (UTC) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Pass fsvq instead of vq as parameter to virtio_fs_enqueue_req(). We will retrieve vq from fsvq under spin lock. Later in the patch series we will retrieve vq only if fsvq is still connected other vq might have been cleaned up by device ->remove code and we will return error. Signed-off-by: Vivek Goyal Reviewed-by: Stefan Hajnoczi --- fs/fuse/virtio_fs.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index e9497b565dd8..9d30530e3ca9 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -698,14 +698,15 @@ static unsigned int sg_init_fuse_args(struct scatterlist *sg, } /* Add a request to a virtqueue and kick the device */ -static int virtio_fs_enqueue_req(struct virtqueue *vq, struct fuse_req *req) +static int virtio_fs_enqueue_req(struct virtio_fs_vq *fsvq, + struct fuse_req *req) { /* requests need at least 4 elements */ struct scatterlist *stack_sgs[6]; struct scatterlist stack_sg[ARRAY_SIZE(stack_sgs)]; struct scatterlist **sgs = stack_sgs; struct scatterlist *sg = stack_sg; - struct virtio_fs_vq *fsvq; + struct virtqueue *vq; unsigned int argbuf_used = 0; unsigned int out_sgs = 0; unsigned int in_sgs = 0; @@ -752,9 +753,9 @@ static int virtio_fs_enqueue_req(struct virtqueue *vq, struct fuse_req *req) for (i = 0; i < total_sgs; i++) sgs[i] = &sg[i]; - fsvq = vq_to_fsvq(vq); spin_lock(&fsvq->lock); + vq = fsvq->vq; ret = virtqueue_add_sgs(vq, sgs, out_sgs, in_sgs, req, GFP_ATOMIC); if (ret < 0) { /* TODO handle full virtqueue */ @@ -824,7 +825,7 @@ __releases(fiq->waitq.lock) /* TODO check for FR_INTERRUPTED? */ retry: - ret = virtio_fs_enqueue_req(fs->vqs[queue_id].vq, req); + ret = virtio_fs_enqueue_req(&fs->vqs[queue_id], req); if (ret < 0) { if (ret == -ENOMEM || ret == -ENOSPC) { /* Virtqueue full. Retry submission */ From patchwork Thu Sep 5 19:48:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11133887 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 628EB14B4 for ; Thu, 5 Sep 2019 19:51:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 1A366206A3 for ; Thu, 5 Sep 2019 19:51:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391526AbfIETuy (ORCPT ); Thu, 5 Sep 2019 15:50:54 -0400 Received: from mx1.redhat.com ([209.132.183.28]:60344 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732510AbfIETt0 (ORCPT ); Thu, 5 Sep 2019 15:49:26 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id F071E3175295; Thu, 5 Sep 2019 19:49:25 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id 44F1B100194E; Thu, 5 Sep 2019 19:49:18 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id D06FE22539C; Thu, 5 Sep 2019 15:49:17 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, miklos@szeredi.hu Cc: linux-kernel@vger.kernel.org, virtio-fs@redhat.com, vgoyal@redhat.com, stefanha@redhat.com, dgilbert@redhat.com, mst@redhat.com Subject: [PATCH 04/18] virtiofs: Check connected state for VQ_REQUEST queue as well Date: Thu, 5 Sep 2019 15:48:45 -0400 Message-Id: <20190905194859.16219-5-vgoyal@redhat.com> In-Reply-To: <20190905194859.16219-1-vgoyal@redhat.com> References: <20190905194859.16219-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.49]); Thu, 05 Sep 2019 19:49:26 +0000 (UTC) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Right now we are checking ->connected state only for VQ_HIPRIO. Now we want to make use of this method for all queues. So check it for VQ_REQUEST as well. This will be helpful if device has been removed and virtqueue is gone. In that case ->connected will be false and request can't be submitted anymore and user space will see error -ENOTCONN. Signed-off-by: Vivek Goyal Reviewed-by: Stefan Hajnoczi --- fs/fuse/virtio_fs.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index 9d30530e3ca9..c46dd4d284d6 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -755,6 +755,12 @@ static int virtio_fs_enqueue_req(struct virtio_fs_vq *fsvq, spin_lock(&fsvq->lock); + if (!fsvq->connected) { + spin_unlock(&fsvq->lock); + ret = -ENOTCONN; + goto out; + } + vq = fsvq->vq; ret = virtqueue_add_sgs(vq, sgs, out_sgs, in_sgs, req, GFP_ATOMIC); if (ret < 0) { From patchwork Thu Sep 5 19:48:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11133857 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CA220112B for ; Thu, 5 Sep 2019 19:49:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8CB792082E for ; Thu, 5 Sep 2019 19:49:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390311AbfIETtn (ORCPT ); Thu, 5 Sep 2019 15:49:43 -0400 Received: from mx1.redhat.com ([209.132.183.28]:60452 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390162AbfIETte (ORCPT ); Thu, 5 Sep 2019 15:49:34 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4DB32308FBAC; Thu, 5 Sep 2019 19:49:34 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id 66C2419C77; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id D781822539D; Thu, 5 Sep 2019 15:49:17 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, miklos@szeredi.hu Cc: linux-kernel@vger.kernel.org, virtio-fs@redhat.com, vgoyal@redhat.com, stefanha@redhat.com, dgilbert@redhat.com, mst@redhat.com Subject: [PATCH 05/18] Maintain count of in flight requests for VQ_REQUEST queue Date: Thu, 5 Sep 2019 15:48:46 -0400 Message-Id: <20190905194859.16219-6-vgoyal@redhat.com> In-Reply-To: <20190905194859.16219-1-vgoyal@redhat.com> References: <20190905194859.16219-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Thu, 05 Sep 2019 19:49:34 +0000 (UTC) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org As of now we maintain this count only for VQ_HIPRIO. Maintain it for VQ_REQUEST as well so that later it can be used to drain VQ_REQUEST queue. Signed-off-by: Vivek Goyal Reviewed-by: Stefan Hajnoczi --- fs/fuse/virtio_fs.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index c46dd4d284d6..5df97dfee37d 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -360,6 +360,9 @@ static void virtio_fs_requests_done_work(struct work_struct *work) spin_unlock(&fpq->lock); fuse_request_end(fc, req); + spin_lock(&fsvq->lock); + fsvq->in_flight--; + spin_unlock(&fsvq->lock); } } @@ -769,6 +772,7 @@ static int virtio_fs_enqueue_req(struct virtio_fs_vq *fsvq, goto out; } + fsvq->in_flight++; notify = virtqueue_kick_prepare(vq); spin_unlock(&fsvq->lock); From patchwork Thu Sep 5 19:48:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11133863 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0F37D112B for ; Thu, 5 Sep 2019 19:50:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BB99620825 for ; Thu, 5 Sep 2019 19:50:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388524AbfIETt1 (ORCPT ); Thu, 5 Sep 2019 15:49:27 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37434 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731696AbfIETt0 (ORCPT ); Thu, 5 Sep 2019 15:49:26 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 89A8210C6973; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id 666B960C18; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id DDF2E22539E; Thu, 5 Sep 2019 15:49:17 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, miklos@szeredi.hu Cc: linux-kernel@vger.kernel.org, virtio-fs@redhat.com, vgoyal@redhat.com, stefanha@redhat.com, dgilbert@redhat.com, mst@redhat.com Subject: [PATCH 06/18] virtiofs: ->remove should not clean virtiofs fuse devices Date: Thu, 5 Sep 2019 15:48:47 -0400 Message-Id: <20190905194859.16219-7-vgoyal@redhat.com> In-Reply-To: <20190905194859.16219-1-vgoyal@redhat.com> References: <20190905194859.16219-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.65]); Thu, 05 Sep 2019 19:49:26 +0000 (UTC) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org We maintain a fuse device per virt queue. This fuse devices are allocated and installed during mount time and should be cleaned up when super block is going away. Device removal should not clean it. Device removal should stop queues and virtuques can go away. Signed-off-by: Vivek Goyal Reviewed-by: Stefan Hajnoczi --- fs/fuse/virtio_fs.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index 5df97dfee37d..f68a25ca9e9d 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -497,8 +497,6 @@ static void virtio_fs_remove(struct virtio_device *vdev) { struct virtio_fs *fs = vdev->priv; - virtio_fs_free_devs(fs); - vdev->config->reset(vdev); virtio_fs_cleanup_vqs(vdev, fs); From patchwork Thu Sep 5 19:48:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11133849 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ED6D214B4 for ; Thu, 5 Sep 2019 19:49:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A50ED20825 for ; Thu, 5 Sep 2019 19:49:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388632AbfIETt2 (ORCPT ); Thu, 5 Sep 2019 15:49:28 -0400 Received: from mx1.redhat.com ([209.132.183.28]:55684 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732060AbfIETt0 (ORCPT ); Thu, 5 Sep 2019 15:49:26 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8DE2F18C4266; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id 668FF600F8; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id E279922539F; Thu, 5 Sep 2019 15:49:17 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, miklos@szeredi.hu Cc: linux-kernel@vger.kernel.org, virtio-fs@redhat.com, vgoyal@redhat.com, stefanha@redhat.com, dgilbert@redhat.com, mst@redhat.com Subject: [PATCH 07/18] virtiofs: Stop virtiofs queues when device is being removed Date: Thu, 5 Sep 2019 15:48:48 -0400 Message-Id: <20190905194859.16219-8-vgoyal@redhat.com> In-Reply-To: <20190905194859.16219-1-vgoyal@redhat.com> References: <20190905194859.16219-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.62]); Thu, 05 Sep 2019 19:49:26 +0000 (UTC) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Stop all the virt queues when device is going away. This will ensure that no new requests are submitted to virtqueue and and request will end with error -ENOTCONN. Signed-off-by: Vivek Goyal Reviewed-by: Stefan Hajnoczi --- fs/fuse/virtio_fs.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index f68a25ca9e9d..90e7b2f345e5 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -493,10 +493,24 @@ static int virtio_fs_probe(struct virtio_device *vdev) return ret; } +static void virtio_fs_stop_all_queues(struct virtio_fs *fs) +{ + struct virtio_fs_vq *fsvq; + int i; + + for (i = 0; i < fs->nvqs; i++) { + fsvq = &fs->vqs[i]; + spin_lock(&fsvq->lock); + fsvq->connected = false; + spin_unlock(&fsvq->lock); + } +} + static void virtio_fs_remove(struct virtio_device *vdev) { struct virtio_fs *fs = vdev->priv; + virtio_fs_stop_all_queues(fs); vdev->config->reset(vdev); virtio_fs_cleanup_vqs(vdev, fs); From patchwork Thu Sep 5 19:48:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11133869 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4BC38112B for ; Thu, 5 Sep 2019 19:50:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0D979206BB for ; Thu, 5 Sep 2019 19:50:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391506AbfIETuS (ORCPT ); Thu, 5 Sep 2019 15:50:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]:59162 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388034AbfIETt0 (ORCPT ); Thu, 5 Sep 2019 15:49:26 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8F37C10A8122; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6861160166; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id E940B2253A0; Thu, 5 Sep 2019 15:49:17 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, miklos@szeredi.hu Cc: linux-kernel@vger.kernel.org, virtio-fs@redhat.com, vgoyal@redhat.com, stefanha@redhat.com, dgilbert@redhat.com, mst@redhat.com Subject: [PATCH 08/18] virtiofs: Drain all pending requests during ->remove time Date: Thu, 5 Sep 2019 15:48:49 -0400 Message-Id: <20190905194859.16219-9-vgoyal@redhat.com> In-Reply-To: <20190905194859.16219-1-vgoyal@redhat.com> References: <20190905194859.16219-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.64]); Thu, 05 Sep 2019 19:49:26 +0000 (UTC) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org When device is going away, drain all pending requests. Signed-off-by: Vivek Goyal --- fs/fuse/virtio_fs.c | 83 ++++++++++++++++++++++++++++----------------- 1 file changed, 51 insertions(+), 32 deletions(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index 90e7b2f345e5..d5730a50b303 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -63,6 +63,55 @@ static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) return &vq_to_fsvq(vq)->fud->pq; } +static void virtio_fs_drain_queue(struct virtio_fs_vq *fsvq) +{ + WARN_ON(fsvq->in_flight < 0); + + /* Wait for in flight requests to finish.*/ + while (1) { + spin_lock(&fsvq->lock); + if (!fsvq->in_flight) { + spin_unlock(&fsvq->lock); + break; + } + spin_unlock(&fsvq->lock); + usleep_range(1000, 2000); + } + + flush_work(&fsvq->done_work); + flush_delayed_work(&fsvq->dispatch_work); +} + +static inline void drain_hiprio_queued_reqs(struct virtio_fs_vq *fsvq) +{ + struct virtio_fs_forget *forget; + + spin_lock(&fsvq->lock); + while (1) { + forget = list_first_entry_or_null(&fsvq->queued_reqs, + struct virtio_fs_forget, list); + if (!forget) + break; + list_del(&forget->list); + kfree(forget); + } + spin_unlock(&fsvq->lock); +} + +static void virtio_fs_drain_all_queues(struct virtio_fs *fs) +{ + struct virtio_fs_vq *fsvq; + int i; + + for (i = 0; i < fs->nvqs; i++) { + fsvq = &fs->vqs[i]; + if (i == VQ_HIPRIO) + drain_hiprio_queued_reqs(fsvq); + + virtio_fs_drain_queue(fsvq); + } +} + /* Add a new instance to the list or return -EEXIST if tag name exists*/ static int virtio_fs_add_instance(struct virtio_fs *fs) { @@ -511,6 +560,7 @@ static void virtio_fs_remove(struct virtio_device *vdev) struct virtio_fs *fs = vdev->priv; virtio_fs_stop_all_queues(fs); + virtio_fs_drain_all_queues(fs); vdev->config->reset(vdev); virtio_fs_cleanup_vqs(vdev, fs); @@ -865,37 +915,6 @@ __releases(fiq->waitq.lock) } } -static void virtio_fs_flush_hiprio_queue(struct virtio_fs_vq *fsvq) -{ - struct virtio_fs_forget *forget; - - WARN_ON(fsvq->in_flight < 0); - - /* Go through pending forget requests and free them */ - spin_lock(&fsvq->lock); - while (1) { - forget = list_first_entry_or_null(&fsvq->queued_reqs, - struct virtio_fs_forget, list); - if (!forget) - break; - list_del(&forget->list); - kfree(forget); - } - - spin_unlock(&fsvq->lock); - - /* Wait for in flight requests to finish.*/ - while (1) { - spin_lock(&fsvq->lock); - if (!fsvq->in_flight) { - spin_unlock(&fsvq->lock); - break; - } - spin_unlock(&fsvq->lock); - usleep_range(1000, 2000); - } -} - const static struct fuse_iqueue_ops virtio_fs_fiq_ops = { .wake_forget_and_unlock = virtio_fs_wake_forget_and_unlock, .wake_interrupt_and_unlock = virtio_fs_wake_interrupt_and_unlock, @@ -988,7 +1007,7 @@ static void virtio_kill_sb(struct super_block *sb) spin_lock(&fsvq->lock); fsvq->connected = false; spin_unlock(&fsvq->lock); - virtio_fs_flush_hiprio_queue(fsvq); + virtio_fs_drain_all_queues(vfs); fuse_kill_sb_anon(sb); virtio_fs_free_devs(vfs); From patchwork Thu Sep 5 19:48:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11133853 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B4E3B112B for ; Thu, 5 Sep 2019 19:49:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 76EBA20825 for ; Thu, 5 Sep 2019 19:49:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390304AbfIETtn (ORCPT ); Thu, 5 Sep 2019 15:49:43 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38692 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390132AbfIETte (ORCPT ); Thu, 5 Sep 2019 15:49:34 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 299CC7E423; Thu, 5 Sep 2019 19:49:34 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id 670905D9CA; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id EFC332253A1; Thu, 5 Sep 2019 15:49:17 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, miklos@szeredi.hu Cc: linux-kernel@vger.kernel.org, virtio-fs@redhat.com, vgoyal@redhat.com, stefanha@redhat.com, dgilbert@redhat.com, mst@redhat.com Subject: [PATCH 09/18] virtiofs: Add an helper to start all the queues Date: Thu, 5 Sep 2019 15:48:50 -0400 Message-Id: <20190905194859.16219-10-vgoyal@redhat.com> In-Reply-To: <20190905194859.16219-1-vgoyal@redhat.com> References: <20190905194859.16219-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Thu, 05 Sep 2019 19:49:34 +0000 (UTC) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This just marks are the queues are connected and ready to accept the request. Signed-off-by: Vivek Goyal Reviewed-by: Stefan Hajnoczi --- fs/fuse/virtio_fs.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index d5730a50b303..f2936daca39c 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -112,6 +112,19 @@ static void virtio_fs_drain_all_queues(struct virtio_fs *fs) } } +static void virtio_fs_start_all_queues(struct virtio_fs *fs) +{ + struct virtio_fs_vq *fsvq; + int i; + + for (i = 0; i < fs->nvqs; i++) { + fsvq = &fs->vqs[i]; + spin_lock(&fsvq->lock); + fsvq->connected = true; + spin_unlock(&fsvq->lock); + } +} + /* Add a new instance to the list or return -EEXIST if tag name exists*/ static int virtio_fs_add_instance(struct virtio_fs *fs) { @@ -483,10 +496,10 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev, if (ret < 0) goto out; - for (i = 0; i < fs->nvqs; i++) { + for (i = 0; i < fs->nvqs; i++) fs->vqs[i].vq = vqs[i]; - fs->vqs[i].connected = true; - } + + virtio_fs_start_all_queues(fs); out: kfree(names); kfree(callbacks); From patchwork Thu Sep 5 19:48:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11133855 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2A8D314ED for ; Thu, 5 Sep 2019 19:49:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id E14BD20825 for ; Thu, 5 Sep 2019 19:49:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390321AbfIETtn (ORCPT ); Thu, 5 Sep 2019 15:49:43 -0400 Received: from mx1.redhat.com ([209.132.183.28]:60875 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390115AbfIETtd (ORCPT ); Thu, 5 Sep 2019 15:49:33 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7683930089A1; Thu, 5 Sep 2019 19:49:33 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id 675FA6060D; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id 022522253A2; Thu, 5 Sep 2019 15:49:18 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, miklos@szeredi.hu Cc: linux-kernel@vger.kernel.org, virtio-fs@redhat.com, vgoyal@redhat.com, stefanha@redhat.com, dgilbert@redhat.com, mst@redhat.com Subject: [PATCH 10/18] virtiofs: Do not use device managed mem for virtio_fs and virtio_fs_vq Date: Thu, 5 Sep 2019 15:48:51 -0400 Message-Id: <20190905194859.16219-11-vgoyal@redhat.com> In-Reply-To: <20190905194859.16219-1-vgoyal@redhat.com> References: <20190905194859.16219-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]); Thu, 05 Sep 2019 19:49:33 +0000 (UTC) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org These data structures should go away when virtio_fs object is going away. When deivce is going away, we need to just make sure virtqueues can go away and after that none of the code accesses vq and all the requests get error. So allocate memory for virtio_fs and virtio_fs_vq normally and free it at right time. This patch still frees up memory during device remove time. A later patch will make virtio_fs object reference counted and this memory will be freed when last reference to object is dropped. Signed-off-by: Vivek Goyal Reviewed-by: Stefan Hajnoczi --- fs/fuse/virtio_fs.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index f2936daca39c..1ea0f889e804 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -446,7 +446,7 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev, vq_callback_t **callbacks; const char **names; unsigned int i; - int ret; + int ret = 0; virtio_cread(vdev, struct virtio_fs_config, num_queues, &fs->num_queues); @@ -454,9 +454,7 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev, return -EINVAL; fs->nvqs = 1 + fs->num_queues; - - fs->vqs = devm_kcalloc(&vdev->dev, fs->nvqs, - sizeof(fs->vqs[VQ_HIPRIO]), GFP_KERNEL); + fs->vqs = kcalloc(fs->nvqs, sizeof(fs->vqs[VQ_HIPRIO]), GFP_KERNEL); if (!fs->vqs) return -ENOMEM; @@ -504,6 +502,8 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev, kfree(names); kfree(callbacks); kfree(vqs); + if (ret) + kfree(fs->vqs); return ret; } @@ -519,7 +519,7 @@ static int virtio_fs_probe(struct virtio_device *vdev) struct virtio_fs *fs; int ret; - fs = devm_kzalloc(&vdev->dev, sizeof(*fs), GFP_KERNEL); + fs = kzalloc(sizeof(*fs), GFP_KERNEL); if (!fs) return -ENOMEM; vdev->priv = fs; @@ -552,6 +552,7 @@ static int virtio_fs_probe(struct virtio_device *vdev) out: vdev->priv = NULL; + kfree(fs); return ret; } @@ -582,6 +583,8 @@ static void virtio_fs_remove(struct virtio_device *vdev) mutex_unlock(&virtio_fs_mutex); vdev->priv = NULL; + kfree(fs->vqs); + kfree(fs); } #ifdef CONFIG_PM_SLEEP From patchwork Thu Sep 5 19:48:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11133871 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 71F0014ED for ; Thu, 5 Sep 2019 19:50:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2A6CD206BB for ; Thu, 5 Sep 2019 19:50:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391503AbfIETuS (ORCPT ); Thu, 5 Sep 2019 15:50:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]:60334 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388067AbfIETt1 (ORCPT ); Thu, 5 Sep 2019 15:49:27 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E1282308FC20; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id BBF88600F8; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id 08BE92253A3; Thu, 5 Sep 2019 15:49:18 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, miklos@szeredi.hu Cc: linux-kernel@vger.kernel.org, virtio-fs@redhat.com, vgoyal@redhat.com, stefanha@redhat.com, dgilbert@redhat.com, mst@redhat.com Subject: [PATCH 11/18] virtiofs: stop and drain queues after sending DESTROY Date: Thu, 5 Sep 2019 15:48:52 -0400 Message-Id: <20190905194859.16219-12-vgoyal@redhat.com> In-Reply-To: <20190905194859.16219-1-vgoyal@redhat.com> References: <20190905194859.16219-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Thu, 05 Sep 2019 19:49:26 +0000 (UTC) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org During virtio_kill_sb() we first stop forget queue and drain it and then call fuse_kill_sb_anon(). This will result in sending DESTROY request to fuse server. Once finished, stop all the queues and drain one more time just to be sure and then free up the devices. Given drain queues will call flush_work() on various workers, remove this logic from virtio_free_devs(). Signed-off-by: Vivek Goyal Reviewed-by: Stefan Hajnoczi --- fs/fuse/virtio_fs.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index 1ea0f889e804..a76bd5a04521 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -180,9 +180,6 @@ static void virtio_fs_free_devs(struct virtio_fs *fs) if (!fsvq->fud) continue; - flush_work(&fsvq->done_work); - flush_delayed_work(&fsvq->dispatch_work); - /* TODO need to quiesce/end_requests/decrement dev_count */ fuse_dev_free(fsvq->fud); fsvq->fud = NULL; @@ -994,6 +991,8 @@ static int virtio_fs_fill_super(struct super_block *sb) atomic_inc(&fc->dev_count); } + /* Previous unmount will stop all queues. Start these again */ + virtio_fs_start_all_queues(fs); fuse_send_init(fc, init_req); return 0; @@ -1026,6 +1025,12 @@ static void virtio_kill_sb(struct super_block *sb) virtio_fs_drain_all_queues(vfs); fuse_kill_sb_anon(sb); + + /* fuse_kill_sb_anon() must have sent destroy. Stop all queues + * and drain one more time and free fuse devices. + */ + virtio_fs_stop_all_queues(vfs); + virtio_fs_drain_all_queues(vfs); virtio_fs_free_devs(vfs); } From patchwork Thu Sep 5 19:48:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11133873 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 32C50112B for ; Thu, 5 Sep 2019 19:50:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id DF111206BB for ; Thu, 5 Sep 2019 19:50:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391500AbfIETuR (ORCPT ); Thu, 5 Sep 2019 15:50:17 -0400 Received: from mx1.redhat.com ([209.132.183.28]:38614 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388286AbfIETt1 (ORCPT ); Thu, 5 Sep 2019 15:49:27 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E4CCC7FDCD; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id BF6E01001B00; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id 0F63D2253A4; Thu, 5 Sep 2019 15:49:18 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, miklos@szeredi.hu Cc: linux-kernel@vger.kernel.org, virtio-fs@redhat.com, vgoyal@redhat.com, stefanha@redhat.com, dgilbert@redhat.com, mst@redhat.com Subject: [PATCH 12/18] virtiofs: Use virtio_fs_free_devs() in error path Date: Thu, 5 Sep 2019 15:48:53 -0400 Message-Id: <20190905194859.16219-13-vgoyal@redhat.com> In-Reply-To: <20190905194859.16219-1-vgoyal@redhat.com> References: <20190905194859.16219-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Thu, 05 Sep 2019 19:49:26 +0000 (UTC) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org We already have an helper to cleanup fuse devices. Use that instead of duplicating the code. Signed-off-by: Vivek Goyal Reviewed-by: Stefan Hajnoczi --- fs/fuse/virtio_fs.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index a76bd5a04521..40259368a6bd 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -999,8 +999,7 @@ static int virtio_fs_fill_super(struct super_block *sb) err_free_init_req: fuse_request_free(init_req); err_free_fuse_devs: - for (i = 0; i < fs->nvqs; i++) - fuse_dev_free(fs->vqs[i].fud); + virtio_fs_free_devs(fs); err: return err; } From patchwork Thu Sep 5 19:48:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11133875 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EDA6F112B for ; Thu, 5 Sep 2019 19:50:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A5A3320870 for ; Thu, 5 Sep 2019 19:50:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391496AbfIETuR (ORCPT ); Thu, 5 Sep 2019 15:50:17 -0400 Received: from mx1.redhat.com ([209.132.183.28]:40038 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388106AbfIETt1 (ORCPT ); Thu, 5 Sep 2019 15:49:27 -0400 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E8FA9307D851; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id C33BA5D6A3; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id 160592253A5; Thu, 5 Sep 2019 15:49:18 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, miklos@szeredi.hu Cc: linux-kernel@vger.kernel.org, virtio-fs@redhat.com, vgoyal@redhat.com, stefanha@redhat.com, dgilbert@redhat.com, mst@redhat.com Subject: [PATCH 13/18] virtiofs: Do not access virtqueue in request submission path Date: Thu, 5 Sep 2019 15:48:54 -0400 Message-Id: <20190905194859.16219-14-vgoyal@redhat.com> In-Reply-To: <20190905194859.16219-1-vgoyal@redhat.com> References: <20190905194859.16219-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.48]); Thu, 05 Sep 2019 19:49:27 +0000 (UTC) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org In request submission path it is possible that virtqueue is already gone due to driver->remove(). So do not access it in dev_dbg(). Use pr_debug() instead. If virtuqueue is gone, this will result in NULL pointer deference. Signed-off-by: Vivek Goyal Reviewed-by: Stefan Hajnoczi --- fs/fuse/virtio_fs.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index 40259368a6bd..01bbf2c0e144 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -888,10 +888,10 @@ __releases(fiq->waitq.lock) fs = fiq->priv; fc = fs->vqs[queue_id].fud->fc; - dev_dbg(&fs->vqs[queue_id].vq->vdev->dev, - "%s: opcode %u unique %#llx nodeid %#llx in.len %u out.len %u\n", - __func__, req->in.h.opcode, req->in.h.unique, req->in.h.nodeid, - req->in.h.len, fuse_len_args(req->out.numargs, req->out.args)); + pr_debug("%s: opcode %u unique %#llx nodeid %#llx in.len %u out.len %u" + "\n", __func__, req->in.h.opcode, req->in.h.unique, + req->in.h.nodeid, req->in.h.len, + fuse_len_args(req->out.numargs, req->out.args)); fpq = &fs->vqs[queue_id].fud->pq; spin_lock(&fpq->lock); From patchwork Thu Sep 5 19:48:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11133859 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 532C4112B for ; Thu, 5 Sep 2019 19:50:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0CE502145D for ; Thu, 5 Sep 2019 19:50:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2389041AbfIETt2 (ORCPT ); Thu, 5 Sep 2019 15:49:28 -0400 Received: from mx1.redhat.com ([209.132.183.28]:1774 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388189AbfIETt1 (ORCPT ); Thu, 5 Sep 2019 15:49:27 -0400 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id ECCAB8831E; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id C70AA100194E; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id 1BBEC2253A6; Thu, 5 Sep 2019 15:49:18 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, miklos@szeredi.hu Cc: linux-kernel@vger.kernel.org, virtio-fs@redhat.com, vgoyal@redhat.com, stefanha@redhat.com, dgilbert@redhat.com, mst@redhat.com Subject: [PATCH 14/18] virtiofs: Add a fuse_iqueue operation to put() reference Date: Thu, 5 Sep 2019 15:48:55 -0400 Message-Id: <20190905194859.16219-15-vgoyal@redhat.com> In-Reply-To: <20190905194859.16219-1-vgoyal@redhat.com> References: <20190905194859.16219-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Thu, 05 Sep 2019 19:49:27 +0000 (UTC) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Soon I will make virtio_fs object reference counted, where reference will be taken by device as well as by fuse_conn (fuse_conn->fuse_iqueue->fiq_priv). When fuse_connection is going away, it should put its reference on virtio_fs object. So add a fuse_iqueue method which can be used to call into virtio_fs to put the reference on the object (fiq_priv). Signed-off-by: Vivek Goyal Signed-off-by: Vivek Goyal --- fs/fuse/fuse_i.h | 5 +++++ fs/fuse/inode.c | 6 ++++++ 2 files changed, 11 insertions(+) diff --git a/fs/fuse/fuse_i.h b/fs/fuse/fuse_i.h index 85e2dcad68c1..04e2c000d63f 100644 --- a/fs/fuse/fuse_i.h +++ b/fs/fuse/fuse_i.h @@ -479,6 +479,11 @@ struct fuse_iqueue_ops { */ void (*wake_pending_and_unlock)(struct fuse_iqueue *fiq) __releases(fiq->waitq.lock); + + /** + * Put a reference on fiq_priv. + */ + void (*put)(struct fuse_iqueue *fiq); }; /** /dev/fuse input queue operations */ diff --git a/fs/fuse/inode.c b/fs/fuse/inode.c index 7fa0dcc6f565..70a433bdf01f 100644 --- a/fs/fuse/inode.c +++ b/fs/fuse/inode.c @@ -631,8 +631,14 @@ EXPORT_SYMBOL_GPL(fuse_conn_init); void fuse_conn_put(struct fuse_conn *fc) { if (refcount_dec_and_test(&fc->count)) { + struct fuse_iqueue *fiq = &fc->iq; + if (fc->destroy_req) fuse_request_free(fc->destroy_req); + if (fiq->priv && fiq->ops->put) { + fiq->ops->put(fiq); + fiq->priv = NULL; + } put_pid_ns(fc->pid_ns); put_user_ns(fc->user_ns); fc->release(fc); From patchwork Thu Sep 5 19:48:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11133865 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 76F6414ED for ; Thu, 5 Sep 2019 19:50:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 37EB020825 for ; Thu, 5 Sep 2019 19:50:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391469AbfIETuJ (ORCPT ); Thu, 5 Sep 2019 15:50:09 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51300 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388226AbfIETt1 (ORCPT ); Thu, 5 Sep 2019 15:49:27 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EC01110F2402; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id C94AA60C18; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id 2184C2253A7; Thu, 5 Sep 2019 15:49:18 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, miklos@szeredi.hu Cc: linux-kernel@vger.kernel.org, virtio-fs@redhat.com, vgoyal@redhat.com, stefanha@redhat.com, dgilbert@redhat.com, mst@redhat.com Subject: [PATCH 15/18] virtiofs: Make virtio_fs object refcounted Date: Thu, 5 Sep 2019 15:48:56 -0400 Message-Id: <20190905194859.16219-16-vgoyal@redhat.com> In-Reply-To: <20190905194859.16219-1-vgoyal@redhat.com> References: <20190905194859.16219-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.66]); Thu, 05 Sep 2019 19:49:27 +0000 (UTC) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This object is used both by fuse_connection as well virt device. So make this object reference counted and that makes it easy to define life cycle of the object. Now deivce can be removed while filesystem is still mounted. This will cleanup all the virtqueues but virtio_fs object will still be around and will be cleaned when filesystem is unmounted and sb/fc drops its reference. Removing a device also stops all virt queues and any new reuqest gets error -ENOTCONN. All existing in flight requests are drained before ->remove returns. Signed-off-by: Vivek Goyal Signed-off-by: Vivek Goyal Reviewed-by: Stefan Hajnoczi --- fs/fuse/virtio_fs.c | 52 +++++++++++++++++++++++++++++++++++++-------- 1 file changed, 43 insertions(+), 9 deletions(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index 01bbf2c0e144..29ec2f5bbbe2 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -37,6 +37,7 @@ struct virtio_fs_vq { /* A virtio-fs device instance */ struct virtio_fs { + struct kref refcount; struct list_head list; /* on virtio_fs_instances */ char *tag; struct virtio_fs_vq *vqs; @@ -63,6 +64,27 @@ static inline struct fuse_pqueue *vq_to_fpq(struct virtqueue *vq) return &vq_to_fsvq(vq)->fud->pq; } +static void release_virtiofs_obj(struct kref *ref) +{ + struct virtio_fs *vfs = container_of(ref, struct virtio_fs, refcount); + + kfree(vfs->vqs); + kfree(vfs); +} + +static void virtiofs_put(struct virtio_fs *fs) +{ + mutex_lock(&virtio_fs_mutex); + kref_put(&fs->refcount, release_virtiofs_obj); + mutex_unlock(&virtio_fs_mutex); +} + +static void virtio_fs_put(struct fuse_iqueue *fiq) +{ + struct virtio_fs *vfs = fiq->priv; + virtiofs_put(vfs); +} + static void virtio_fs_drain_queue(struct virtio_fs_vq *fsvq) { WARN_ON(fsvq->in_flight < 0); @@ -156,8 +178,10 @@ static struct virtio_fs *virtio_fs_find_instance(const char *tag) mutex_lock(&virtio_fs_mutex); list_for_each_entry(fs, &virtio_fs_instances, list) { - if (strcmp(fs->tag, tag) == 0) + if (strcmp(fs->tag, tag) == 0) { + kref_get(&fs->refcount); goto found; + } } fs = NULL; /* not found */ @@ -519,6 +543,7 @@ static int virtio_fs_probe(struct virtio_device *vdev) fs = kzalloc(sizeof(*fs), GFP_KERNEL); if (!fs) return -ENOMEM; + kref_init(&fs->refcount); vdev->priv = fs; ret = virtio_fs_read_tag(vdev, fs); @@ -570,18 +595,18 @@ static void virtio_fs_remove(struct virtio_device *vdev) { struct virtio_fs *fs = vdev->priv; + mutex_lock(&virtio_fs_mutex); + list_del_init(&fs->list); + mutex_unlock(&virtio_fs_mutex); + virtio_fs_stop_all_queues(fs); virtio_fs_drain_all_queues(fs); vdev->config->reset(vdev); virtio_fs_cleanup_vqs(vdev, fs); - mutex_lock(&virtio_fs_mutex); - list_del(&fs->list); - mutex_unlock(&virtio_fs_mutex); - vdev->priv = NULL; - kfree(fs->vqs); - kfree(fs); + /* Put device reference on virtio_fs object */ + virtiofs_put(fs); } #ifdef CONFIG_PM_SLEEP @@ -932,6 +957,7 @@ const static struct fuse_iqueue_ops virtio_fs_fiq_ops = { .wake_forget_and_unlock = virtio_fs_wake_forget_and_unlock, .wake_interrupt_and_unlock = virtio_fs_wake_interrupt_and_unlock, .wake_pending_and_unlock = virtio_fs_wake_pending_and_unlock, + .put = virtio_fs_put, }; static int virtio_fs_fill_super(struct super_block *sb) @@ -1026,7 +1052,9 @@ static void virtio_kill_sb(struct super_block *sb) fuse_kill_sb_anon(sb); /* fuse_kill_sb_anon() must have sent destroy. Stop all queues - * and drain one more time and free fuse devices. + * and drain one more time and free fuse devices. Freeing fuse + * devices will drop their reference on fuse_conn and that in + * turn will drop its reference on virtio_fs object. */ virtio_fs_stop_all_queues(vfs); virtio_fs_drain_all_queues(vfs); @@ -1060,6 +1088,10 @@ static int virtio_fs_get_tree(struct fs_context *fsc) struct fuse_conn *fc; int err; + /* This gets a reference on virtio_fs object. This ptr gets installed + * in fc->iq->priv. Once fuse_conn is going away, it calls ->put() + * to drop the reference to this object. + */ fs = virtio_fs_find_instance(fsc->source); if (!fs) { pr_info("virtio-fs: tag <%s> not found\n", fsc->source); @@ -1067,8 +1099,10 @@ static int virtio_fs_get_tree(struct fs_context *fsc) } fc = kzalloc(sizeof(struct fuse_conn), GFP_KERNEL); - if (!fc) + if (!fc) { + virtiofs_put(fs); return -ENOMEM; + } fuse_conn_init(fc, get_user_ns(current_user_ns()), &virtio_fs_fiq_ops, fs); From patchwork Thu Sep 5 19:48:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11133861 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6067414B4 for ; Thu, 5 Sep 2019 19:50:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 21310206BB for ; Thu, 5 Sep 2019 19:50:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390470AbfIETuA (ORCPT ); Thu, 5 Sep 2019 15:50:00 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50886 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388426AbfIETt2 (ORCPT ); Thu, 5 Sep 2019 15:49:28 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 6A6BF59451; Thu, 5 Sep 2019 19:49:27 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id D40106012D; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id 26FF12253A8; Thu, 5 Sep 2019 15:49:18 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, miklos@szeredi.hu Cc: linux-kernel@vger.kernel.org, virtio-fs@redhat.com, vgoyal@redhat.com, stefanha@redhat.com, dgilbert@redhat.com, mst@redhat.com Subject: [PATCH 16/18] virtiofs: Use virtio_fs_mutex for races w.r.t ->remove and mount path Date: Thu, 5 Sep 2019 15:48:57 -0400 Message-Id: <20190905194859.16219-17-vgoyal@redhat.com> In-Reply-To: <20190905194859.16219-1-vgoyal@redhat.com> References: <20190905194859.16219-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Thu, 05 Sep 2019 19:49:27 +0000 (UTC) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org It is possible that a mount is in progress and device is being removed at the same time. Use virtio_fs_mutex to avoid races. This also takes care of bunch of races and removes some TODO items. Signed-off-by: Vivek Goyal Reviewed-by: Stefan Hajnoczi Signed-off-by: Vivek Goyal Reviewed-by: Stefan Hajnoczi --- fs/fuse/virtio_fs.c | 32 ++++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index 29ec2f5bbbe2..c483482185b6 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -13,7 +13,9 @@ #include #include "fuse_i.h" -/* List of virtio-fs device instances and a lock for the list */ +/* List of virtio-fs device instances and a lock for the list. Also provides + * mutual exclusion in device removal and mounting path + */ static DEFINE_MUTEX(virtio_fs_mutex); static LIST_HEAD(virtio_fs_instances); @@ -72,17 +74,19 @@ static void release_virtiofs_obj(struct kref *ref) kfree(vfs); } +/* Make sure virtiofs_mutex is held */ static void virtiofs_put(struct virtio_fs *fs) { - mutex_lock(&virtio_fs_mutex); kref_put(&fs->refcount, release_virtiofs_obj); - mutex_unlock(&virtio_fs_mutex); } static void virtio_fs_put(struct fuse_iqueue *fiq) { struct virtio_fs *vfs = fiq->priv; + + mutex_lock(&virtio_fs_mutex); virtiofs_put(vfs); + mutex_unlock(&virtio_fs_mutex); } static void virtio_fs_drain_queue(struct virtio_fs_vq *fsvq) @@ -596,9 +600,8 @@ static void virtio_fs_remove(struct virtio_device *vdev) struct virtio_fs *fs = vdev->priv; mutex_lock(&virtio_fs_mutex); + /* This device is going away. No one should get new reference */ list_del_init(&fs->list); - mutex_unlock(&virtio_fs_mutex); - virtio_fs_stop_all_queues(fs); virtio_fs_drain_all_queues(fs); vdev->config->reset(vdev); @@ -607,6 +610,7 @@ static void virtio_fs_remove(struct virtio_device *vdev) vdev->priv = NULL; /* Put device reference on virtio_fs object */ virtiofs_put(fs); + mutex_unlock(&virtio_fs_mutex); } #ifdef CONFIG_PM_SLEEP @@ -978,10 +982,15 @@ static int virtio_fs_fill_super(struct super_block *sb) .no_force_umount = true, }; - /* TODO lock */ - if (fs->vqs[VQ_REQUEST].fud) { - pr_err("virtio-fs: device already in use\n"); - err = -EBUSY; + mutex_lock(&virtio_fs_mutex); + + /* After holding mutex, make sure virtiofs device is still there. + * Though we are holding a refernce to it, drive ->remove might + * still have cleaned up virtual queues. In that case bail out. + */ + err = -EINVAL; + if (list_empty(&fs->list)) { + pr_info("virtio-fs: tag <%s> not found\n", fs->tag); goto err; } @@ -1007,7 +1016,6 @@ static int virtio_fs_fill_super(struct super_block *sb) fc = fs->vqs[VQ_REQUEST].fud->fc; - /* TODO take fuse_mutex around this loop? */ for (i = 0; i < fs->nvqs; i++) { struct virtio_fs_vq *fsvq = &fs->vqs[i]; @@ -1020,6 +1028,7 @@ static int virtio_fs_fill_super(struct super_block *sb) /* Previous unmount will stop all queues. Start these again */ virtio_fs_start_all_queues(fs); fuse_send_init(fc, init_req); + mutex_unlock(&virtio_fs_mutex); return 0; err_free_init_req: @@ -1027,6 +1036,7 @@ static int virtio_fs_fill_super(struct super_block *sb) err_free_fuse_devs: virtio_fs_free_devs(fs); err: + mutex_unlock(&virtio_fs_mutex); return err; } @@ -1100,7 +1110,9 @@ static int virtio_fs_get_tree(struct fs_context *fsc) fc = kzalloc(sizeof(struct fuse_conn), GFP_KERNEL); if (!fc) { + mutex_lock(&virtio_fs_mutex); virtiofs_put(fs); + mutex_unlock(&virtio_fs_mutex); return -ENOMEM; } From patchwork Thu Sep 5 19:48:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11133877 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1C29E14B4 for ; Thu, 5 Sep 2019 19:50:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C6ACD20825 for ; Thu, 5 Sep 2019 19:50:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391492AbfIETuR (ORCPT ); Thu, 5 Sep 2019 15:50:17 -0400 Received: from mx1.redhat.com ([209.132.183.28]:60340 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388269AbfIETt1 (ORCPT ); Thu, 5 Sep 2019 15:49:27 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 01828308FBAC; Thu, 5 Sep 2019 19:49:27 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id CFF0660127; Thu, 5 Sep 2019 19:49:26 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id 2B9612253A9; Thu, 5 Sep 2019 15:49:18 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, miklos@szeredi.hu Cc: linux-kernel@vger.kernel.org, virtio-fs@redhat.com, vgoyal@redhat.com, stefanha@redhat.com, dgilbert@redhat.com, mst@redhat.com Subject: [PATCH 17/18] virtiofs: Remove TODO to quiesce/end_requests Date: Thu, 5 Sep 2019 15:48:58 -0400 Message-Id: <20190905194859.16219-18-vgoyal@redhat.com> In-Reply-To: <20190905194859.16219-1-vgoyal@redhat.com> References: <20190905194859.16219-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]); Thu, 05 Sep 2019 19:49:27 +0000 (UTC) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org We now stop queues and drain all the pending requests from all virtqueues. So this is not a TODO anymore. Got rid of incrementing fc->dev_count as well. It did not seem meaningful for virtio_fs. Signed-off-by: Vivek Goyal Reviewed-by: Stefan Hajnoczi --- fs/fuse/virtio_fs.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index c483482185b6..eadaea6eb8e2 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -208,7 +208,6 @@ static void virtio_fs_free_devs(struct virtio_fs *fs) if (!fsvq->fud) continue; - /* TODO need to quiesce/end_requests/decrement dev_count */ fuse_dev_free(fsvq->fud); fsvq->fud = NULL; } @@ -1022,7 +1021,6 @@ static int virtio_fs_fill_super(struct super_block *sb) if (i == VQ_REQUEST) continue; /* already initialized */ fuse_dev_install(fsvq->fud, fc); - atomic_inc(&fc->dev_count); } /* Previous unmount will stop all queues. Start these again */ From patchwork Thu Sep 5 19:48:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 11133851 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1BDDE14B4 for ; Thu, 5 Sep 2019 19:49:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C86DE20870 for ; Thu, 5 Sep 2019 19:49:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2389945AbfIETt2 (ORCPT ); Thu, 5 Sep 2019 15:49:28 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49358 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388340AbfIETt1 (ORCPT ); Thu, 5 Sep 2019 15:49:27 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2CCCC30860B9; Thu, 5 Sep 2019 19:49:27 +0000 (UTC) Received: from horse.redhat.com (unknown [10.18.25.137]) by smtp.corp.redhat.com (Postfix) with ESMTP id 093FD5C1D4; Thu, 5 Sep 2019 19:49:27 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id 30FDB2253AA; Thu, 5 Sep 2019 15:49:18 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtualization@lists.linux-foundation.org, miklos@szeredi.hu Cc: linux-kernel@vger.kernel.org, virtio-fs@redhat.com, vgoyal@redhat.com, stefanha@redhat.com, dgilbert@redhat.com, mst@redhat.com Subject: [PATCH 18/18] virtiofs: Remove TODO item from virtio_fs_free_devs() Date: Thu, 5 Sep 2019 15:48:59 -0400 Message-Id: <20190905194859.16219-19-vgoyal@redhat.com> In-Reply-To: <20190905194859.16219-1-vgoyal@redhat.com> References: <20190905194859.16219-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]); Thu, 05 Sep 2019 19:49:27 +0000 (UTC) Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org virtio_fs_free_devs() is now called from ->kill_sb(). By this time all device queues have been quiesced. I am assuming that while ->kill_sb() is in progress, another mount instance will wait for it to finish (sb->s_umount mutex provides mutual exclusion). W.r.t ->remove path, we should be fine as we are not touching vdev or virtqueues. And we have reference on virtio_fs object, so we know rest of the data structures are valid. So I can't see the need of any additional locking yet. Signed-off-by: Vivek Goyal Reviewed-by: Stefan Hajnoczi --- fs/fuse/virtio_fs.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index eadaea6eb8e2..61aa3eba7b22 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -200,8 +200,6 @@ static void virtio_fs_free_devs(struct virtio_fs *fs) { unsigned int i; - /* TODO lock */ - for (i = 0; i < fs->nvqs; i++) { struct virtio_fs_vq *fsvq = &fs->vqs[i];