From patchwork Thu Sep 30 14:38:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 12528581 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9908BC433F5 for ; Thu, 30 Sep 2021 14:39:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7E79F61A08 for ; Thu, 30 Sep 2021 14:39:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350910AbhI3OlQ (ORCPT ); Thu, 30 Sep 2021 10:41:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:32334 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349279AbhI3OlN (ORCPT ); Thu, 30 Sep 2021 10:41:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633012770; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GgAYzhk++zDg+2draM1yDlVHhr5FM6w/twV29vqzoks=; b=ED7dkOGbOqde5fkWj5ZGj+5Ju4SjXwnA2PBPmPz8rdztGAbWORxyQLFra1cTpXCXUwJuK8 wQ+lCJfrALu6nr4Zas8I8SWfkYxlZt7D4YLCr1LIuz/jrRl4oogtHR3c5jBTX8PgH3TrX/ vbqAzsnnubhDDKm8WhvIAqysOJHe/Rk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-468-PZjaIFS5PO6-FEdphC70SQ-1; Thu, 30 Sep 2021 10:39:28 -0400 X-MC-Unique: PZjaIFS5PO6-FEdphC70SQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8A9D6192376D; Thu, 30 Sep 2021 14:39:27 +0000 (UTC) Received: from horse.redhat.com (unknown [10.22.16.146]) by smtp.corp.redhat.com (Postfix) with ESMTP id 38675100760B; Thu, 30 Sep 2021 14:39:07 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id BCF12228280; Thu, 30 Sep 2021 10:39:06 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtio-fs@redhat.com, miklos@szeredi.hu, stefanha@redhat.com Cc: vgoyal@redhat.com, iangelak@redhat.com, jaggel@bu.edu, dgilbert@redhat.com Subject: [PATCH 1/8] virtiofs: Disable interrupt requests properly Date: Thu, 30 Sep 2021 10:38:43 -0400 Message-Id: <20210930143850.1188628-2-vgoyal@redhat.com> In-Reply-To: <20210930143850.1188628-1-vgoyal@redhat.com> References: <20210930143850.1188628-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org virtiofs does not support dealing with fuse INTERRUPT requests at all. But still we set can clear FR_SENT bit which is needed only if INTERRUPT requests are being handled. Also, given current code it is possible that virtiofs server is handling a request and in guest a signal comes, it will wake up process and queue existing request to fiq->interrupts and never remove it. request_wait_answer() { if (!fc->no_interupt) { if (test_bit(FR_SENT, &req->flags)) queue_interrupt(req); } } Given virtiofs does not support interrupt requests at this point of time, disable it (Set fc->no_interrupt = 1). This should make sure requests can't be queued on fiq->interrupts. Signed-off-by: Vivek Goyal --- fs/fuse/virtio_fs.c | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index 0ad89c6629d7..b9256b8f277f 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -545,7 +545,6 @@ static void copy_args_from_argbuf(struct fuse_args *args, struct fuse_req *req) static void virtio_fs_request_complete(struct fuse_req *req, struct virtio_fs_vq *fsvq) { - struct fuse_pqueue *fpq = &fsvq->fud->pq; struct fuse_args *args; struct fuse_args_pages *ap; unsigned int len, i, thislen; @@ -574,10 +573,6 @@ static void virtio_fs_request_complete(struct fuse_req *req, } } - spin_lock(&fpq->lock); - clear_bit(FR_SENT, &req->flags); - spin_unlock(&fpq->lock); - fuse_request_end(req); spin_lock(&fsvq->lock); dec_in_flight_req(fsvq); @@ -1196,9 +1191,6 @@ static int virtio_fs_enqueue_req(struct virtio_fs_vq *fsvq, spin_lock(&fpq->lock); list_add_tail(&req->list, fpq->processing); spin_unlock(&fpq->lock); - set_bit(FR_SENT, &req->flags); - /* matches barrier in request_wait_answer() */ - smp_mb__after_atomic(); if (!in_flight) inc_in_flight_req(fsvq); @@ -1448,6 +1440,7 @@ static int virtio_fs_get_tree(struct fs_context *fsc) fc->delete_stale = true; fc->auto_submounts = true; fc->sync_fs = true; + fc->no_interrupt = true; /* Tell FUSE to split requests that exceed the virtqueue's size */ fc->max_pages_limit = min_t(unsigned int, fc->max_pages_limit, From patchwork Thu Sep 30 14:38:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 12528569 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C923C433F5 for ; Thu, 30 Sep 2021 14:39:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 26B3A61A0A for ; Thu, 30 Sep 2021 14:39:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350909AbhI3Ok4 (ORCPT ); Thu, 30 Sep 2021 10:40:56 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:47866 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350878AbhI3Okx (ORCPT ); Thu, 30 Sep 2021 10:40:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633012750; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CFkyMjzkyJBm3WnZsiRLAeCCVWKPXTDbggPUFe0w8QQ=; b=ew7kLravks+OIVsvewxz77UdESlNakooswRJ64w1gXcU5pWCzxgNRQGdJcuo9jwS3RuG+b /0jbxAzsqUxfb7N6EqqmOacJQ6UN+HDfB4Th38ZHJZnpeKzCuZ0zy1oDDRqyqa83h7V0i4 8S9ezovNmTDWHGXasfp2pfieU4xFjOU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-158-xTjNnye1OUqsnO2Iy_XL9g-1; Thu, 30 Sep 2021 10:39:08 -0400 X-MC-Unique: xTjNnye1OUqsnO2Iy_XL9g-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 837799126D; Thu, 30 Sep 2021 14:39:07 +0000 (UTC) Received: from horse.redhat.com (unknown [10.22.16.146]) by smtp.corp.redhat.com (Postfix) with ESMTP id 31A3318EDF; Thu, 30 Sep 2021 14:39:07 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id C1FA5228281; Thu, 30 Sep 2021 10:39:06 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtio-fs@redhat.com, miklos@szeredi.hu, stefanha@redhat.com Cc: vgoyal@redhat.com, iangelak@redhat.com, jaggel@bu.edu, dgilbert@redhat.com Subject: [PATCH 2/8] virtiofs: Fix a comment about fuse_dev allocation Date: Thu, 30 Sep 2021 10:38:44 -0400 Message-Id: <20210930143850.1188628-3-vgoyal@redhat.com> In-Reply-To: <20210930143850.1188628-1-vgoyal@redhat.com> References: <20210930143850.1188628-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Now virtiofs allocates fuse_dev for all queues and does not rely on fuse common code to allocate for request queue. Fix the comment. Signed-off-by: Vivek Goyal --- fs/fuse/virtio_fs.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index b9256b8f277f..f7c58a4b996d 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -1307,7 +1307,7 @@ static int virtio_fs_fill_super(struct super_block *sb, struct fs_context *fsc) } err = -ENOMEM; - /* Allocate fuse_dev for hiprio and notification queues */ + /* Allocate fuse_dev for all the queues */ for (i = 0; i < fs->nvqs; i++) { struct virtio_fs_vq *fsvq = &fs->vqs[i]; From patchwork Thu Sep 30 14:38:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 12528571 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9A7EC433EF for ; Thu, 30 Sep 2021 14:39:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BF06D61A0A for ; Thu, 30 Sep 2021 14:39:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350934AbhI3Ok6 (ORCPT ); Thu, 30 Sep 2021 10:40:58 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:29376 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350879AbhI3Okx (ORCPT ); Thu, 30 Sep 2021 10:40:53 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633012750; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UtUkQWJbDRAqbfT3IX+l0UQCv/ZZhkQb3oKNdmjXXWQ=; b=T65eObLUKUuRz0BcQTWv4Xyvl4O+lmpiJPEGEWdeO80ZdBto3S6cS2XjyJNYpu2Bm4HvJ3 ysMX/hCUSWAB7ISOmIYKTjVuR3VUbT5pPFCkdVBN7L8K69KIjsoi2e4HVVVOfEO+kydTUj UUw4lrGZqAlSBMUB/LcUEDB+dtjhloc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-235-douBEBxUPGKUfpWHua6YJw-1; Thu, 30 Sep 2021 10:39:08 -0400 X-MC-Unique: douBEBxUPGKUfpWHua6YJw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 84CD4802947; Thu, 30 Sep 2021 14:39:07 +0000 (UTC) Received: from horse.redhat.com (unknown [10.22.16.146]) by smtp.corp.redhat.com (Postfix) with ESMTP id 315C65F4E1; Thu, 30 Sep 2021 14:39:07 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id C756C228282; Thu, 30 Sep 2021 10:39:06 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtio-fs@redhat.com, miklos@szeredi.hu, stefanha@redhat.com Cc: vgoyal@redhat.com, iangelak@redhat.com, jaggel@bu.edu, dgilbert@redhat.com Subject: [PATCH 3/8] virtiofs: Add an index to keep track of first request queue Date: Thu, 30 Sep 2021 10:38:45 -0400 Message-Id: <20210930143850.1188628-4-vgoyal@redhat.com> In-Reply-To: <20210930143850.1188628-1-vgoyal@redhat.com> References: <20210930143850.1188628-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org We have many virtqueues and first queue which carries fuse normal requests (except forget requests) has index pointed to by enum VQ_REQUEST. This works fine as long as number of queues are not dynamic. I am about to introduce one more virtqueue, called notification queue, which will be present only if device on host supports it. That means index of request queue will change depending on if notification queue is present or not. So, add a variable to keep track of that index and this will help when notification queue is added in next patch. Signed-off-by: Vivek Goyal Signed-off-by: Ioannis Angelakopoulos --- fs/fuse/virtio_fs.c | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index f7c58a4b996d..cb3c7bf8cce4 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -61,6 +61,7 @@ struct virtio_fs { unsigned int nvqs; /* number of virtqueues */ unsigned int num_request_queues; /* number of request queues */ struct dax_device *dax_dev; + unsigned int first_reqq_idx; /* First request queue idx */ /* DAX memory window where file contents are mapped */ void *window_kaddr; @@ -676,7 +677,9 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev, if (fs->num_request_queues == 0) return -EINVAL; - fs->nvqs = VQ_REQUEST + fs->num_request_queues; + /* One hiprio queue and rest are request queues */ + fs->nvqs = 1 + fs->num_request_queues; + fs->first_reqq_idx = 1; fs->vqs = kcalloc(fs->nvqs, sizeof(fs->vqs[VQ_HIPRIO]), GFP_KERNEL); if (!fs->vqs) return -ENOMEM; @@ -696,10 +699,11 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev, names[VQ_HIPRIO] = fs->vqs[VQ_HIPRIO].name; /* Initialize the requests virtqueues */ - for (i = VQ_REQUEST; i < fs->nvqs; i++) { + for (i = fs->first_reqq_idx; i < fs->nvqs; i++) { char vq_name[VQ_NAME_LEN]; - snprintf(vq_name, VQ_NAME_LEN, "requests.%u", i - VQ_REQUEST); + snprintf(vq_name, VQ_NAME_LEN, "requests.%u", + i - fs->first_reqq_idx); virtio_fs_init_vq(&fs->vqs[i], vq_name, VQ_REQUEST); callbacks[i] = virtio_fs_vq_done; names[i] = fs->vqs[i].name; @@ -1217,7 +1221,7 @@ static int virtio_fs_enqueue_req(struct virtio_fs_vq *fsvq, static void virtio_fs_wake_pending_and_unlock(struct fuse_iqueue *fiq) __releases(fiq->lock) { - unsigned int queue_id = VQ_REQUEST; /* TODO multiqueue */ + unsigned int queue_id; struct virtio_fs *fs; struct fuse_req *req; struct virtio_fs_vq *fsvq; @@ -1231,6 +1235,7 @@ __releases(fiq->lock) spin_unlock(&fiq->lock); fs = fiq->priv; + queue_id = fs->first_reqq_idx; pr_debug("%s: opcode %u unique %#llx nodeid %#llx in.len %u out.len %u\n", __func__, req->in.h.opcode, req->in.h.unique, @@ -1411,6 +1416,7 @@ static int virtio_fs_get_tree(struct fs_context *fsc) struct fuse_mount *fm; unsigned int virtqueue_size; int err = -EIO; + struct virtio_fs_vq *first_req_fsvq; /* This gets a reference on virtio_fs object. This ptr gets installed * in fc->iq->priv. Once fuse_conn is going away, it calls ->put() @@ -1422,7 +1428,8 @@ static int virtio_fs_get_tree(struct fs_context *fsc) return -EINVAL; } - virtqueue_size = virtqueue_get_vring_size(fs->vqs[VQ_REQUEST].vq); + first_req_fsvq = &fs->vqs[fs->first_reqq_idx]; + virtqueue_size = virtqueue_get_vring_size(first_req_fsvq->vq); if (WARN_ON(virtqueue_size <= FUSE_HEADER_OVERHEAD)) goto out_err; From patchwork Thu Sep 30 14:38:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 12528567 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EEA95C433F5 for ; Thu, 30 Sep 2021 14:39:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D637E61A08 for ; Thu, 30 Sep 2021 14:39:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350827AbhI3Oky (ORCPT ); Thu, 30 Sep 2021 10:40:54 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:56162 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350747AbhI3Okw (ORCPT ); Thu, 30 Sep 2021 10:40:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633012749; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KgZVtOTTDdsKFqb7q2YTJxpKsWagNPSr07ZEAPEpvbg=; b=OMhIbJ0V+CFXNDQVUYENB8apK7POL0Detko9neUEcX0tUrqml162PJI2PPOn7a3Jg6i32p WYuv9/6gkcH/791UvfcLA19cosHTM54T+jkOPvOrSFgJIdrc/0KhAWvxsu6r1iY+OFGv7w zZayLgfe6MmRxPBBpZoRavweV7DkV1M= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-576-SFuzmHIsP3SX-Vdef9YU8w-1; Thu, 30 Sep 2021 10:39:08 -0400 X-MC-Unique: SFuzmHIsP3SX-Vdef9YU8w-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 938D1100CC99; Thu, 30 Sep 2021 14:39:07 +0000 (UTC) Received: from horse.redhat.com (unknown [10.22.16.146]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2A49C5D9CA; Thu, 30 Sep 2021 14:39:07 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id CCA19228283; Thu, 30 Sep 2021 10:39:06 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtio-fs@redhat.com, miklos@szeredi.hu, stefanha@redhat.com Cc: vgoyal@redhat.com, iangelak@redhat.com, jaggel@bu.edu, dgilbert@redhat.com Subject: [PATCH 4/8] virtiofs: Decouple queue index and queue type Date: Thu, 30 Sep 2021 10:38:46 -0400 Message-Id: <20210930143850.1188628-5-vgoyal@redhat.com> In-Reply-To: <20210930143850.1188628-1-vgoyal@redhat.com> References: <20210930143850.1188628-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Right now we use a single enum {VQ_HIPRIO, VQ_REQUEST} and this seems to be being used to communicate both virtqueue index as well as virtqueue type. For example, virtio_fs_init_vq(..,..,..,vq_type) expects queue type in vq_type parameter. In rest of the code we are also using this enum as queue index. This is little confusing. At the same time, queue index situation is about to become little complicated and dynamic with the introduction of notification queue. Request queue index is not going to be determined at compile time. It will be dynamic based on whether notification queue is offered by device or not. So do not use this enum for both the purposes. Instead use it only to denote virtqueue type. For queue index, use macros where queue index is fixed and use a variable where queue index is not fixed. In the previous patch we are already using a variable ->first_reqq_idx for request queue index. This patch defines VQ_HIPRIO_IDX to keep track of hiprio virtqueue index. This patch also renames the enum elements to make it explicit that these representing virtqueue type (and not index). Signed-off-by: Vivek Goyal --- fs/fuse/virtio_fs.c | 29 ++++++++++++++++------------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index cb3c7bf8cce4..eef9591de640 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -24,6 +24,8 @@ */ #define FUSE_HEADER_OVERHEAD 4 +#define VQ_HIPRIO_IDX 0 + /* List of virtio-fs device instances and a lock for the list. Also provides * mutual exclusion in device removal and mounting path */ @@ -31,8 +33,8 @@ static DEFINE_MUTEX(virtio_fs_mutex); static LIST_HEAD(virtio_fs_instances); enum { - VQ_HIPRIO, - VQ_REQUEST + VQ_TYPE_HIPRIO, + VQ_TYPE_REQUEST }; #define VQ_NAME_LEN 24 @@ -651,7 +653,7 @@ static void virtio_fs_init_vq(struct virtio_fs_vq *fsvq, char *name, INIT_LIST_HEAD(&fsvq->end_reqs); init_completion(&fsvq->in_flight_zero); - if (vq_type == VQ_REQUEST) { + if (vq_type == VQ_TYPE_REQUEST) { INIT_WORK(&fsvq->done_work, virtio_fs_requests_done_work); INIT_DELAYED_WORK(&fsvq->dispatch_work, virtio_fs_request_dispatch_work); @@ -680,23 +682,24 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev, /* One hiprio queue and rest are request queues */ fs->nvqs = 1 + fs->num_request_queues; fs->first_reqq_idx = 1; - fs->vqs = kcalloc(fs->nvqs, sizeof(fs->vqs[VQ_HIPRIO]), GFP_KERNEL); + fs->vqs = kcalloc(fs->nvqs, sizeof(fs->vqs[VQ_HIPRIO_IDX]), GFP_KERNEL); if (!fs->vqs) return -ENOMEM; - vqs = kmalloc_array(fs->nvqs, sizeof(vqs[VQ_HIPRIO]), GFP_KERNEL); - callbacks = kmalloc_array(fs->nvqs, sizeof(callbacks[VQ_HIPRIO]), + vqs = kmalloc_array(fs->nvqs, sizeof(vqs[VQ_HIPRIO_IDX]), GFP_KERNEL); + callbacks = kmalloc_array(fs->nvqs, sizeof(callbacks[VQ_HIPRIO_IDX]), GFP_KERNEL); - names = kmalloc_array(fs->nvqs, sizeof(names[VQ_HIPRIO]), GFP_KERNEL); + names = kmalloc_array(fs->nvqs, sizeof(names[VQ_HIPRIO_IDX]), + GFP_KERNEL); if (!vqs || !callbacks || !names) { ret = -ENOMEM; goto out; } /* Initialize the hiprio/forget request virtqueue */ - callbacks[VQ_HIPRIO] = virtio_fs_vq_done; - virtio_fs_init_vq(&fs->vqs[VQ_HIPRIO], "hiprio", VQ_HIPRIO); - names[VQ_HIPRIO] = fs->vqs[VQ_HIPRIO].name; + callbacks[VQ_HIPRIO_IDX] = virtio_fs_vq_done; + virtio_fs_init_vq(&fs->vqs[VQ_HIPRIO_IDX], "hiprio", VQ_TYPE_HIPRIO); + names[VQ_HIPRIO_IDX] = fs->vqs[VQ_HIPRIO_IDX].name; /* Initialize the requests virtqueues */ for (i = fs->first_reqq_idx; i < fs->nvqs; i++) { @@ -704,7 +707,7 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev, snprintf(vq_name, VQ_NAME_LEN, "requests.%u", i - fs->first_reqq_idx); - virtio_fs_init_vq(&fs->vqs[i], vq_name, VQ_REQUEST); + virtio_fs_init_vq(&fs->vqs[i], vq_name, VQ_TYPE_REQUEST); callbacks[i] = virtio_fs_vq_done; names[i] = fs->vqs[i].name; } @@ -985,7 +988,7 @@ __releases(fiq->lock) unique = fuse_get_unique(fiq); fs = fiq->priv; - fsvq = &fs->vqs[VQ_HIPRIO]; + fsvq = &fs->vqs[VQ_HIPRIO_IDX]; spin_unlock(&fiq->lock); /* Allocate a buffer for the request */ @@ -1359,7 +1362,7 @@ static void virtio_fs_conn_destroy(struct fuse_mount *fm) { struct fuse_conn *fc = fm->fc; struct virtio_fs *vfs = fc->iq.priv; - struct virtio_fs_vq *fsvq = &vfs->vqs[VQ_HIPRIO]; + struct virtio_fs_vq *fsvq = &vfs->vqs[VQ_HIPRIO_IDX]; /* Stop dax worker. Soon evict_inodes() will be called which * will free all memory ranges belonging to all inodes. From patchwork Thu Sep 30 14:38:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 12528577 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6002C433EF for ; Thu, 30 Sep 2021 14:39:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AE31D61A05 for ; Thu, 30 Sep 2021 14:39:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350935AbhI3OlC (ORCPT ); Thu, 30 Sep 2021 10:41:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:53625 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1349383AbhI3Okz (ORCPT ); Thu, 30 Sep 2021 10:40:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633012752; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jiLS8XApW9VZIpvV4gJd0qiBAQUUcTvEoIZE1yWjx7c=; b=TB25m12jASBhrfB9h6rMrN/jYHGA6OvbJnMvnwIg9DJtLShqBKZfghi1ulIC6t4Uqs/50n g5q9MFWVlWzm+dSM394mPcCHE9mRxW3rnftAXBiYLzVAnM1FASk/9GCN8fKN6p/5zWlSBi EyeBgCRt1U5uhJA1oa1vulhBQ6s6GTA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-357-1JuoCIkFM-u3ErcFNkUlfw-1; Thu, 30 Sep 2021 10:39:10 -0400 X-MC-Unique: 1JuoCIkFM-u3ErcFNkUlfw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 38A568464D0; Thu, 30 Sep 2021 14:39:08 +0000 (UTC) Received: from horse.redhat.com (unknown [10.22.16.146]) by smtp.corp.redhat.com (Postfix) with ESMTP id DB92A5DF26; Thu, 30 Sep 2021 14:39:07 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id D242A228284; Thu, 30 Sep 2021 10:39:06 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtio-fs@redhat.com, miklos@szeredi.hu, stefanha@redhat.com Cc: vgoyal@redhat.com, iangelak@redhat.com, jaggel@bu.edu, dgilbert@redhat.com Subject: [PATCH 5/8] virtiofs: Add a virtqueue for notifications Date: Thu, 30 Sep 2021 10:38:47 -0400 Message-Id: <20210930143850.1188628-6-vgoyal@redhat.com> In-Reply-To: <20210930143850.1188628-1-vgoyal@redhat.com> References: <20210930143850.1188628-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Add a new virtqueue for notifications. This will allow the device to send notifications to guest. This queue is created only if the device supports it. This is negotiated using feature bit VIRTIO_FS_F_NOTIFICATION. Given the architecture of virtqueue, one needs to queue up pre-allocated elements in the notification queue and the device can pop these elements and fill the notification info and send it back. The size of the notification buffer is negotiable and is specified by the device through config space. This will allow us to add and support more notification types without having to change the spec. Signed-off-by: Vivek Goyal Signed-off-by: Ioannis Angelakopoulos --- fs/fuse/virtio_fs.c | 203 +++++++++++++++++++++++++++++++-- include/uapi/linux/virtio_fs.h | 5 + 2 files changed, 196 insertions(+), 12 deletions(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index eef9591de640..b70a22a79901 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -25,6 +25,7 @@ #define FUSE_HEADER_OVERHEAD 4 #define VQ_HIPRIO_IDX 0 +#define VQ_NOTIFY_IDX 1 /* List of virtio-fs device instances and a lock for the list. Also provides * mutual exclusion in device removal and mounting path @@ -34,10 +35,12 @@ static LIST_HEAD(virtio_fs_instances); enum { VQ_TYPE_HIPRIO, - VQ_TYPE_REQUEST + VQ_TYPE_REQUEST, + VQ_TYPE_NOTIFY }; #define VQ_NAME_LEN 24 +#define VQ_NOTIFY_ELEMS 16 /* Number of notification elements */ /* Per-virtqueue state */ struct virtio_fs_vq { @@ -46,6 +49,8 @@ struct virtio_fs_vq { struct work_struct done_work; struct list_head queued_reqs; struct list_head end_reqs; /* End these requests */ + struct virtio_fs_notify_node *notify_nodes; + struct list_head notify_reqs; /* List for queuing notify requests */ struct delayed_work dispatch_work; struct fuse_dev *fud; bool connected; @@ -64,6 +69,8 @@ struct virtio_fs { unsigned int num_request_queues; /* number of request queues */ struct dax_device *dax_dev; unsigned int first_reqq_idx; /* First request queue idx */ + bool notify_enabled; + unsigned int notify_buf_size; /* Size of notification buffer */ /* DAX memory window where file contents are mapped */ void *window_kaddr; @@ -91,6 +98,19 @@ struct virtio_fs_req_work { static int virtio_fs_enqueue_req(struct virtio_fs_vq *fsvq, struct fuse_req *req, bool in_flight); +/* Size of virtio_fs_notify specified by fs->notify_buf_size. */ +struct virtio_fs_notify { + struct fuse_out_header out_hdr; + char outarg[]; +}; + +struct virtio_fs_notify_node { + struct list_head list; + struct virtio_fs_notify notify; +}; + +static int virtio_fs_enqueue_all_notify(struct virtio_fs_vq *fsvq); + enum { OPT_DAX, }; @@ -136,6 +156,11 @@ static inline struct virtio_fs_vq *vq_to_fsvq(struct virtqueue *vq) return &fs->vqs[vq->index]; } +static inline struct virtio_fs *fsvq_to_fs(struct virtio_fs_vq *fsvq) +{ + return (struct virtio_fs *)fsvq->vq->vdev->priv; +} + /* Should be called with fsvq->lock held. */ static inline void inc_in_flight_req(struct virtio_fs_vq *fsvq) { @@ -151,10 +176,17 @@ static inline void dec_in_flight_req(struct virtio_fs_vq *fsvq) complete(&fsvq->in_flight_zero); } +static void virtio_fs_free_notify_nodes(struct virtio_fs *fs) +{ + if (fs->notify_enabled && fs->vqs) + kfree(fs->vqs[VQ_NOTIFY_IDX].notify_nodes); +} + static void release_virtio_fs_obj(struct kref *ref) { struct virtio_fs *vfs = container_of(ref, struct virtio_fs, refcount); + virtio_fs_free_notify_nodes(vfs); kfree(vfs->vqs); kfree(vfs); } @@ -201,6 +233,13 @@ static void virtio_fs_drain_all_queues_locked(struct virtio_fs *fs) int i; for (i = 0; i < fs->nvqs; i++) { + /* + * Can't wait to drain notification queue as it always + * had pending requests so that server can use those + * to send notifications + */ + if (fs->notify_enabled && (i == VQ_NOTIFY_IDX)) + continue; fsvq = &fs->vqs[i]; virtio_fs_drain_queue(fsvq); } @@ -229,6 +268,8 @@ static void virtio_fs_start_all_queues(struct virtio_fs *fs) spin_lock(&fsvq->lock); fsvq->connected = true; spin_unlock(&fsvq->lock); + if (fs->notify_enabled && (i == VQ_NOTIFY_IDX)) + virtio_fs_enqueue_all_notify(fsvq); } } @@ -477,6 +518,98 @@ static void virtio_fs_hiprio_dispatch_work(struct work_struct *work) } } +static int virtio_fs_init_notify_vq(struct virtio_fs *fs, + struct virtio_fs_vq *fsvq) +{ + struct virtio_fs_notify_node *notifyn; + unsigned int notify_node_sz = sizeof(struct list_head) + + fs->notify_buf_size; + int i; + + fsvq->notify_nodes = kcalloc(VQ_NOTIFY_ELEMS, notify_node_sz, + GFP_KERNEL); + if (!fsvq->notify_nodes) + return -ENOMEM; + + for (i = 0; i < VQ_NOTIFY_ELEMS; i++) { + notifyn = (void *)fsvq->notify_nodes + (i * notify_node_sz); + list_add_tail(¬ifyn->list, &fsvq->notify_reqs); + } + + return 0; +} + +static int virtio_fs_enqueue_all_notify(struct virtio_fs_vq *fsvq) +{ + struct scatterlist sg[1]; + int ret; + bool kick; + struct virtio_fs *fs = fsvq_to_fs(fsvq); + struct virtio_fs_notify_node *notifyn, *next; + unsigned int notify_sz; + + notify_sz = fs->notify_buf_size; + spin_lock(&fsvq->lock); + list_for_each_entry_safe(notifyn, next, &fsvq->notify_reqs, list) { + list_del_init(¬ifyn->list); + sg_init_one(sg, ¬ifyn->notify, notify_sz); + ret = virtqueue_add_inbuf(fsvq->vq, sg, 1, notifyn, GFP_ATOMIC); + if (ret) { + list_add_tail(¬ifyn->list, &fsvq->notify_reqs); + spin_unlock(&fsvq->lock); + return ret; + } + inc_in_flight_req(fsvq); + } + + kick = virtqueue_kick_prepare(fsvq->vq); + spin_unlock(&fsvq->lock); + if (kick) + virtqueue_notify(fsvq->vq); + return 0; +} + +static void virtio_fs_notify_done_work(struct work_struct *work) +{ + struct virtio_fs_vq *fsvq = container_of(work, struct virtio_fs_vq, + done_work); + struct virtqueue *vq = fsvq->vq; + LIST_HEAD(reqs); + struct virtio_fs_notify_node *notifyn, *next; + + spin_lock(&fsvq->lock); + do { + unsigned int len; + + virtqueue_disable_cb(vq); + + while ((notifyn = virtqueue_get_buf(vq, &len)) != NULL) + list_add_tail(¬ifyn->list, &reqs); + + } while (!virtqueue_enable_cb(vq) && likely(!virtqueue_is_broken(vq))); + spin_unlock(&fsvq->lock); + + /* Process notify */ + list_for_each_entry_safe(notifyn, next, &reqs, list) { + spin_lock(&fsvq->lock); + dec_in_flight_req(fsvq); + list_del_init(¬ifyn->list); + list_add_tail(¬ifyn->list, &fsvq->notify_reqs); + spin_unlock(&fsvq->lock); + } + + /* + * If queue is connected, queue notifications again. If not, + * these will be queued again when virtuqueue is restarted. + */ + if (fsvq->connected) + virtio_fs_enqueue_all_notify(fsvq); +} + +static void virtio_fs_notify_dispatch_work(struct work_struct *work) +{ +} + /* Allocate and copy args into req->argbuf */ static int copy_args_to_argbuf(struct fuse_req *req) { @@ -644,24 +777,34 @@ static void virtio_fs_vq_done(struct virtqueue *vq) schedule_work(&fsvq->done_work); } -static void virtio_fs_init_vq(struct virtio_fs_vq *fsvq, char *name, - int vq_type) +static int virtio_fs_init_vq(struct virtio_fs *fs, struct virtio_fs_vq *fsvq, + char *name, int vq_type) { + int ret = 0; + strncpy(fsvq->name, name, VQ_NAME_LEN); spin_lock_init(&fsvq->lock); INIT_LIST_HEAD(&fsvq->queued_reqs); INIT_LIST_HEAD(&fsvq->end_reqs); + INIT_LIST_HEAD(&fsvq->notify_reqs); init_completion(&fsvq->in_flight_zero); if (vq_type == VQ_TYPE_REQUEST) { INIT_WORK(&fsvq->done_work, virtio_fs_requests_done_work); INIT_DELAYED_WORK(&fsvq->dispatch_work, virtio_fs_request_dispatch_work); + } else if (vq_type == VQ_TYPE_NOTIFY) { + INIT_WORK(&fsvq->done_work, virtio_fs_notify_done_work); + INIT_DELAYED_WORK(&fsvq->dispatch_work, + virtio_fs_notify_dispatch_work); + ret = virtio_fs_init_notify_vq(fs, fsvq); } else { INIT_WORK(&fsvq->done_work, virtio_fs_hiprio_done_work); INIT_DELAYED_WORK(&fsvq->dispatch_work, virtio_fs_hiprio_dispatch_work); } + + return ret; } /* Initialize virtqueues */ @@ -679,9 +822,28 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev, if (fs->num_request_queues == 0) return -EINVAL; - /* One hiprio queue and rest are request queues */ - fs->nvqs = 1 + fs->num_request_queues; - fs->first_reqq_idx = 1; + if (virtio_has_feature(vdev, VIRTIO_FS_F_NOTIFICATION)) { + fs->notify_enabled = true; + virtio_cread(vdev, struct virtio_fs_config, notify_buf_size, + &fs->notify_buf_size); + if (fs->notify_buf_size <= sizeof(struct fuse_out_header)) { + pr_err("virtio-fs: Invalid value %d of notification buffer size\n", + fs->notify_buf_size); + return -EINVAL; + } + pr_info("virtio-fs: device supports notification. Notification_buf_size=%u\n", + fs->notify_buf_size); + } + + if (fs->notify_enabled) { + /* One additional queue for hiprio and one for notifications */ + fs->nvqs = 2 + fs->num_request_queues; + fs->first_reqq_idx = VQ_NOTIFY_IDX + 1; + } else { + fs->nvqs = 1 + fs->num_request_queues; + fs->first_reqq_idx = 1; + } + fs->vqs = kcalloc(fs->nvqs, sizeof(fs->vqs[VQ_HIPRIO_IDX]), GFP_KERNEL); if (!fs->vqs) return -ENOMEM; @@ -698,16 +860,32 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev, /* Initialize the hiprio/forget request virtqueue */ callbacks[VQ_HIPRIO_IDX] = virtio_fs_vq_done; - virtio_fs_init_vq(&fs->vqs[VQ_HIPRIO_IDX], "hiprio", VQ_TYPE_HIPRIO); + ret = virtio_fs_init_vq(fs, &fs->vqs[VQ_HIPRIO_IDX], "hiprio", + VQ_TYPE_HIPRIO); + if (ret < 0) + goto out; names[VQ_HIPRIO_IDX] = fs->vqs[VQ_HIPRIO_IDX].name; + /* Initialize notification queue */ + if (fs->notify_enabled) { + callbacks[VQ_NOTIFY_IDX] = virtio_fs_vq_done; + ret = virtio_fs_init_vq(fs, &fs->vqs[VQ_NOTIFY_IDX], + "notification", VQ_TYPE_NOTIFY); + if (ret < 0) + goto out; + names[VQ_NOTIFY_IDX] = fs->vqs[VQ_NOTIFY_IDX].name; + } + /* Initialize the requests virtqueues */ for (i = fs->first_reqq_idx; i < fs->nvqs; i++) { char vq_name[VQ_NAME_LEN]; snprintf(vq_name, VQ_NAME_LEN, "requests.%u", i - fs->first_reqq_idx); - virtio_fs_init_vq(&fs->vqs[i], vq_name, VQ_TYPE_REQUEST); + ret = virtio_fs_init_vq(fs, &fs->vqs[i], vq_name, + VQ_TYPE_REQUEST); + if (ret < 0) + goto out; callbacks[i] = virtio_fs_vq_done; names[i] = fs->vqs[i].name; } @@ -718,14 +896,14 @@ static int virtio_fs_setup_vqs(struct virtio_device *vdev, for (i = 0; i < fs->nvqs; i++) fs->vqs[i].vq = vqs[i]; - - virtio_fs_start_all_queues(fs); out: kfree(names); kfree(callbacks); kfree(vqs); - if (ret) + if (ret) { + virtio_fs_free_notify_nodes(fs); kfree(fs->vqs); + } return ret; } @@ -889,6 +1067,7 @@ static int virtio_fs_probe(struct virtio_device *vdev) * requests need to be sent before we return. */ virtio_device_ready(vdev); + virtio_fs_start_all_queues(fs); ret = virtio_fs_add_instance(fs); if (ret < 0) @@ -958,7 +1137,7 @@ static const struct virtio_device_id id_table[] = { {}, }; -static const unsigned int feature_table[] = {}; +static const unsigned int feature_table[] = {VIRTIO_FS_F_NOTIFICATION}; static struct virtio_driver virtio_fs_driver = { .driver.name = KBUILD_MODNAME, diff --git a/include/uapi/linux/virtio_fs.h b/include/uapi/linux/virtio_fs.h index bea38291421b..3a9bbccf4115 100644 --- a/include/uapi/linux/virtio_fs.h +++ b/include/uapi/linux/virtio_fs.h @@ -8,12 +8,17 @@ #include #include +/* Feature bits */ +#define VIRTIO_FS_F_NOTIFICATION 0 /* Notification queue supported */ + struct virtio_fs_config { /* Filesystem name (UTF-8, not NUL-terminated, padded with NULs) */ __u8 tag[36]; /* Number of request queues */ __le32 num_request_queues; + /* Size of notification buffer */ + __u32 notify_buf_size; } __attribute__((packed)); /* For the id field in virtio_pci_shm_cap */ From patchwork Thu Sep 30 14:38:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 12528579 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA1AFC433EF for ; Thu, 30 Sep 2021 14:39:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D58D361A05 for ; Thu, 30 Sep 2021 14:39:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349383AbhI3OlQ (ORCPT ); Thu, 30 Sep 2021 10:41:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:37161 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350747AbhI3OlO (ORCPT ); Thu, 30 Sep 2021 10:41:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633012772; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wvqSbl75duLv3ADG85bHjzf/xYZ44FjWfaebFV5SmIg=; b=F9bm8lsO1EreMLD37oMNYTjDRZSLbWMN2tlEDKIbDo2aUIjOJZxN/6p9WgkbclPLKbHFbR QZpOPHFCGUkBqD3LHKV318/4NDTf+DpxBfmGwQkoXMMvSctBL8jeTP5mqcacFKdVUhDKgX FkObjfvR2Ah8Ds+9IoVHV5jLI2o0fzI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-77-TVxHZ3i4MHujIPRaIurg2w-1; Thu, 30 Sep 2021 10:39:28 -0400 X-MC-Unique: TVxHZ3i4MHujIPRaIurg2w-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 88169824FB0; Thu, 30 Sep 2021 14:39:27 +0000 (UTC) Received: from horse.redhat.com (unknown [10.22.16.146]) by smtp.corp.redhat.com (Postfix) with ESMTP id CCED9100AE2C; Thu, 30 Sep 2021 14:39:07 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id D6354228285; Thu, 30 Sep 2021 10:39:06 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtio-fs@redhat.com, miklos@szeredi.hu, stefanha@redhat.com Cc: vgoyal@redhat.com, iangelak@redhat.com, jaggel@bu.edu, dgilbert@redhat.com Subject: [PATCH 6/8] virtiofs: Add a helper to end request and decrement inflight number Date: Thu, 30 Sep 2021 10:38:48 -0400 Message-Id: <20210930143850.1188628-7-vgoyal@redhat.com> In-Reply-To: <20210930143850.1188628-1-vgoyal@redhat.com> References: <20210930143850.1188628-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Add a helper function to end fuse request and decrement number of inflight requests. This pattern is already used at two places and I am planning to use it two more times in later patches. Adding a helper reduces number of lines of code and improves readability. Signed-off-by: Vivek Goyal --- fs/fuse/virtio_fs.c | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index b70a22a79901..8d33879d62fb 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -380,6 +380,15 @@ static void virtio_fs_hiprio_done_work(struct work_struct *work) spin_unlock(&fsvq->lock); } +static void end_req_dec_in_flight(struct fuse_req *req, + struct virtio_fs_vq *fsvq) +{ + fuse_request_end(req); + spin_lock(&fsvq->lock); + dec_in_flight_req(fsvq); + spin_unlock(&fsvq->lock); +} + static void virtio_fs_request_dispatch_work(struct work_struct *work) { struct fuse_req *req; @@ -425,12 +434,9 @@ static void virtio_fs_request_dispatch_work(struct work_struct *work) return; } req->out.h.error = ret; - spin_lock(&fsvq->lock); - dec_in_flight_req(fsvq); - spin_unlock(&fsvq->lock); pr_err("virtio-fs: virtio_fs_enqueue_req() failed %d\n", ret); - fuse_request_end(req); + end_req_dec_in_flight(req, fsvq); } } } @@ -709,10 +715,7 @@ static void virtio_fs_request_complete(struct fuse_req *req, } } - fuse_request_end(req); - spin_lock(&fsvq->lock); - dec_in_flight_req(fsvq); - spin_unlock(&fsvq->lock); + end_req_dec_in_flight(req, fsvq); } static void virtio_fs_complete_req_work(struct work_struct *work) From patchwork Thu Sep 30 14:38:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 12528575 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10591C433F5 for ; Thu, 30 Sep 2021 14:39:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E8AA561A05 for ; Thu, 30 Sep 2021 14:39:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350883AbhI3OlB (ORCPT ); Thu, 30 Sep 2021 10:41:01 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:39871 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350747AbhI3Okz (ORCPT ); Thu, 30 Sep 2021 10:40:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633012752; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nKHXLIf3Jq2uyWxFlT1LB9ypzk2D5+Zf6I6nd7kOZKg=; b=aM8Aud4kl6OMhokyefjAMSi61OLsJKKvxL9vx6Vnv5swO+GEAoYak5myG7a6mjef6OBQ1W lnGEU474DIg8irHWbp3WQVVymQcju13lpqcY+X2sgxbl5CkPULMZUwOBsnzDqHDB4ZcHVT ZOVhfAMxSBSEYi4Ym5G0JfeMKWJgSOQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-495-OLjrOzmePVCCnv8lGe0zjw-1; Thu, 30 Sep 2021 10:39:10 -0400 X-MC-Unique: OLjrOzmePVCCnv8lGe0zjw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1D80E101AFBD; Thu, 30 Sep 2021 14:39:08 +0000 (UTC) Received: from horse.redhat.com (unknown [10.22.16.146]) by smtp.corp.redhat.com (Postfix) with ESMTP id D90F25F4E2; Thu, 30 Sep 2021 14:39:07 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id DB5AE228286; Thu, 30 Sep 2021 10:39:06 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtio-fs@redhat.com, miklos@szeredi.hu, stefanha@redhat.com Cc: vgoyal@redhat.com, iangelak@redhat.com, jaggel@bu.edu, dgilbert@redhat.com Subject: [PATCH 7/8] virtiofs: Add new notification type FUSE_NOTIFY_LOCK Date: Thu, 30 Sep 2021 10:38:49 -0400 Message-Id: <20210930143850.1188628-8-vgoyal@redhat.com> In-Reply-To: <20210930143850.1188628-1-vgoyal@redhat.com> References: <20210930143850.1188628-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org Add a new notification type FUSE_NOTIFY_LOCK. This notification can be sent by file server to signifiy that a previous locking request has completed and actual caller should be woken up. As of now we don't support blocking variant of posix locks and daemon returns -EOPNOTSUPP. Reason being that it can lead to deadlocks. Virtqueue size is limited and it is possible we fill virtqueue with all the requests of fcntl(F_SETLKW) and wait for reply. And later a subsequent unlock request can't make progress because virtqueue is full. And that means F_SETLKW can't make progress and we are deadlocked. This problem is not limited to posix locks only. I think blocking remote flock and open file description locks should face the same issue. Right now fuse does not support open file description locks, so its not a problem. But fuse/virtiofs does support remote flock and they can use same mechanism too. Use notification queue to solve this problem. After submitting lock request device will send a reply asking requester to wait. Once lock is available, requester will get a notification saying lock is available. That way we don't keep the request virtueue busy while we are waiting for lock and further unlock requests can make progress. When we get a reply in response to lock request, we need a way to know if we need to wait for notification or not. I have overloaded the fuse_out_header->error field. If value is ->error is 1, that's a signal to caller to wait for lock notification. Overloading ->error in this way is not the best way to do it. But I am running out of ideas. Signed-off-by: Vivek Goyal Signed-off-by: Ioannis Angelakopoulos --- fs/fuse/virtio_fs.c | 73 +++++++++++++++++++++++++++++++++++++++ include/uapi/linux/fuse.h | 11 +++++- 2 files changed, 83 insertions(+), 1 deletion(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index 8d33879d62fb..1634ea2d0555 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -48,6 +48,7 @@ struct virtio_fs_vq { struct virtqueue *vq; /* protected by ->lock */ struct work_struct done_work; struct list_head queued_reqs; + struct list_head wait_reqs; /* Requests waiting for notification */ struct list_head end_reqs; /* End these requests */ struct virtio_fs_notify_node *notify_nodes; struct list_head notify_reqs; /* List for queuing notify requests */ @@ -575,13 +576,72 @@ static int virtio_fs_enqueue_all_notify(struct virtio_fs_vq *fsvq) return 0; } +static int notify_complete_waiting_req(struct virtio_fs *vfs, + struct fuse_notify_lock_out *out_args) +{ + /* TODO: Handle multiqueue */ + struct virtio_fs_vq *fsvq = &vfs->vqs[vfs->first_reqq_idx]; + struct fuse_req *req, *next; + bool found = false; + + /* Find waiting request with the unique number and end it */ + spin_lock(&fsvq->lock); + list_for_each_entry_safe(req, next, &fsvq->wait_reqs, list) { + if (req->in.h.unique == out_args->unique) { + list_del_init(&req->list); + clear_bit(FR_SENT, &req->flags); + /* Transfer error code from notify */ + req->out.h.error = out_args->error; + found = true; + break; + } + } + spin_unlock(&fsvq->lock); + + /* + * TODO: It is possible that some re-ordering happens in notify + * comes before request is complete. Deal with it. + */ + if (found) { + end_req_dec_in_flight(req, fsvq); + } else + pr_debug("virtio-fs: Did not find waiting request with unique=0x%llx\n", + out_args->unique); + + return 0; +} + +static int virtio_fs_handle_notify(struct virtio_fs *vfs, + struct virtio_fs_notify *notify) +{ + int ret = 0; + struct fuse_out_header *oh = ¬ify->out_hdr; + struct fuse_notify_lock_out *lo; + + /* + * For notifications, oh.unique is 0 and oh->error contains code + * for which notification as arrived. + */ + switch (oh->error) { + case FUSE_NOTIFY_LOCK: + lo = (struct fuse_notify_lock_out *) ¬ify->outarg; + notify_complete_waiting_req(vfs, lo); + break; + default: + pr_err("virtio-fs: Unexpected notification %d\n", oh->error); + } + return ret; +} + static void virtio_fs_notify_done_work(struct work_struct *work) { struct virtio_fs_vq *fsvq = container_of(work, struct virtio_fs_vq, done_work); struct virtqueue *vq = fsvq->vq; + struct virtio_fs *vfs = vq->vdev->priv; LIST_HEAD(reqs); struct virtio_fs_notify_node *notifyn, *next; + struct fuse_out_header *oh; spin_lock(&fsvq->lock); do { @@ -597,6 +657,10 @@ static void virtio_fs_notify_done_work(struct work_struct *work) /* Process notify */ list_for_each_entry_safe(notifyn, next, &reqs, list) { + oh = ¬ifyn->notify.out_hdr; + WARN_ON(oh->unique); + /* Handle notification */ + virtio_fs_handle_notify(vfs, ¬ifyn->notify); spin_lock(&fsvq->lock); dec_in_flight_req(fsvq); list_del_init(¬ifyn->list); @@ -696,6 +760,14 @@ static void virtio_fs_request_complete(struct fuse_req *req, * TODO verify that server properly follows FUSE protocol * (oh.uniq, oh.len) */ + if (req->out.h.error == 1) { + /* Wait for notification to complete request */ + spin_lock(&fsvq->lock); + list_add_tail(&req->list, &fsvq->wait_reqs); + spin_unlock(&fsvq->lock); + return; + } + args = req->args; copy_args_from_argbuf(args, req); @@ -788,6 +860,7 @@ static int virtio_fs_init_vq(struct virtio_fs *fs, struct virtio_fs_vq *fsvq, strncpy(fsvq->name, name, VQ_NAME_LEN); spin_lock_init(&fsvq->lock); INIT_LIST_HEAD(&fsvq->queued_reqs); + INIT_LIST_HEAD(&fsvq->wait_reqs); INIT_LIST_HEAD(&fsvq->end_reqs); INIT_LIST_HEAD(&fsvq->notify_reqs); init_completion(&fsvq->in_flight_zero); diff --git a/include/uapi/linux/fuse.h b/include/uapi/linux/fuse.h index 36ed092227fa..46838551ea84 100644 --- a/include/uapi/linux/fuse.h +++ b/include/uapi/linux/fuse.h @@ -184,6 +184,8 @@ * * 7.34 * - add FUSE_SYNCFS + * 7.35 + * - add FUSE_NOTIFY_LOCK */ #ifndef _LINUX_FUSE_H @@ -219,7 +221,7 @@ #define FUSE_KERNEL_VERSION 7 /** Minor version number of this interface */ -#define FUSE_KERNEL_MINOR_VERSION 34 +#define FUSE_KERNEL_MINOR_VERSION 35 /** The node ID of the root inode */ #define FUSE_ROOT_ID 1 @@ -529,6 +531,7 @@ enum fuse_notify_code { FUSE_NOTIFY_STORE = 4, FUSE_NOTIFY_RETRIEVE = 5, FUSE_NOTIFY_DELETE = 6, + FUSE_NOTIFY_LOCK = 7, FUSE_NOTIFY_CODE_MAX, }; @@ -920,6 +923,12 @@ struct fuse_notify_retrieve_in { uint64_t dummy4; }; +struct fuse_notify_lock_out { + uint64_t unique; + int32_t error; + int32_t padding; +}; + /* Device ioctls: */ #define FUSE_DEV_IOC_MAGIC 229 #define FUSE_DEV_IOC_CLONE _IOR(FUSE_DEV_IOC_MAGIC, 0, uint32_t) From patchwork Thu Sep 30 14:38:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vivek Goyal X-Patchwork-Id: 12528573 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0EEB0C433EF for ; Thu, 30 Sep 2021 14:39:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ED94161A08 for ; Thu, 30 Sep 2021 14:39:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350941AbhI3OlA (ORCPT ); Thu, 30 Sep 2021 10:41:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:58891 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350885AbhI3Oky (ORCPT ); Thu, 30 Sep 2021 10:40:54 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633012751; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=85YKaekpqa4GzE6JJfO31w2ZdTF4VUZmgvqMvc0z83A=; b=XTfaWjW8gMIZuiFZv/IZVnA4VwW+A5W5QUYO0qfT3JZYIavE1MKsOrDEY/aJjO+jaPZeQ5 FVMX/BfaIdWbtNK1rEvkBiAwHELAHD++BGMhIGsCbmHabtKnbfzgLIRBwoxCnpgjEbfmXq aa1T2Wv7CJdAVgHcZOBjXVOm+kvyHMo= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-33-HeAQVmKePcG_Zx7npk-dbQ-1; Thu, 30 Sep 2021 10:39:09 -0400 X-MC-Unique: HeAQVmKePcG_Zx7npk-dbQ-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2234B1882FB1; Thu, 30 Sep 2021 14:39:08 +0000 (UTC) Received: from horse.redhat.com (unknown [10.22.16.146]) by smtp.corp.redhat.com (Postfix) with ESMTP id D11ED60BD8; Thu, 30 Sep 2021 14:39:07 +0000 (UTC) Received: by horse.redhat.com (Postfix, from userid 10451) id E0875228287; Thu, 30 Sep 2021 10:39:06 -0400 (EDT) From: Vivek Goyal To: linux-fsdevel@vger.kernel.org, virtio-fs@redhat.com, miklos@szeredi.hu, stefanha@redhat.com Cc: vgoyal@redhat.com, iangelak@redhat.com, jaggel@bu.edu, dgilbert@redhat.com Subject: [PATCH 8/8] virtiofs: Handle reordering of reply and notification event Date: Thu, 30 Sep 2021 10:38:50 -0400 Message-Id: <20210930143850.1188628-9-vgoyal@redhat.com> In-Reply-To: <20210930143850.1188628-1-vgoyal@redhat.com> References: <20210930143850.1188628-1-vgoyal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org So far we are relying that for a locking requrest reply always comes first and then notification comes in. Both of these travel on different virtqueues and there is no guarantee that client will always see/process them in this order. Though it is a very unlikely event, but it is possible that notification comes first and then reply to lock request comes. Hence take care of this possibility. If notification arrives and does not find a corresponding request waiting, queue up notification in a list. When request reply will come, it will check if corresponding notification has already arrived. If yes, it will finish the request otherwise wait for notification to arrive. Signed-off-by: Vivek Goyal --- fs/fuse/virtio_fs.c | 134 ++++++++++++++++++++++++++++++++++++++------ 1 file changed, 116 insertions(+), 18 deletions(-) diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index 1634ea2d0555..028ca9198bc4 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -42,6 +42,17 @@ enum { #define VQ_NAME_LEN 24 #define VQ_NOTIFY_ELEMS 16 /* Number of notification elements */ +/* + * virtio_fs_vq->lock spinlock nesting subclasses: + * + * 0: normal + * 1: nested + */ +enum { + VQ_LOCK_NORMAL, + VQ_LOCK_NESTED +}; + /* Per-virtqueue state */ struct virtio_fs_vq { spinlock_t lock; @@ -52,6 +63,8 @@ struct virtio_fs_vq { struct list_head end_reqs; /* End these requests */ struct virtio_fs_notify_node *notify_nodes; struct list_head notify_reqs; /* List for queuing notify requests */ + /* Notifications to be processed when request reply arrives. */ + struct list_head wait_notify; struct delayed_work dispatch_work; struct fuse_dev *fud; bool connected; @@ -576,13 +589,32 @@ static int virtio_fs_enqueue_all_notify(struct virtio_fs_vq *fsvq) return 0; } +/* + * Find a waiting process/request and complete it. If request and + * notification got re-ordered, it is possible that there is no + * waiting request yet. In that case queue up notification and + * return 1 so that caller does not try to reuse notifiy node yet. + * + * Return: + * negative value : in case of error + * 0 : If request processed successfully + * 1 : Could not find waiting processed. Request got queued. + * Do not reuse this notify node yet. + */ static int notify_complete_waiting_req(struct virtio_fs *vfs, struct fuse_notify_lock_out *out_args) { /* TODO: Handle multiqueue */ struct virtio_fs_vq *fsvq = &vfs->vqs[vfs->first_reqq_idx]; + struct virtio_fs_vq *notify_fsvq = &vfs->vqs[VQ_NOTIFY_IDX]; struct fuse_req *req, *next; + struct virtio_fs_notify *notify; + struct virtio_fs_notify_node *notifyn; bool found = false; + int ret = 0; + + notify = container_of((void *)out_args, struct virtio_fs_notify, outarg); + notifyn = container_of(notify, struct virtio_fs_notify_node, notify); /* Find waiting request with the unique number and end it */ spin_lock(&fsvq->lock); @@ -596,25 +628,39 @@ static int notify_complete_waiting_req(struct virtio_fs *vfs, break; } } - spin_unlock(&fsvq->lock); - /* - * TODO: It is possible that some re-ordering happens in notify - * comes before request is complete. Deal with it. + * It probably is a rare case of re-ordering where notification + * arrived/processed before reply. Queue up the notification. */ + spin_lock_nested(¬ify_fsvq->lock, VQ_LOCK_NESTED); + if (!found) { + list_add_tail(¬ifyn->list, ¬ify_fsvq->wait_notify); + ret = 1; + } + spin_unlock(¬ify_fsvq->lock); + spin_unlock(&fsvq->lock); + if (found) { end_req_dec_in_flight(req, fsvq); - } else - pr_debug("virtio-fs: Did not find waiting request with unique=0x%llx\n", - out_args->unique); + } - return 0; + return ret; +} + +static void notify_node_reuse(struct virtio_fs_vq *notify_fsvq, + struct virtio_fs_notify_node *notifyn) +{ + spin_lock(¬ify_fsvq->lock); + list_add_tail(¬ifyn->list, ¬ify_fsvq->notify_reqs); + spin_unlock(¬ify_fsvq->lock); } static int virtio_fs_handle_notify(struct virtio_fs *vfs, - struct virtio_fs_notify *notify) + struct virtio_fs_notify_node *notifyn) { - int ret = 0; + int ret = 0, no_reuse = 0; + struct virtio_fs_notify *notify = ¬ifyn->notify; + struct virtio_fs_vq *notify_fsvq = &vfs->vqs[VQ_NOTIFY_IDX]; struct fuse_out_header *oh = ¬ify->out_hdr; struct fuse_notify_lock_out *lo; @@ -625,11 +671,15 @@ static int virtio_fs_handle_notify(struct virtio_fs *vfs, switch (oh->error) { case FUSE_NOTIFY_LOCK: lo = (struct fuse_notify_lock_out *) ¬ify->outarg; - notify_complete_waiting_req(vfs, lo); + no_reuse = notify_complete_waiting_req(vfs, lo); break; default: pr_err("virtio-fs: Unexpected notification %d\n", oh->error); } + + if (!no_reuse) + notify_node_reuse(notify_fsvq, notifyn); + return ret; } @@ -659,12 +709,11 @@ static void virtio_fs_notify_done_work(struct work_struct *work) list_for_each_entry_safe(notifyn, next, &reqs, list) { oh = ¬ifyn->notify.out_hdr; WARN_ON(oh->unique); + list_del_init(¬ifyn->list); /* Handle notification */ - virtio_fs_handle_notify(vfs, ¬ifyn->notify); + virtio_fs_handle_notify(vfs, notifyn); spin_lock(&fsvq->lock); dec_in_flight_req(fsvq); - list_del_init(¬ifyn->list); - list_add_tail(¬ifyn->list, &fsvq->notify_reqs); spin_unlock(&fsvq->lock); } @@ -747,6 +796,57 @@ static void copy_args_from_argbuf(struct fuse_args *args, struct fuse_req *req) req->argbuf = NULL; } +static void virtio_fs_wait_or_complete_request(struct virtio_fs_vq *fsvq, + struct fuse_req *req) +{ + struct virtqueue *vq = fsvq->vq; + struct virtio_fs *vfs = vq->vdev->priv; + struct virtio_fs_vq *notify_fsvq = &vfs->vqs[VQ_NOTIFY_IDX]; + bool found = false; + struct virtio_fs_notify_node *notifyn, *next; + + /* + * Note, notification queue lock nests inside request queue + * lock and not otherwise. + */ + spin_lock(&fsvq->lock); + spin_lock_nested(¬ify_fsvq->lock, VQ_LOCK_NESTED); + /* + * Check if there is already a notification event in case of + * reply and notification event got re-ordered. Very unlikley. + */ + list_for_each_entry_safe(notifyn, next, ¬ify_fsvq->wait_notify, + list) { + struct fuse_notify_lock_out *lo; + + lo = (struct fuse_notify_lock_out *) ¬ifyn->notify.outarg; + + if (req->in.h.unique == lo->unique) { + list_del_init(¬ifyn->list); + clear_bit(FR_SENT, &req->flags); + /* Transfer error code from notify */ + req->out.h.error = lo->error; + found = true; + break; + } + } + + if (!found) { + /* Wait for notification to arrive. */ + list_add_tail(&req->list, &fsvq->wait_reqs); + } + + spin_unlock(¬ify_fsvq->lock); + spin_unlock(&fsvq->lock); + + if (found) { + end_req_dec_in_flight(req, fsvq); + notify_node_reuse(notify_fsvq, notifyn); + if (fsvq->connected) + virtio_fs_enqueue_all_notify(fsvq); + } +} + /* Work function for request completion */ static void virtio_fs_request_complete(struct fuse_req *req, struct virtio_fs_vq *fsvq) @@ -762,10 +862,7 @@ static void virtio_fs_request_complete(struct fuse_req *req, */ if (req->out.h.error == 1) { /* Wait for notification to complete request */ - spin_lock(&fsvq->lock); - list_add_tail(&req->list, &fsvq->wait_reqs); - spin_unlock(&fsvq->lock); - return; + return virtio_fs_wait_or_complete_request(fsvq, req); } args = req->args; @@ -863,6 +960,7 @@ static int virtio_fs_init_vq(struct virtio_fs *fs, struct virtio_fs_vq *fsvq, INIT_LIST_HEAD(&fsvq->wait_reqs); INIT_LIST_HEAD(&fsvq->end_reqs); INIT_LIST_HEAD(&fsvq->notify_reqs); + INIT_LIST_HEAD(&fsvq->wait_notify); init_completion(&fsvq->in_flight_zero); if (vq_type == VQ_TYPE_REQUEST) {