From patchwork Fri Apr 15 06:41:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: zhenwei pi X-Patchwork-Id: 12814621 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1A83C433F5 for ; Fri, 15 Apr 2022 06:45:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350665AbiDOGru (ORCPT ); Fri, 15 Apr 2022 02:47:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59444 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350656AbiDOGrr (ORCPT ); Fri, 15 Apr 2022 02:47:47 -0400 Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2602698F7D for ; Thu, 14 Apr 2022 23:45:19 -0700 (PDT) Received: by mail-pf1-x42b.google.com with SMTP id bo5so6672435pfb.4 for ; Thu, 14 Apr 2022 23:45:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hK0o+XIz0DMOZNeVzUU8qM3p2W7xQ1P7hXtT21YfSPQ=; b=B2OOmJogkpAkdVB0bcv4kA+up1RUJ5TQH3sLlkHyUcBtlw+A1OkhGjRYp/mdoywQ+T wlrscCKLMyCnduFe64OmZgpckIu12bD4JBG0DwIim+DsRerF2lT+GShm2ae4WszVrkwn gxIXz+rv6ImrYNjqbM9G5jSOMLnrjVA73OpKXbxpKNtX2ZrzlvQKLZoEy9znehEtaxVc kqZ+pOg5L3XuIUWSn7N/QYThYj2VDlxVfPXgUo9MYaT2YG4NeVb/ZqdX7FXbUQ1d7s3w B2uSHXRZz8ObyWyDd0W7VvU/EA+B5Y+thyUsG9OMOlMdCe5sD8aHcB5gAt3GRtF2PmCJ iv4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hK0o+XIz0DMOZNeVzUU8qM3p2W7xQ1P7hXtT21YfSPQ=; b=QapKNwprYP9p5J+zIGg24EVHebCm5+w5jWc8BN28YaxgE0J4ZvpgK0gwtdH9IidAX3 2lw+WmYLKldPHlVnaAULkVTUTzcXxIIrijjpd0QOfz3Z9RZyrAHfzQc5gjoF0KAsFI/S ESUxujyB2Dcvvx2kas569VC6Yj4jy7xAB+yosIRexoG2ll2iaVx6+I3EQU0YyEn1dDiT zrjd0MwaumfkEukEumzxs4zSTNNNIn3C7G/iKH34VCDb9BfiycC+yM6Je5rSwja/ejqB CP+T3bRxcJIyCRDDlkUmRwpByimGm2HqQXWeIMjDJx224N11FlqwQvttCd/5hjgvpHwL XzoA== X-Gm-Message-State: AOAM533WkJZAwNvb+iOs8PxBJJpnNkAeK2mCAGnWk9IeCXyHANYYsUvW 4tHMA7RDm9pe72AnFa9bmPBW8Q== X-Google-Smtp-Source: ABdhPJxs1C+d3qMPQoDOE4V6RxeBLdiUIjkXcVIUgBnrqUibdRdkhEKulih25Cz7JbkX5brB3POEgQ== X-Received: by 2002:a63:a01:0:b0:39d:ac8f:d24f with SMTP id 1-20020a630a01000000b0039dac8fd24fmr5226913pgk.610.1650005118529; Thu, 14 Apr 2022 23:45:18 -0700 (PDT) Received: from always-x1.bytedance.net ([61.120.150.70]) by smtp.gmail.com with ESMTPSA id d8-20020a056a00198800b004fab740dbe6sm1867385pfl.15.2022.04.14.23.45.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 23:45:17 -0700 (PDT) From: zhenwei pi To: arei.gonglei@huawei.com, mst@redhat.com Cc: jasowang@redhat.com, herbert@gondor.apana.org.au, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-crypto@vger.kernel.org, helei.sig11@bytedance.com, davem@davemloft.net, zhenwei pi Subject: [PATCH 1/4] virtio-crypto: wait ctrl queue instead of busy polling Date: Fri, 15 Apr 2022 14:41:33 +0800 Message-Id: <20220415064136.304661-2-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220415064136.304661-1-pizhenwei@bytedance.com> References: <20220415064136.304661-1-pizhenwei@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Originally, after submitting request into virtio crypto control queue, the guest side polls the result from the virt queue. There are two problems: 1, The queue depth is always 1, the performance of a virtio crypto device gets limited. Multi user processes share a single control queue, and hit spin lock race from control queue. Test on Intel Platinum 8260, a single worker gets ~35K/s create/close session operations, and 8 workers get ~40K/s operations with 800% CPU utilization. 2, The control request is supposed to get handled immediately, but in the current implementation of QEMU(v6.2), the vCPU thread kicks another thread to do this work, the latency also gets unstable. Tracking latency of virtio_crypto_alg_akcipher_close_session in 5s: usecs : count distribution 0 -> 1 : 0 | | 2 -> 3 : 7 | | 4 -> 7 : 72 | | 8 -> 15 : 186485 |************************| 16 -> 31 : 687 | | 32 -> 63 : 5 | | 64 -> 127 : 3 | | 128 -> 255 : 1 | | 256 -> 511 : 0 | | 512 -> 1023 : 0 | | 1024 -> 2047 : 0 | | 2048 -> 4095 : 0 | | 4096 -> 8191 : 0 | | 8192 -> 16383 : 2 | | To improve the performance of control queue, wait completion instead of busy polling without lock race, and get completed by control queue callback. Test this patch, the guest side get ~200K/s operations with 300% CPU utilization. Signed-off-by: zhenwei pi --- drivers/crypto/virtio/Makefile | 1 + .../virtio/virtio_crypto_akcipher_algs.c | 87 ++++++------ drivers/crypto/virtio/virtio_crypto_common.c | 61 ++++++++ drivers/crypto/virtio/virtio_crypto_common.h | 23 ++- drivers/crypto/virtio/virtio_crypto_core.c | 2 +- .../virtio/virtio_crypto_skcipher_algs.c | 134 ++++++++---------- 6 files changed, 183 insertions(+), 125 deletions(-) create mode 100644 drivers/crypto/virtio/virtio_crypto_common.c diff --git a/drivers/crypto/virtio/Makefile b/drivers/crypto/virtio/Makefile index bfa6cbae342e..49c1fa80e465 100644 --- a/drivers/crypto/virtio/Makefile +++ b/drivers/crypto/virtio/Makefile @@ -3,5 +3,6 @@ obj-$(CONFIG_CRYPTO_DEV_VIRTIO) += virtio_crypto.o virtio_crypto-objs := \ virtio_crypto_skcipher_algs.o \ virtio_crypto_akcipher_algs.o \ + virtio_crypto_common.o \ virtio_crypto_mgr.o \ virtio_crypto_core.o diff --git a/drivers/crypto/virtio/virtio_crypto_akcipher_algs.c b/drivers/crypto/virtio/virtio_crypto_akcipher_algs.c index f3ec9420215e..bf7c1aa4be37 100644 --- a/drivers/crypto/virtio/virtio_crypto_akcipher_algs.c +++ b/drivers/crypto/virtio/virtio_crypto_akcipher_algs.c @@ -102,107 +102,100 @@ static int virtio_crypto_alg_akcipher_init_session(struct virtio_crypto_akcipher { struct scatterlist outhdr_sg, key_sg, inhdr_sg, *sgs[3]; struct virtio_crypto *vcrypto = ctx->vcrypto; + struct virtio_crypto_ctrl_request *vc_ctrl_req = NULL; uint8_t *pkey; - unsigned int inlen; - int err; + int err = -ENOMEM; unsigned int num_out = 0, num_in = 0; + int node = dev_to_node(&vcrypto->vdev->dev); pkey = kmemdup(key, keylen, GFP_ATOMIC); if (!pkey) return -ENOMEM; - spin_lock(&vcrypto->ctrl_lock); - memcpy(&vcrypto->ctrl.header, header, sizeof(vcrypto->ctrl.header)); - memcpy(&vcrypto->ctrl.u, para, sizeof(vcrypto->ctrl.u)); - vcrypto->input.status = cpu_to_le32(VIRTIO_CRYPTO_ERR); + vc_ctrl_req = kzalloc_node(sizeof(*vc_ctrl_req), GFP_KERNEL, node); + if (!vc_ctrl_req) + goto out; - sg_init_one(&outhdr_sg, &vcrypto->ctrl, sizeof(vcrypto->ctrl)); + memcpy(&vc_ctrl_req->ctrl.header, header, sizeof(vc_ctrl_req->ctrl.header)); + memcpy(&vc_ctrl_req->ctrl.u, para, sizeof(vc_ctrl_req->ctrl.u)); + sg_init_one(&outhdr_sg, &vc_ctrl_req->ctrl, sizeof(vc_ctrl_req->ctrl)); sgs[num_out++] = &outhdr_sg; sg_init_one(&key_sg, pkey, keylen); sgs[num_out++] = &key_sg; - sg_init_one(&inhdr_sg, &vcrypto->input, sizeof(vcrypto->input)); + vc_ctrl_req->input.status = cpu_to_le32(VIRTIO_CRYPTO_ERR); + sg_init_one(&inhdr_sg, &vc_ctrl_req->input, sizeof(vc_ctrl_req->input)); sgs[num_out + num_in++] = &inhdr_sg; - err = virtqueue_add_sgs(vcrypto->ctrl_vq, sgs, num_out, num_in, vcrypto, GFP_ATOMIC); + err = virtio_crypto_ctrl_vq_request(vcrypto, sgs, num_out, num_in, vc_ctrl_req); if (err < 0) goto out; - virtqueue_kick(vcrypto->ctrl_vq); - while (!virtqueue_get_buf(vcrypto->ctrl_vq, &inlen) && - !virtqueue_is_broken(vcrypto->ctrl_vq)) - cpu_relax(); - - if (le32_to_cpu(vcrypto->input.status) != VIRTIO_CRYPTO_OK) { + if (le32_to_cpu(vc_ctrl_req->input.status) != VIRTIO_CRYPTO_OK) { + pr_err("virtio_crypto: Create session failed status: %u\n", + le32_to_cpu(vc_ctrl_req->input.status)); err = -EINVAL; goto out; } - ctx->session_id = le64_to_cpu(vcrypto->input.session_id); + ctx->session_id = le64_to_cpu(vc_ctrl_req->input.session_id); ctx->session_valid = true; err = 0; out: - spin_unlock(&vcrypto->ctrl_lock); + kfree_sensitive(vc_ctrl_req); kfree_sensitive(pkey); - if (err < 0) - pr_err("virtio_crypto: Create session failed status: %u\n", - le32_to_cpu(vcrypto->input.status)); - return err; } static int virtio_crypto_alg_akcipher_close_session(struct virtio_crypto_akcipher_ctx *ctx) { struct scatterlist outhdr_sg, inhdr_sg, *sgs[2]; + struct virtio_crypto_ctrl_request *vc_ctrl_req = NULL; struct virtio_crypto_destroy_session_req *destroy_session; struct virtio_crypto *vcrypto = ctx->vcrypto; - unsigned int num_out = 0, num_in = 0, inlen; + unsigned int num_out = 0, num_in = 0; int err; + int node = dev_to_node(&vcrypto->vdev->dev); - spin_lock(&vcrypto->ctrl_lock); - if (!ctx->session_valid) { - err = 0; - goto out; - } - vcrypto->ctrl_status.status = VIRTIO_CRYPTO_ERR; - vcrypto->ctrl.header.opcode = cpu_to_le32(VIRTIO_CRYPTO_AKCIPHER_DESTROY_SESSION); - vcrypto->ctrl.header.queue_id = 0; + if (!ctx->session_valid) + return 0; + + vc_ctrl_req = kzalloc_node(sizeof(*vc_ctrl_req), GFP_KERNEL, node); + if (!vc_ctrl_req) + return -ENOMEM; - destroy_session = &vcrypto->ctrl.u.destroy_session; + vc_ctrl_req->ctrl.header.opcode = cpu_to_le32(VIRTIO_CRYPTO_AKCIPHER_DESTROY_SESSION); + vc_ctrl_req->ctrl.header.queue_id = 0; + + destroy_session = &vc_ctrl_req->ctrl.u.destroy_session; destroy_session->session_id = cpu_to_le64(ctx->session_id); - sg_init_one(&outhdr_sg, &vcrypto->ctrl, sizeof(vcrypto->ctrl)); + sg_init_one(&outhdr_sg, &vc_ctrl_req->ctrl, sizeof(vc_ctrl_req->ctrl)); sgs[num_out++] = &outhdr_sg; - sg_init_one(&inhdr_sg, &vcrypto->ctrl_status.status, sizeof(vcrypto->ctrl_status.status)); + vc_ctrl_req->ctrl_status.status = VIRTIO_CRYPTO_ERR; + sg_init_one(&inhdr_sg, &vc_ctrl_req->ctrl_status.status, + sizeof(vc_ctrl_req->ctrl_status.status)); sgs[num_out + num_in++] = &inhdr_sg; - err = virtqueue_add_sgs(vcrypto->ctrl_vq, sgs, num_out, num_in, vcrypto, GFP_ATOMIC); + err = virtio_crypto_ctrl_vq_request(vcrypto, sgs, num_out, num_in, vc_ctrl_req); if (err < 0) goto out; - virtqueue_kick(vcrypto->ctrl_vq); - while (!virtqueue_get_buf(vcrypto->ctrl_vq, &inlen) && - !virtqueue_is_broken(vcrypto->ctrl_vq)) - cpu_relax(); - - if (vcrypto->ctrl_status.status != VIRTIO_CRYPTO_OK) { + if (vc_ctrl_req->ctrl_status.status != VIRTIO_CRYPTO_OK) { err = -EINVAL; + pr_err("virtio_crypto: Close session failed status: %u, session_id: 0x%llx\n", + vc_ctrl_req->ctrl_status.status, destroy_session->session_id); goto out; } - err = 0; ctx->session_valid = false; - + err = 0; out: - spin_unlock(&vcrypto->ctrl_lock); - if (err < 0) { - pr_err("virtio_crypto: Close session failed status: %u, session_id: 0x%llx\n", - vcrypto->ctrl_status.status, destroy_session->session_id); - } + kfree_sensitive(vc_ctrl_req); return err; } diff --git a/drivers/crypto/virtio/virtio_crypto_common.c b/drivers/crypto/virtio/virtio_crypto_common.c new file mode 100644 index 000000000000..93df73c40dd3 --- /dev/null +++ b/drivers/crypto/virtio/virtio_crypto_common.c @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* Common functions and helpers + * + * Authors: zhenwei pi + * + * Copyright 2022 Bytedance CO., LTD. + */ + +#include "virtio_crypto_common.h" + +static void virtio_crypto_ctrlq_callback(struct virtio_crypto_ctrl_request *vc_ctrl_req) +{ + complete(&vc_ctrl_req->compl); +} + +int virtio_crypto_ctrl_vq_request(struct virtio_crypto *vcrypto, struct scatterlist *sgs[], + unsigned int out_sgs, unsigned int in_sgs, + struct virtio_crypto_ctrl_request *vc_ctrl_req) +{ + int err; + unsigned long flags; + + init_completion(&vc_ctrl_req->compl); + vc_ctrl_req->ctrl_cb = virtio_crypto_ctrlq_callback; + + spin_lock_irqsave(&vcrypto->ctrl_lock, flags); + err = virtqueue_add_sgs(vcrypto->ctrl_vq, sgs, out_sgs, in_sgs, vc_ctrl_req, GFP_ATOMIC); + if (err < 0) { + spin_unlock_irqrestore(&vcrypto->ctrl_lock, flags); + return err; + } + + virtqueue_kick(vcrypto->ctrl_vq); + spin_unlock_irqrestore(&vcrypto->ctrl_lock, flags); + + wait_for_completion(&vc_ctrl_req->compl); + + return 0; +} + +void virtcrypto_ctrlq_callback(struct virtqueue *vq) +{ + struct virtio_crypto *vcrypto = vq->vdev->priv; + struct virtio_crypto_ctrl_request *vc_ctrl_req; + unsigned long flags; + unsigned int len; + + spin_lock_irqsave(&vcrypto->ctrl_lock, flags); + do { + virtqueue_disable_cb(vq); + while ((vc_ctrl_req = virtqueue_get_buf(vq, &len)) != NULL) { + spin_unlock_irqrestore(&vcrypto->ctrl_lock, flags); + if (vc_ctrl_req->ctrl_cb) + vc_ctrl_req->ctrl_cb(vc_ctrl_req); + spin_lock_irqsave(&vcrypto->ctrl_lock, flags); + } + if (unlikely(virtqueue_is_broken(vq))) + break; + } while (!virtqueue_enable_cb(vq)); + spin_unlock_irqrestore(&vcrypto->ctrl_lock, flags); +} diff --git a/drivers/crypto/virtio/virtio_crypto_common.h b/drivers/crypto/virtio/virtio_crypto_common.h index e693d4ee83a6..125ad2300b83 100644 --- a/drivers/crypto/virtio/virtio_crypto_common.h +++ b/drivers/crypto/virtio/virtio_crypto_common.h @@ -13,6 +13,7 @@ #include #include #include +#include /* Internal representation of a data virtqueue */ @@ -65,11 +66,6 @@ struct virtio_crypto { /* Maximum size of per request */ u64 max_size; - /* Control VQ buffers: protected by the ctrl_lock */ - struct virtio_crypto_op_ctrl_req ctrl; - struct virtio_crypto_session_input input; - struct virtio_crypto_inhdr ctrl_status; - unsigned long status; atomic_t ref_count; struct list_head list; @@ -85,6 +81,18 @@ struct virtio_crypto_sym_session_info { __u64 session_id; }; +struct virtio_crypto_ctrl_request; +typedef void (*virtio_crypto_ctrl_callback) + (struct virtio_crypto_ctrl_request *vc_ctrl_req); + +struct virtio_crypto_ctrl_request { + struct virtio_crypto_op_ctrl_req ctrl; + struct virtio_crypto_session_input input; + struct virtio_crypto_inhdr ctrl_status; + virtio_crypto_ctrl_callback ctrl_cb; + struct completion compl; +}; + struct virtio_crypto_request; typedef void (*virtio_crypto_data_callback) (struct virtio_crypto_request *vc_req, int len); @@ -135,4 +143,9 @@ void virtio_crypto_skcipher_algs_unregister(struct virtio_crypto *vcrypto); int virtio_crypto_akcipher_algs_register(struct virtio_crypto *vcrypto); void virtio_crypto_akcipher_algs_unregister(struct virtio_crypto *vcrypto); +void virtcrypto_ctrlq_callback(struct virtqueue *vq); +int virtio_crypto_ctrl_vq_request(struct virtio_crypto *vcrypto, struct scatterlist *sgs[], + unsigned int out_sgs, unsigned int in_sgs, + struct virtio_crypto_ctrl_request *vc_ctrl_req); + #endif /* _VIRTIO_CRYPTO_COMMON_H */ diff --git a/drivers/crypto/virtio/virtio_crypto_core.c b/drivers/crypto/virtio/virtio_crypto_core.c index c6f482db0bc0..e668d4b1bc6a 100644 --- a/drivers/crypto/virtio/virtio_crypto_core.c +++ b/drivers/crypto/virtio/virtio_crypto_core.c @@ -73,7 +73,7 @@ static int virtcrypto_find_vqs(struct virtio_crypto *vi) goto err_names; /* Parameters for control virtqueue */ - callbacks[total_vqs - 1] = NULL; + callbacks[total_vqs - 1] = virtcrypto_ctrlq_callback; names[total_vqs - 1] = "controlq"; /* Allocate/initialize parameters for data virtqueues */ diff --git a/drivers/crypto/virtio/virtio_crypto_skcipher_algs.c b/drivers/crypto/virtio/virtio_crypto_skcipher_algs.c index a618c46a52b8..b8999dab3e66 100644 --- a/drivers/crypto/virtio/virtio_crypto_skcipher_algs.c +++ b/drivers/crypto/virtio/virtio_crypto_skcipher_algs.c @@ -118,11 +118,14 @@ static int virtio_crypto_alg_skcipher_init_session( int encrypt) { struct scatterlist outhdr, key_sg, inhdr, *sgs[3]; - unsigned int tmp; struct virtio_crypto *vcrypto = ctx->vcrypto; int op = encrypt ? VIRTIO_CRYPTO_OP_ENCRYPT : VIRTIO_CRYPTO_OP_DECRYPT; - int err; + int err = -ENOMEM; unsigned int num_out = 0, num_in = 0; + struct virtio_crypto_ctrl_request *vc_ctrl_req = NULL; + struct virtio_crypto_ctrl_header *header; + struct virtio_crypto_sym_create_session_req *sym_create_session; + int node = dev_to_node(&vcrypto->vdev->dev); /* * Avoid to do DMA from the stack, switch to using @@ -133,26 +136,25 @@ static int virtio_crypto_alg_skcipher_init_session( if (!cipher_key) return -ENOMEM; - spin_lock(&vcrypto->ctrl_lock); + vc_ctrl_req = kzalloc_node(sizeof(*vc_ctrl_req), GFP_KERNEL, node); + if (!vc_ctrl_req) + goto out; + /* Pad ctrl header */ - vcrypto->ctrl.header.opcode = - cpu_to_le32(VIRTIO_CRYPTO_CIPHER_CREATE_SESSION); - vcrypto->ctrl.header.algo = cpu_to_le32(alg); + header = &vc_ctrl_req->ctrl.header; + header->opcode = cpu_to_le32(VIRTIO_CRYPTO_CIPHER_CREATE_SESSION); + header->algo = cpu_to_le32(alg); /* Set the default dataqueue id to 0 */ - vcrypto->ctrl.header.queue_id = 0; + header->queue_id = 0; - vcrypto->input.status = cpu_to_le32(VIRTIO_CRYPTO_ERR); /* Pad cipher's parameters */ - vcrypto->ctrl.u.sym_create_session.op_type = - cpu_to_le32(VIRTIO_CRYPTO_SYM_OP_CIPHER); - vcrypto->ctrl.u.sym_create_session.u.cipher.para.algo = - vcrypto->ctrl.header.algo; - vcrypto->ctrl.u.sym_create_session.u.cipher.para.keylen = - cpu_to_le32(keylen); - vcrypto->ctrl.u.sym_create_session.u.cipher.para.op = - cpu_to_le32(op); - - sg_init_one(&outhdr, &vcrypto->ctrl, sizeof(vcrypto->ctrl)); + sym_create_session = &vc_ctrl_req->ctrl.u.sym_create_session; + sym_create_session->op_type = cpu_to_le32(VIRTIO_CRYPTO_SYM_OP_CIPHER); + sym_create_session->u.cipher.para.algo = header->algo; + sym_create_session->u.cipher.para.keylen = cpu_to_le32(keylen); + sym_create_session->u.cipher.para.op = cpu_to_le32(op); + + sg_init_one(&outhdr, &vc_ctrl_req->ctrl, sizeof(vc_ctrl_req->ctrl)); sgs[num_out++] = &outhdr; /* Set key */ @@ -160,45 +162,34 @@ static int virtio_crypto_alg_skcipher_init_session( sgs[num_out++] = &key_sg; /* Return status and session id back */ - sg_init_one(&inhdr, &vcrypto->input, sizeof(vcrypto->input)); + vc_ctrl_req->input.status = cpu_to_le32(VIRTIO_CRYPTO_ERR); + sg_init_one(&inhdr, &vc_ctrl_req->input, sizeof(vc_ctrl_req->input)); sgs[num_out + num_in++] = &inhdr; - err = virtqueue_add_sgs(vcrypto->ctrl_vq, sgs, num_out, - num_in, vcrypto, GFP_ATOMIC); - if (err < 0) { - spin_unlock(&vcrypto->ctrl_lock); - kfree_sensitive(cipher_key); - return err; - } - virtqueue_kick(vcrypto->ctrl_vq); + err = virtio_crypto_ctrl_vq_request(vcrypto, sgs, num_out, num_in, vc_ctrl_req); + if (err < 0) + goto out; - /* - * Trapping into the hypervisor, so the request should be - * handled immediately. - */ - while (!virtqueue_get_buf(vcrypto->ctrl_vq, &tmp) && - !virtqueue_is_broken(vcrypto->ctrl_vq)) - cpu_relax(); - - if (le32_to_cpu(vcrypto->input.status) != VIRTIO_CRYPTO_OK) { - spin_unlock(&vcrypto->ctrl_lock); + if (le32_to_cpu(vc_ctrl_req->input.status) != VIRTIO_CRYPTO_OK) { pr_err("virtio_crypto: Create session failed status: %u\n", - le32_to_cpu(vcrypto->input.status)); - kfree_sensitive(cipher_key); - return -EINVAL; + le32_to_cpu(vc_ctrl_req->input.status)); + err = -EINVAL; + goto out; } if (encrypt) ctx->enc_sess_info.session_id = - le64_to_cpu(vcrypto->input.session_id); + le64_to_cpu(vc_ctrl_req->input.session_id); else ctx->dec_sess_info.session_id = - le64_to_cpu(vcrypto->input.session_id); - - spin_unlock(&vcrypto->ctrl_lock); + le64_to_cpu(vc_ctrl_req->input.session_id); + err = 0; +out: + kfree_sensitive(vc_ctrl_req); kfree_sensitive(cipher_key); - return 0; + + return err; } static int virtio_crypto_alg_skcipher_close_session( @@ -206,21 +197,25 @@ static int virtio_crypto_alg_skcipher_close_session( int encrypt) { struct scatterlist outhdr, status_sg, *sgs[2]; - unsigned int tmp; struct virtio_crypto_destroy_session_req *destroy_session; struct virtio_crypto *vcrypto = ctx->vcrypto; int err; unsigned int num_out = 0, num_in = 0; + struct virtio_crypto_ctrl_request *vc_ctrl_req = NULL; + struct virtio_crypto_ctrl_header *header; + int node = dev_to_node(&vcrypto->vdev->dev); + + vc_ctrl_req = kzalloc_node(sizeof(*vc_ctrl_req), GFP_KERNEL, node); + if (!vc_ctrl_req) + return -ENOMEM; - spin_lock(&vcrypto->ctrl_lock); - vcrypto->ctrl_status.status = VIRTIO_CRYPTO_ERR; /* Pad ctrl header */ - vcrypto->ctrl.header.opcode = - cpu_to_le32(VIRTIO_CRYPTO_CIPHER_DESTROY_SESSION); + header = &vc_ctrl_req->ctrl.header; + header->opcode = cpu_to_le32(VIRTIO_CRYPTO_CIPHER_DESTROY_SESSION); /* Set the default virtqueue id to 0 */ - vcrypto->ctrl.header.queue_id = 0; + header->queue_id = 0; - destroy_session = &vcrypto->ctrl.u.destroy_session; + destroy_session = &vc_ctrl_req->ctrl.u.destroy_session; if (encrypt) destroy_session->session_id = @@ -229,37 +224,32 @@ static int virtio_crypto_alg_skcipher_close_session( destroy_session->session_id = cpu_to_le64(ctx->dec_sess_info.session_id); - sg_init_one(&outhdr, &vcrypto->ctrl, sizeof(vcrypto->ctrl)); + sg_init_one(&outhdr, &vc_ctrl_req->ctrl, sizeof(vc_ctrl_req->ctrl)); sgs[num_out++] = &outhdr; /* Return status and session id back */ - sg_init_one(&status_sg, &vcrypto->ctrl_status.status, - sizeof(vcrypto->ctrl_status.status)); + vc_ctrl_req->ctrl_status.status = VIRTIO_CRYPTO_ERR; + sg_init_one(&status_sg, &vc_ctrl_req->ctrl_status.status, + sizeof(vc_ctrl_req->ctrl_status.status)); sgs[num_out + num_in++] = &status_sg; - err = virtqueue_add_sgs(vcrypto->ctrl_vq, sgs, num_out, - num_in, vcrypto, GFP_ATOMIC); - if (err < 0) { - spin_unlock(&vcrypto->ctrl_lock); - return err; - } - virtqueue_kick(vcrypto->ctrl_vq); - - while (!virtqueue_get_buf(vcrypto->ctrl_vq, &tmp) && - !virtqueue_is_broken(vcrypto->ctrl_vq)) - cpu_relax(); + err = virtio_crypto_ctrl_vq_request(vcrypto, sgs, num_out, num_in, vc_ctrl_req); + if (err < 0) + goto out; - if (vcrypto->ctrl_status.status != VIRTIO_CRYPTO_OK) { - spin_unlock(&vcrypto->ctrl_lock); + if (vc_ctrl_req->ctrl_status.status != VIRTIO_CRYPTO_OK) { pr_err("virtio_crypto: Close session failed status: %u, session_id: 0x%llx\n", - vcrypto->ctrl_status.status, + vc_ctrl_req->ctrl_status.status, destroy_session->session_id); - return -EINVAL; + err = -EINVAL; } - spin_unlock(&vcrypto->ctrl_lock); - return 0; + err = 0; +out: + kfree_sensitive(vc_ctrl_req); + + return err; } static int virtio_crypto_alg_skcipher_init_sessions( From patchwork Fri Apr 15 06:41:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: zhenwei pi X-Patchwork-Id: 12814622 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3087DC4332F for ; Fri, 15 Apr 2022 06:45:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350674AbiDOGrv (ORCPT ); Fri, 15 Apr 2022 02:47:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59478 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350668AbiDOGru (ORCPT ); Fri, 15 Apr 2022 02:47:50 -0400 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A7C2F98F7D for ; Thu, 14 Apr 2022 23:45:23 -0700 (PDT) Received: by mail-pf1-x431.google.com with SMTP id 22so243861pfu.1 for ; Thu, 14 Apr 2022 23:45:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HIfkyA0kHsUAS/WiTZijAfUV1CeLSCyZNaoBawxM4dQ=; b=bRNhcDrRB4NS8pkIyJ0hmHz8NeqERb22Ss/1ljlGJ1GB7zLBKH9J4g7Q6f6Fy9tDKB GVozb8zX2giAR9+DvLMs7bmzTzrFTUwNoNkT+uOLZaiDlcMYge25ogOzrdkN+K3EAmP8 sQLk25OzbV4bjfS9mTvknf0xWdLd2xCTXnQqG455z+v66utPUCXAhgywZzMjKfUesuVG bLYhYFJaj7IePr9CME3+lYjMKRcCFrQQvOFPvE59Pg3wuFK37O3mJzuHslF95TJKtdEe pyrWeNT8Nd9b77qtm/fRf6WPvlz2uhClm8PB+XiPfov6ea0fM1jcfTdpfGHs210PA1lV V9Yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HIfkyA0kHsUAS/WiTZijAfUV1CeLSCyZNaoBawxM4dQ=; b=XhuikYG9bEtyd8i06DQyT09ryxBCNMF1DI+rM+TJ/Xg4J/5TP5LuyTpDr+mc3wiKxW n3g0lw9qboVEmY79ym9Rbf5RILIeFrT56ocpj1ha+1DmM+BQ+FY/gOPq7DuRBwHDfO0U HhivCVXhsfvp1Tcy1uDRYB+w244w/B5x8TsYfYLQQu5RxuC6nTEUkDk+Dn7fKFfO5rYZ 8kQN50FqaWC4bdFp9C+fDrf1HyMWtZvXZYJbIZgl1knLaxr8BAEmbU3hr2MfT6uMpV65 h5X1d4Nm1B05Pk60EXjPd6KTqcTSzPzqzk8FFiE/dNcLbSK7+jHg/fSloSTDHOed79sC vWzQ== X-Gm-Message-State: AOAM532U/RhbxfjGZoKmHqkchLw/qWt40af/kExfgj1Xb4yPRMp9PVGJ 3cbqh+172jsuvOuBgZIKFNdZ7Q== X-Google-Smtp-Source: ABdhPJxL1fEBgglkis/EMPLSlJi9UALXaNzImIYVN25Au5p37RRe+JGZt7QPDDzenL0BV+xqlxWkGA== X-Received: by 2002:a05:6a00:1788:b0:508:70fe:4e62 with SMTP id s8-20020a056a00178800b0050870fe4e62mr4175773pfg.70.1650005123188; Thu, 14 Apr 2022 23:45:23 -0700 (PDT) Received: from always-x1.bytedance.net ([61.120.150.70]) by smtp.gmail.com with ESMTPSA id d8-20020a056a00198800b004fab740dbe6sm1867385pfl.15.2022.04.14.23.45.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 23:45:22 -0700 (PDT) From: zhenwei pi To: arei.gonglei@huawei.com, mst@redhat.com Cc: jasowang@redhat.com, herbert@gondor.apana.org.au, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-crypto@vger.kernel.org, helei.sig11@bytedance.com, davem@davemloft.net, zhenwei pi Subject: [PATCH 2/4] virtio-crypto: move helpers into virtio_crypto_common.c Date: Fri, 15 Apr 2022 14:41:34 +0800 Message-Id: <20220415064136.304661-3-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220415064136.304661-1-pizhenwei@bytedance.com> References: <20220415064136.304661-1-pizhenwei@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Move virtcrypto_clear_request and virtcrypto_dataq_callback into virtio_crypto_common.c to make code clear. Then the xx_core.c supports: - probe/remove/irq affinity seting for a virtio device - basic virtio related operations xx_common.c supports: - common helpers/functions for algos Signed-off-by: zhenwei pi --- drivers/crypto/virtio/virtio_crypto_common.c | 31 +++++++++++++++++++ drivers/crypto/virtio/virtio_crypto_common.h | 2 ++ drivers/crypto/virtio/virtio_crypto_core.c | 32 -------------------- 3 files changed, 33 insertions(+), 32 deletions(-) diff --git a/drivers/crypto/virtio/virtio_crypto_common.c b/drivers/crypto/virtio/virtio_crypto_common.c index 93df73c40dd3..4a23524896fe 100644 --- a/drivers/crypto/virtio/virtio_crypto_common.c +++ b/drivers/crypto/virtio/virtio_crypto_common.c @@ -8,6 +8,14 @@ #include "virtio_crypto_common.h" +void virtcrypto_clear_request(struct virtio_crypto_request *vc_req) +{ + if (vc_req) { + kfree_sensitive(vc_req->req_data); + kfree(vc_req->sgs); + } +} + static void virtio_crypto_ctrlq_callback(struct virtio_crypto_ctrl_request *vc_ctrl_req) { complete(&vc_ctrl_req->compl); @@ -59,3 +67,26 @@ void virtcrypto_ctrlq_callback(struct virtqueue *vq) } while (!virtqueue_enable_cb(vq)); spin_unlock_irqrestore(&vcrypto->ctrl_lock, flags); } + +void virtcrypto_dataq_callback(struct virtqueue *vq) +{ + struct virtio_crypto *vcrypto = vq->vdev->priv; + struct virtio_crypto_request *vc_req; + unsigned long flags; + unsigned int len; + unsigned int qid = vq->index; + + spin_lock_irqsave(&vcrypto->data_vq[qid].lock, flags); + do { + virtqueue_disable_cb(vq); + while ((vc_req = virtqueue_get_buf(vq, &len)) != NULL) { + spin_unlock_irqrestore( + &vcrypto->data_vq[qid].lock, flags); + if (vc_req->alg_cb) + vc_req->alg_cb(vc_req, len); + spin_lock_irqsave( + &vcrypto->data_vq[qid].lock, flags); + } + } while (!virtqueue_enable_cb(vq)); + spin_unlock_irqrestore(&vcrypto->data_vq[qid].lock, flags); +} diff --git a/drivers/crypto/virtio/virtio_crypto_common.h b/drivers/crypto/virtio/virtio_crypto_common.h index 125ad2300b83..a003935f91a9 100644 --- a/drivers/crypto/virtio/virtio_crypto_common.h +++ b/drivers/crypto/virtio/virtio_crypto_common.h @@ -148,4 +148,6 @@ int virtio_crypto_ctrl_vq_request(struct virtio_crypto *vcrypto, struct scatterl unsigned int out_sgs, unsigned int in_sgs, struct virtio_crypto_ctrl_request *vc_ctrl_req); +void virtcrypto_dataq_callback(struct virtqueue *vq); + #endif /* _VIRTIO_CRYPTO_COMMON_H */ diff --git a/drivers/crypto/virtio/virtio_crypto_core.c b/drivers/crypto/virtio/virtio_crypto_core.c index e668d4b1bc6a..d8edefcb966c 100644 --- a/drivers/crypto/virtio/virtio_crypto_core.c +++ b/drivers/crypto/virtio/virtio_crypto_core.c @@ -13,38 +13,6 @@ #include "virtio_crypto_common.h" -void -virtcrypto_clear_request(struct virtio_crypto_request *vc_req) -{ - if (vc_req) { - kfree_sensitive(vc_req->req_data); - kfree(vc_req->sgs); - } -} - -static void virtcrypto_dataq_callback(struct virtqueue *vq) -{ - struct virtio_crypto *vcrypto = vq->vdev->priv; - struct virtio_crypto_request *vc_req; - unsigned long flags; - unsigned int len; - unsigned int qid = vq->index; - - spin_lock_irqsave(&vcrypto->data_vq[qid].lock, flags); - do { - virtqueue_disable_cb(vq); - while ((vc_req = virtqueue_get_buf(vq, &len)) != NULL) { - spin_unlock_irqrestore( - &vcrypto->data_vq[qid].lock, flags); - if (vc_req->alg_cb) - vc_req->alg_cb(vc_req, len); - spin_lock_irqsave( - &vcrypto->data_vq[qid].lock, flags); - } - } while (!virtqueue_enable_cb(vq)); - spin_unlock_irqrestore(&vcrypto->data_vq[qid].lock, flags); -} - static int virtcrypto_find_vqs(struct virtio_crypto *vi) { vq_callback_t **callbacks; From patchwork Fri Apr 15 06:41:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: zhenwei pi X-Patchwork-Id: 12814623 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA4C9C433F5 for ; Fri, 15 Apr 2022 06:45:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1350691AbiDOGsJ (ORCPT ); Fri, 15 Apr 2022 02:48:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59692 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343934AbiDOGsH (ORCPT ); Fri, 15 Apr 2022 02:48:07 -0400 Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A3945B0D0D for ; Thu, 14 Apr 2022 23:45:28 -0700 (PDT) Received: by mail-pj1-x102e.google.com with SMTP id ll10so6979546pjb.5 for ; Thu, 14 Apr 2022 23:45:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eUPRo1W7HXTqRUmuszkjpWErgWaPyulrOyER93ew8mA=; b=2okK/vk4qQTljwBk1QYv8jssjKnLC1JWY6AXCN8hrTti/FJJ9mD+eNzxXeBbs+jshV AWEUMm0LmqJSBH/7058T7s6qFIKwViTM9mC5t3WNsTLJD21HO2MayRpqye5vrVeHx2Fk p1KyRNx72f3Lx/Q8nz+f2wJsYzFgsckeQXIMFOAZ+Pgb2CGZwskKTyUucasgjjisAtA7 nxrk5PcUU0vN7wVsPu9vyZaarfvQO1dgcjhd5A3mxbr//0sV6vCZjwfbj/DzB7p7cbfN OH7l0ZGqqMf/TwdXfQKp8ots3JjEDr07bevn8Yh4I5I+nielXDSLVPOXLwupIMlP/RZ0 9TQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eUPRo1W7HXTqRUmuszkjpWErgWaPyulrOyER93ew8mA=; b=AR8eLSyB06ypPGkDT7cfH0NQmGN4+6Q4KTvlJCC+husD+Fn4fZVwXtDOwIFSnLczxe IRlf0Sx2QUMt8w/fXKwfBRr8H30JhkF/q5kTXsqQl9F/+tzVJiuxwnWPpKriMWAJ6SCh J6gA7HAJCid6SuN8XzgUBWtsVSUSxEbrTnwZ8plrSMSMXOJHTlrBUMPCOlDYinFFPa/8 S01t9QDbsI01cYJvFOCcrj+LyJSZWy0tqwF8+srZL1KBqMolj2/X4FZtwBegsP91Nhq7 BJM+9zexNKuqIEBK36goZK1YbrvGJDutQIaveZgwQRTbA0GH59OuKF6Ck25GV1CjMoSF fsBQ== X-Gm-Message-State: AOAM531C1//a3yzDr3DzZzZgbLN2m6KI13utWMYgtT93vZgsAI990biF RKvnftYE35ehEQ7m/TZP1r/VWA== X-Google-Smtp-Source: ABdhPJxcShAeTRpf1TUgLEfubOBQZZ90/nwfksxqrBB43jrycMr1mv1lYJHhKntT+PHRDW956nfgSA== X-Received: by 2002:a17:902:b10f:b0:156:612f:318d with SMTP id q15-20020a170902b10f00b00156612f318dmr50126545plr.143.1650005128147; Thu, 14 Apr 2022 23:45:28 -0700 (PDT) Received: from always-x1.bytedance.net ([61.120.150.70]) by smtp.gmail.com with ESMTPSA id d8-20020a056a00198800b004fab740dbe6sm1867385pfl.15.2022.04.14.23.45.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 23:45:27 -0700 (PDT) From: zhenwei pi To: arei.gonglei@huawei.com, mst@redhat.com Cc: jasowang@redhat.com, herbert@gondor.apana.org.au, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-crypto@vger.kernel.org, helei.sig11@bytedance.com, davem@davemloft.net, lei he , zhenwei pi Subject: [PATCH 3/4] virtio-crypto: adjust dst_len at ops callback Date: Fri, 15 Apr 2022 14:41:35 +0800 Message-Id: <20220415064136.304661-4-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220415064136.304661-1-pizhenwei@bytedance.com> References: <20220415064136.304661-1-pizhenwei@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: lei he For some akcipher operations(eg, decryption of pkcs1pad(rsa)), the length of returned result maybe less than akcipher_req->dst_len, we need to recalculate the actual dst_len through the virt-queue protocol. Signed-off-by: lei he Signed-off-by: zhenwei pi --- drivers/crypto/virtio/virtio_crypto_akcipher_algs.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/virtio/virtio_crypto_akcipher_algs.c b/drivers/crypto/virtio/virtio_crypto_akcipher_algs.c index bf7c1aa4be37..7cd932c2d9ee 100644 --- a/drivers/crypto/virtio/virtio_crypto_akcipher_algs.c +++ b/drivers/crypto/virtio/virtio_crypto_akcipher_algs.c @@ -90,9 +90,12 @@ static void virtio_crypto_dataq_akcipher_callback(struct virtio_crypto_request * } akcipher_req = vc_akcipher_req->akcipher_req; - if (vc_akcipher_req->opcode != VIRTIO_CRYPTO_AKCIPHER_VERIFY) + if (vc_akcipher_req->opcode != VIRTIO_CRYPTO_AKCIPHER_VERIFY) { + /* actuall length maybe less than dst buffer */ + akcipher_req->dst_len = len - sizeof(vc_req->status); sg_copy_from_buffer(akcipher_req->dst, sg_nents(akcipher_req->dst), vc_akcipher_req->dst_buf, akcipher_req->dst_len); + } virtio_crypto_akcipher_finalize_req(vc_akcipher_req, akcipher_req, error); } From patchwork Fri Apr 15 06:41:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: zhenwei pi X-Patchwork-Id: 12814624 X-Patchwork-Delegate: herbert@gondor.apana.org.au Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46927C433EF for ; Fri, 15 Apr 2022 06:45:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343934AbiDOGsK (ORCPT ); Fri, 15 Apr 2022 02:48:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59706 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1350718AbiDOGsI (ORCPT ); Fri, 15 Apr 2022 02:48:08 -0400 Received: from mail-pg1-x531.google.com (mail-pg1-x531.google.com [IPv6:2607:f8b0:4864:20::531]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EC526B0A68 for ; Thu, 14 Apr 2022 23:45:32 -0700 (PDT) Received: by mail-pg1-x531.google.com with SMTP id s137so6664528pgs.5 for ; Thu, 14 Apr 2022 23:45:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ouz4DgKg6HX7//LGRrUD97CctBPhdyzV/9EeFTiygxk=; b=gMCyNTTL33kI18vJdBqOu9Mcm2fImvGdxj+ka3vkQktHDiY/CK6mvTSSga/VonfgCK G2yDrY4zAQ73dGXaCnliYP1Sy/I4SgZ0iyzTWjhjjHSEigW4fxdNA2tP2Wf0uxbksfhX cBHCROmEIoL5iGVlxzgLAUEjeRiwP80HJL4kjIiu0Ul8xGUMQGmqpnFWq7dE7QrLdKSO 69iqyiKk38F14+E73N4tvDHBGN2U4xpNoI5fXGuTfLl7tLH4+PRaN5ohLpVXe45yFSXv GER/mPzklCwMsqFB3YjPtmkoDrHVR4kVx5qMb1cFyLLTHiTjMubqehbQDnte90iuhh6h Bb1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ouz4DgKg6HX7//LGRrUD97CctBPhdyzV/9EeFTiygxk=; b=IT8l1V9Zl9CXBuiPEtMsfycwkzFcNKHwhu9zcptFYMicKUsvlrxUthLHHF5ZRxahzn DM6karueglMlcYitQAjECuW+qP0no8M/evvnemgFtFU3to37XHATkif2DDg9npjuqFP+ ojIZkGgYUJ1udgcss8EIUn3OfLKf9qWNUXQVw3iAnLrrFbGSltxoTMb7kcMP07Iq/0aK +mt+3bnGQ0o1eS9QrsNorUBJ8NOsPx1gYS5xY+lmOFYGDrW8yEashwD5hkTBZZobQm6E FyJQ3Hs1ORLALZTf5Cd6+x/f3Z2PHVrkOviqOh5UyG5mOO2GnAoqHCubPk/Bc+UYycsC rHRg== X-Gm-Message-State: AOAM530JISjZEk9g49lmaea6fNj+W8P84f9HKwFERODuDuUWEfRKDjr2 ur7VTN5ilY749ofye1fKl4hlnA== X-Google-Smtp-Source: ABdhPJwyfNEEO9oK4pBktYn2c0uxMeRu2wWbIGD1BpxdATL+9+KfkFN82SS1+bpSVF2rQMXvUC8AAw== X-Received: by 2002:a63:6a88:0:b0:398:54fb:85ba with SMTP id f130-20020a636a88000000b0039854fb85bamr5311202pgc.88.1650005132521; Thu, 14 Apr 2022 23:45:32 -0700 (PDT) Received: from always-x1.bytedance.net ([61.120.150.70]) by smtp.gmail.com with ESMTPSA id d8-20020a056a00198800b004fab740dbe6sm1867385pfl.15.2022.04.14.23.45.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Apr 2022 23:45:32 -0700 (PDT) From: zhenwei pi To: arei.gonglei@huawei.com, mst@redhat.com Cc: jasowang@redhat.com, herbert@gondor.apana.org.au, linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-crypto@vger.kernel.org, helei.sig11@bytedance.com, davem@davemloft.net Subject: [PATCH 4/4] virtio-crypto: enable retry for virtio-crypto-dev Date: Fri, 15 Apr 2022 14:41:36 +0800 Message-Id: <20220415064136.304661-5-pizhenwei@bytedance.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220415064136.304661-1-pizhenwei@bytedance.com> References: <20220415064136.304661-1-pizhenwei@bytedance.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org From: lei he Enable retry for virtio-crypto-dev, so that crypto-engine can process cipher-requests parallelly. Signed-off-by: lei he --- drivers/crypto/virtio/virtio_crypto_core.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/crypto/virtio/virtio_crypto_core.c b/drivers/crypto/virtio/virtio_crypto_core.c index d8edefcb966c..5c0d68c9e894 100644 --- a/drivers/crypto/virtio/virtio_crypto_core.c +++ b/drivers/crypto/virtio/virtio_crypto_core.c @@ -62,7 +62,8 @@ static int virtcrypto_find_vqs(struct virtio_crypto *vi) spin_lock_init(&vi->data_vq[i].lock); vi->data_vq[i].vq = vqs[i]; /* Initialize crypto engine */ - vi->data_vq[i].engine = crypto_engine_alloc_init(dev, 1); + vi->data_vq[i].engine = crypto_engine_alloc_init_and_set(dev, true, NULL, 1, + virtqueue_get_vring_size(vqs[i])); if (!vi->data_vq[i].engine) { ret = -ENOMEM; goto err_engine;