From patchwork Tue May 16 12:51:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Etienne Carriere X-Patchwork-Id: 13243124 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AE2E7C77B75 for ; Tue, 16 May 2023 12:54:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=gSuHt3SSIH4K9qPr0Crlz+yyFyOthW7oxo5PN60yXEU=; b=Mv23m7PhZ2xiUi 4/CXZLRQnESZ+glyVUhgqz+04AhCebX/yZ+Dcw3pu/FMNOLscmr9f0mhDGJI+lfh3/zO4HIw8cDRV 3/dpe48tQD45JS8rhY8OQhj4sTT3tVuOzXQRxXXz78lH864D8Sa20VICsRqtCRgqlm/w1RTRQf5Tu TX9bjHQzTQVsjgaRonNzdbqn/GXV1NfKPF0gwGSNH2KihQeCbE490KAeHN33xVe5b8xh9i7q3huv3 3nvv8hjnt5+uN11WuaYMUqT0RXsHlXj5gdP835umKX0W29NwH2uf0/YTWQx8g+QmnfakWPTPCWnlC ckE8iu6r4aBG/WVGa4fA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pyuC2-005nGs-0V; Tue, 16 May 2023 12:54:18 +0000 Received: from mail-wm1-x332.google.com ([2a00:1450:4864:20::332]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pyuBu-005nCn-1b for linux-arm-kernel@lists.infradead.org; Tue, 16 May 2023 12:54:12 +0000 Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-3f42c865535so82994325e9.1 for ; Tue, 16 May 2023 05:54:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684241648; x=1686833648; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=g96hozmey9eTsJtyTZGw63BMUWAerQsKF27YVAfUNYA=; b=TjCX+fv9u28zn3qtuTeRyISCyzkLKm/urIKHSWnfTMBfpZ1db3sRlH0cYd4WbTltyB WJr1ePd7YpaOR39LaCKRwoG17lruBMexwP0uOnhlfUdzsWlatvCjtQarbFpjWK6GmwKd mSBXb+zFsubN0atWuDW3U+VkeaQ0ZUORCv08c1k/rjT5+v2A8baZCcUiNkIvtKeG2M+c RNEmiVpSRqFfeThquijZnfwmX+RnK82UW+13fbO1Oh6mucOdHY7i7u9IowAc1iIl9H5W vdAy7nBjHqfHXBp69tLFWMs0L5cJX/oW7LwpycvmhsKlP5JqFnAJlNwgIBN+QICuCN26 lVqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684241648; x=1686833648; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=g96hozmey9eTsJtyTZGw63BMUWAerQsKF27YVAfUNYA=; b=AslFoQxJfnOu5Oi0HtpACXF0OUcjnRWrgzEVjoW2Ouhtd9O0XkqGFyPNhRXmFLKdFz 4S4NazKwArIQjc6rpOPWGR1n+pxmygT56KEOntQ00nf1B13K2/97/rE5VMHlGZopEzJD bI4c21vP8pqr7/v/gYlHi9tSr6eB8vIqOpiNLA4Rk8N/ui1rT4pD7cNt4TquUUqMHkZr f+MEg2uFvD8zzDtBkoeIWOhHePaF0TsBbAdkilFU4/P3oI+F+CqfuHTrdMcxN/l+ze4u gw//ad8pPoil7bBrfRlEtC3r3BdJvZ9c6kn2LGgVa4pP7/mc/JZSQeIokjmKNACvsDa+ 5FxQ== X-Gm-Message-State: AC+VfDz8KYPl7dr8OE+22ZDE8wvnmXQSviqwkg2igmyrACxs5pYcgZPo x9VHBozed1n2Rdpbtd7qpaFe4g== X-Google-Smtp-Source: ACHHUZ5nIzv6OoloUrWBujDEUC6gkSbubb+mw1ZaVk5g2AdyhFF5ibvmh2TtP8+XDkxeVhQbeBnpmg== X-Received: by 2002:a1c:cc14:0:b0:3f4:20bd:ba46 with SMTP id h20-20020a1ccc14000000b003f420bdba46mr23711425wmb.5.1684241648269; Tue, 16 May 2023 05:54:08 -0700 (PDT) Received: from lmecxl1178.lme.st.com (87-88-164-239.abo.bbox.fr. [87.88.164.239]) by smtp.gmail.com with ESMTPSA id r18-20020a056000015200b003047d5b8817sm2521963wrx.80.2023.05.16.05.54.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 05:54:07 -0700 (PDT) From: Etienne Carriere To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, op-tee@lists.trustedfirmware.org, Sumit Garg , Jens Wiklander , Sudeep Holla , Cristian Marussi , Vincent Guittot , Etienne Carriere Subject: [PATCH v8 1/4] tee: optee: system call property Date: Tue, 16 May 2023 14:51:08 +0200 Message-Id: <20230516125111.2690204-2-etienne.carriere@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230516125111.2690204-1-etienne.carriere@linaro.org> References: <20230516125111.2690204-1-etienne.carriere@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230516_055410_533312_BAD1AAB2 X-CRM114-Status: GOOD ( 22.34 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Adds an argument to do_call_with_arg() handler to tell whether the call is a system call or nor. This change always sets this info to false hence no functional change. This change prepares management of system invocation proposed in a later change. Co-developed-by: Jens Wiklander Signed-off-by: Jens Wiklander Signed-off-by: Etienne Carriere Reviewed-by: Sumit Garg --- No change since v7 Changes since v6: - Squashed a part of patch v6 3/4 changes into this patch v7 1/4 related to adding boolean system thread attribute into optee driver call queue and SMC/FF-A ABIs API functions. - Removed local variable sys_thread set to constant false value and use false straight as function argument instead. - Comment on struct optee_session::use_sys_thread being read with optee mutex locked is not addressed as still under discussion. No changes since v5 Changes since v4: - New change, extracted from PATCH v4 1/2 (tee: system invocation") and revised to cover preparatory changes in optee driver for system session support with contribution from Jens. --- drivers/tee/optee/call.c | 24 +++++++++++++++++------- drivers/tee/optee/core.c | 5 +++-- drivers/tee/optee/ffa_abi.c | 10 ++++++---- drivers/tee/optee/optee_private.h | 9 ++++++--- drivers/tee/optee/smc_abi.c | 15 ++++++++------- 5 files changed, 40 insertions(+), 23 deletions(-) diff --git a/drivers/tee/optee/call.c b/drivers/tee/optee/call.c index df5fb5410b72..152ae9bb1785 100644 --- a/drivers/tee/optee/call.c +++ b/drivers/tee/optee/call.c @@ -40,7 +40,7 @@ struct optee_shm_arg_entry { }; void optee_cq_wait_init(struct optee_call_queue *cq, - struct optee_call_waiter *w) + struct optee_call_waiter *w, bool sys_thread) { /* * We're preparing to make a call to secure world. In case we can't @@ -328,7 +328,8 @@ int optee_open_session(struct tee_context *ctx, goto out; } - if (optee->ops->do_call_with_arg(ctx, shm, offs)) { + if (optee->ops->do_call_with_arg(ctx, shm, offs, + sess->use_sys_thread)) { msg_arg->ret = TEEC_ERROR_COMMUNICATION; msg_arg->ret_origin = TEEC_ORIGIN_COMMS; } @@ -360,7 +361,8 @@ int optee_open_session(struct tee_context *ctx, return rc; } -int optee_close_session_helper(struct tee_context *ctx, u32 session) +int optee_close_session_helper(struct tee_context *ctx, u32 session, + bool system_thread) { struct optee *optee = tee_get_drvdata(ctx->teedev); struct optee_shm_arg_entry *entry; @@ -374,7 +376,7 @@ int optee_close_session_helper(struct tee_context *ctx, u32 session) msg_arg->cmd = OPTEE_MSG_CMD_CLOSE_SESSION; msg_arg->session = session; - optee->ops->do_call_with_arg(ctx, shm, offs); + optee->ops->do_call_with_arg(ctx, shm, offs, system_thread); optee_free_msg_arg(ctx, entry, offs); @@ -385,6 +387,7 @@ int optee_close_session(struct tee_context *ctx, u32 session) { struct optee_context_data *ctxdata = ctx->data; struct optee_session *sess; + bool system_thread; /* Check that the session is valid and remove it from the list */ mutex_lock(&ctxdata->mutex); @@ -394,9 +397,10 @@ int optee_close_session(struct tee_context *ctx, u32 session) mutex_unlock(&ctxdata->mutex); if (!sess) return -EINVAL; + system_thread = sess->use_sys_thread; kfree(sess); - return optee_close_session_helper(ctx, session); + return optee_close_session_helper(ctx, session, system_thread); } int optee_invoke_func(struct tee_context *ctx, struct tee_ioctl_invoke_arg *arg, @@ -408,12 +412,15 @@ int optee_invoke_func(struct tee_context *ctx, struct tee_ioctl_invoke_arg *arg, struct optee_msg_arg *msg_arg; struct optee_session *sess; struct tee_shm *shm; + bool system_thread; u_int offs; int rc; /* Check that the session is valid */ mutex_lock(&ctxdata->mutex); sess = find_session(ctxdata, arg->session); + if (sess) + system_thread = sess->use_sys_thread; mutex_unlock(&ctxdata->mutex); if (!sess) return -EINVAL; @@ -432,7 +439,7 @@ int optee_invoke_func(struct tee_context *ctx, struct tee_ioctl_invoke_arg *arg, if (rc) goto out; - if (optee->ops->do_call_with_arg(ctx, shm, offs)) { + if (optee->ops->do_call_with_arg(ctx, shm, offs, system_thread)) { msg_arg->ret = TEEC_ERROR_COMMUNICATION; msg_arg->ret_origin = TEEC_ORIGIN_COMMS; } @@ -457,12 +464,15 @@ int optee_cancel_req(struct tee_context *ctx, u32 cancel_id, u32 session) struct optee_shm_arg_entry *entry; struct optee_msg_arg *msg_arg; struct optee_session *sess; + bool system_thread; struct tee_shm *shm; u_int offs; /* Check that the session is valid */ mutex_lock(&ctxdata->mutex); sess = find_session(ctxdata, session); + if (sess) + system_thread = sess->use_sys_thread; mutex_unlock(&ctxdata->mutex); if (!sess) return -EINVAL; @@ -474,7 +484,7 @@ int optee_cancel_req(struct tee_context *ctx, u32 cancel_id, u32 session) msg_arg->cmd = OPTEE_MSG_CMD_CANCEL; msg_arg->session = session; msg_arg->cancel_id = cancel_id; - optee->ops->do_call_with_arg(ctx, shm, offs); + optee->ops->do_call_with_arg(ctx, shm, offs, system_thread); optee_free_msg_arg(ctx, entry, offs); return 0; diff --git a/drivers/tee/optee/core.c b/drivers/tee/optee/core.c index 2a258bd3b6b5..d01ca47f7bde 100644 --- a/drivers/tee/optee/core.c +++ b/drivers/tee/optee/core.c @@ -129,7 +129,8 @@ int optee_open(struct tee_context *ctx, bool cap_memref_null) static void optee_release_helper(struct tee_context *ctx, int (*close_session)(struct tee_context *ctx, - u32 session)) + u32 session, + bool system_thread)) { struct optee_context_data *ctxdata = ctx->data; struct optee_session *sess; @@ -141,7 +142,7 @@ static void optee_release_helper(struct tee_context *ctx, list_for_each_entry_safe(sess, sess_tmp, &ctxdata->sess_list, list_node) { list_del(&sess->list_node); - close_session(ctx, sess->session_id); + close_session(ctx, sess->session_id, sess->use_sys_thread); kfree(sess); } kfree(ctxdata); diff --git a/drivers/tee/optee/ffa_abi.c b/drivers/tee/optee/ffa_abi.c index 0828240f27e6..5fde9d4100e3 100644 --- a/drivers/tee/optee/ffa_abi.c +++ b/drivers/tee/optee/ffa_abi.c @@ -528,7 +528,8 @@ static void optee_handle_ffa_rpc(struct tee_context *ctx, struct optee *optee, static int optee_ffa_yielding_call(struct tee_context *ctx, struct ffa_send_direct_data *data, - struct optee_msg_arg *rpc_arg) + struct optee_msg_arg *rpc_arg, + bool system_thread) { struct optee *optee = tee_get_drvdata(ctx->teedev); struct ffa_device *ffa_dev = optee->ffa.ffa_dev; @@ -541,7 +542,7 @@ static int optee_ffa_yielding_call(struct tee_context *ctx, int rc; /* Initialize waiter */ - optee_cq_wait_init(&optee->call_queue, &w); + optee_cq_wait_init(&optee->call_queue, &w, system_thread); while (true) { rc = msg_ops->sync_send_receive(ffa_dev, data); if (rc) @@ -612,7 +613,8 @@ static int optee_ffa_yielding_call(struct tee_context *ctx, */ static int optee_ffa_do_call_with_arg(struct tee_context *ctx, - struct tee_shm *shm, u_int offs) + struct tee_shm *shm, u_int offs, + bool system_thread) { struct ffa_send_direct_data data = { .data0 = OPTEE_FFA_YIELDING_CALL_WITH_ARG, @@ -642,7 +644,7 @@ static int optee_ffa_do_call_with_arg(struct tee_context *ctx, if (IS_ERR(rpc_arg)) return PTR_ERR(rpc_arg); - return optee_ffa_yielding_call(ctx, &data, rpc_arg); + return optee_ffa_yielding_call(ctx, &data, rpc_arg, system_thread); } /* diff --git a/drivers/tee/optee/optee_private.h b/drivers/tee/optee/optee_private.h index 72685ee0d53f..b68273051454 100644 --- a/drivers/tee/optee/optee_private.h +++ b/drivers/tee/optee/optee_private.h @@ -154,7 +154,8 @@ struct optee; */ struct optee_ops { int (*do_call_with_arg)(struct tee_context *ctx, - struct tee_shm *shm_arg, u_int offs); + struct tee_shm *shm_arg, u_int offs, + bool system_thread); int (*to_msg_param)(struct optee *optee, struct optee_msg_param *msg_params, size_t num_params, const struct tee_param *params); @@ -204,6 +205,7 @@ struct optee { struct optee_session { struct list_head list_node; u32 session_id; + bool use_sys_thread; }; struct optee_context_data { @@ -252,7 +254,8 @@ int optee_supp_send(struct tee_context *ctx, u32 ret, u32 num_params, int optee_open_session(struct tee_context *ctx, struct tee_ioctl_open_session_arg *arg, struct tee_param *param); -int optee_close_session_helper(struct tee_context *ctx, u32 session); +int optee_close_session_helper(struct tee_context *ctx, u32 session, + bool system_thread); int optee_close_session(struct tee_context *ctx, u32 session); int optee_invoke_func(struct tee_context *ctx, struct tee_ioctl_invoke_arg *arg, struct tee_param *param); @@ -301,7 +304,7 @@ static inline void optee_to_msg_param_value(struct optee_msg_param *mp, } void optee_cq_wait_init(struct optee_call_queue *cq, - struct optee_call_waiter *w); + struct optee_call_waiter *w, bool sys_thread); void optee_cq_wait_for_completion(struct optee_call_queue *cq, struct optee_call_waiter *w); void optee_cq_wait_final(struct optee_call_queue *cq, diff --git a/drivers/tee/optee/smc_abi.c b/drivers/tee/optee/smc_abi.c index 49702cb08f4f..e2763cdcf111 100644 --- a/drivers/tee/optee/smc_abi.c +++ b/drivers/tee/optee/smc_abi.c @@ -283,7 +283,7 @@ static void optee_enable_shm_cache(struct optee *optee) struct optee_call_waiter w; /* We need to retry until secure world isn't busy. */ - optee_cq_wait_init(&optee->call_queue, &w); + optee_cq_wait_init(&optee->call_queue, &w, false); while (true) { struct arm_smccc_res res; @@ -308,7 +308,7 @@ static void __optee_disable_shm_cache(struct optee *optee, bool is_mapped) struct optee_call_waiter w; /* We need to retry until secure world isn't busy. */ - optee_cq_wait_init(&optee->call_queue, &w); + optee_cq_wait_init(&optee->call_queue, &w, false); while (true) { union { struct arm_smccc_res smccc; @@ -507,7 +507,7 @@ static int optee_shm_register(struct tee_context *ctx, struct tee_shm *shm, msg_arg->params->u.tmem.buf_ptr = virt_to_phys(pages_list) | (tee_shm_get_page_offset(shm) & (OPTEE_MSG_NONCONTIG_PAGE_SIZE - 1)); - if (optee->ops->do_call_with_arg(ctx, shm_arg, 0) || + if (optee->ops->do_call_with_arg(ctx, shm_arg, 0, false) || msg_arg->ret != TEEC_SUCCESS) rc = -EINVAL; @@ -550,7 +550,7 @@ static int optee_shm_unregister(struct tee_context *ctx, struct tee_shm *shm) msg_arg->params[0].attr = OPTEE_MSG_ATTR_TYPE_RMEM_INPUT; msg_arg->params[0].u.rmem.shm_ref = (unsigned long)shm; - if (optee->ops->do_call_with_arg(ctx, shm_arg, 0) || + if (optee->ops->do_call_with_arg(ctx, shm_arg, 0, false) || msg_arg->ret != TEEC_SUCCESS) rc = -EINVAL; out: @@ -885,7 +885,8 @@ static void optee_handle_rpc(struct tee_context *ctx, * Returns return code from secure world, 0 is OK */ static int optee_smc_do_call_with_arg(struct tee_context *ctx, - struct tee_shm *shm, u_int offs) + struct tee_shm *shm, u_int offs, + bool system_thread) { struct optee *optee = tee_get_drvdata(ctx->teedev); struct optee_call_waiter w; @@ -926,7 +927,7 @@ static int optee_smc_do_call_with_arg(struct tee_context *ctx, reg_pair_from_64(¶m.a1, ¶m.a2, parg); } /* Initialize waiter */ - optee_cq_wait_init(&optee->call_queue, &w); + optee_cq_wait_init(&optee->call_queue, &w, system_thread); while (true) { struct arm_smccc_res res; @@ -977,7 +978,7 @@ static int simple_call_with_arg(struct tee_context *ctx, u32 cmd) return PTR_ERR(msg_arg); msg_arg->cmd = cmd; - optee_smc_do_call_with_arg(ctx, shm, offs); + optee_smc_do_call_with_arg(ctx, shm, offs, false); optee_free_msg_arg(ctx, entry, offs); return 0; From patchwork Tue May 16 12:51:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Etienne Carriere X-Patchwork-Id: 13243125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5B151C7EE2D for ; Tue, 16 May 2023 12:54:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=KnM6CRBBwX21u4msRTetIM5H0KRnjvGxyoKOlCmSOZ8=; b=hivEOXxXBvvI50 2ICWfRb3S0f7qL4LY4g2mXPrjbJwe8MrSrXLtP9pSw/ozxWdIzGomTW5FxdWzTy4lPaPAcFHgbeX1 BtJlIIybVZS/zu+JMhLPwGpLybNUBOHEiOITR7YYxW53lrLbFs/yh3HlDelhb7zj3PdpoKSGvFEh6 nRbR8qoNoXKb4dBcpXjmBNJBiG4eZ6TI9acL+nFqEO4sWgGR50GxSFM/a+DXOksiGGYmTU6kXj4B+ nhVzVcD/i1ZE8a7+0G25HWp7Ids/ADuGK41m4xGEbRKIBOoPSAIZeNuQRuvpLfMabUPX1igaw78KU N6NOJSjyK7smYdeWYlvA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pyuC2-005nHO-2M; Tue, 16 May 2023 12:54:18 +0000 Received: from mail-wm1-x332.google.com ([2a00:1450:4864:20::332]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pyuBv-005nDh-0L for linux-arm-kernel@lists.infradead.org; Tue, 16 May 2023 12:54:12 +0000 Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-3f4c6c4b51eso69358295e9.2 for ; Tue, 16 May 2023 05:54:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684241650; x=1686833650; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rHfKWQN5qNmIaor+bUPf6Gw9IbAjQ9a+/M7m53OG8rQ=; b=bOgfyfAHUwsfg/QA3znVupJQpSN93W4nNi9lrHk6A+85DoGyiX7v6ZnOqhGJwtRcr2 TwER5dqYHdVvr+ZJmlxEL5fa/igp6k6ajhLlT5nOkU2QAv4EOV4Td1cwxC/1gTEYeYVw aj3OQQTqLkncAZ+QWZ+tEIGvoEr5uUObN36jlvlFXAnNDjJZ711M0e6mQ+7m4VBgYslo XYa8cZVakfdbt2S2yrhWVvFQ3uC4qNkmBj8BktuyHuIO5ozeJAqsn8lPsvYxdbuDY5Yf I1Lo1h/u68pdZYAb9REKb2wLxoHDBH0oDVWFMmdSkLhfLJmYmQKWcJnjb5fSrl6YvF5W PlWQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684241650; x=1686833650; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rHfKWQN5qNmIaor+bUPf6Gw9IbAjQ9a+/M7m53OG8rQ=; b=hV3sCIwDTMM/icQKXdNIs6IRa7KUfbDnnn2fwYLsfxhyj5LP6ZwZlCEnFWbFAc7chw q3l49aiIvGWRbYgjTA3mjlSc5xsy3esfiezJK/HkSp+gDRQHiuOW4UU+X4oDuxOmgom7 FB/7RXAHF4ALdGnCgm3HzNvmWtn1rwIEyUbB/hjwPjsjqB6L+2bEZHK7PZOPcVJiJc2p t2kDzFTTt/u3VlJe9+96OaTqpwP620kkShK/iai4EpACrwow+NNLovSmzkxHgtKQgahV X06mLYpXhXbYEr95OAI63RSF+olfSa35nXqPojIEr9bk556JP2cYWBei+UbbPCjF2k5t 6AYA== X-Gm-Message-State: AC+VfDznIWw40isfLDrmi3TEt0byju2Du4PJlIeRBjNk61N4xzczVwLA lz7n1bBeWsnwWPRDIJJxNIkjlw== X-Google-Smtp-Source: ACHHUZ46uSK37InAmYxPUA1vxT7S29fG0nYW5wGQruuVn1DTHGuNdgNv3tjqszcGA+zui9h4A1WU0g== X-Received: by 2002:a1c:f703:0:b0:3f1:979f:a733 with SMTP id v3-20020a1cf703000000b003f1979fa733mr27699054wmh.31.1684241649744; Tue, 16 May 2023 05:54:09 -0700 (PDT) Received: from lmecxl1178.lme.st.com (87-88-164-239.abo.bbox.fr. [87.88.164.239]) by smtp.gmail.com with ESMTPSA id r18-20020a056000015200b003047d5b8817sm2521963wrx.80.2023.05.16.05.54.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 05:54:09 -0700 (PDT) From: Etienne Carriere To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, op-tee@lists.trustedfirmware.org, Sumit Garg , Jens Wiklander , Sudeep Holla , Cristian Marussi , Vincent Guittot , Etienne Carriere Subject: [PATCH v8 2/4] tee: system session Date: Tue, 16 May 2023 14:51:09 +0200 Message-Id: <20230516125111.2690204-3-etienne.carriere@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230516125111.2690204-1-etienne.carriere@linaro.org> References: <20230516125111.2690204-1-etienne.carriere@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230516_055411_148920_80049240 X-CRM114-Status: GOOD ( 17.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Adds kernel client API function tee_client_system_session() for a client to request a system service entry in TEE context. This feature is needed to prevent a system deadlock when several TEE client applications invoke TEE, consuming all TEE thread contexts available in the secure world. The deadlock can happen in the OP-TEE driver for example if all these TEE threads issue an RPC call from TEE to Linux OS to access an eMMC RPMB partition (TEE secure storage) which device clock or regulator controller is accessed through an OP-TEE SCMI services. In that case, Linux SCMI driver must reach OP-TEE SCMI service without waiting until one of the consumed TEE threads is freed. Reviewed-by: Sumit Garg Co-developed-by: Jens Wiklander Signed-off-by: Jens Wiklander Signed-off-by: Etienne Carriere --- No change since v7 Changes since v6: - Fixed commit message from review comments. - Applied Sumit's R-b tag. No change since v5 Changes since v4: - Changes extracted from "[PATCH v4 1/2] tee: system invocation" and revised with Jens contribution to cover only definition of tee driver new API function tee_client_system_session() for kernel clients to register their session as a system session. - Commit message rephrased, including header line changed from "tee: system invocation" to "tee: system session" has the feature relates to system attributes of tee sessions. Changes since v3: - Fixed new SMC funcIDs to reserved/unreserve OP-TEE thread contexts: minor renaming + define as fastcall funcIDs. - Moved system_ctx_count from generic struct tee_context to optee's private struct optee_context_data. This changes optee smc_abi.c to release reserved thread contexts when the optee device is released. - Fixed inline description comments. No change since v2 Change since v1 - Addressed comment on Linux client to claim reservation on TEE context. This brings 2 new operations from client to TEE to request and release system thread contexts: 2 new tee_drv.h API functions, 2 new ops functions in struct tee_driver_ops. The OP-TEE implement shall implement 2 new fastcall SMC funcIDs. - Fixed typos in commit message. --- drivers/tee/tee_core.c | 8 ++++++++ include/linux/tee_drv.h | 16 ++++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/drivers/tee/tee_core.c b/drivers/tee/tee_core.c index 0eb342de0b00..91932835d0f7 100644 --- a/drivers/tee/tee_core.c +++ b/drivers/tee/tee_core.c @@ -1170,6 +1170,14 @@ int tee_client_close_session(struct tee_context *ctx, u32 session) } EXPORT_SYMBOL_GPL(tee_client_close_session); +int tee_client_system_session(struct tee_context *ctx, u32 session) +{ + if (!ctx->teedev->desc->ops->system_session) + return -EINVAL; + return ctx->teedev->desc->ops->system_session(ctx, session); +} +EXPORT_SYMBOL_GPL(tee_client_system_session); + int tee_client_invoke_func(struct tee_context *ctx, struct tee_ioctl_invoke_arg *arg, struct tee_param *param) diff --git a/include/linux/tee_drv.h b/include/linux/tee_drv.h index 17eb1c5205d3..911ddf92dcee 100644 --- a/include/linux/tee_drv.h +++ b/include/linux/tee_drv.h @@ -84,6 +84,7 @@ struct tee_param { * @release: release this open file * @open_session: open a new session * @close_session: close a session + * @system_session: declare session as a system session * @invoke_func: invoke a trusted function * @cancel_req: request cancel of an ongoing invoke or open * @supp_recv: called for supplicant to get a command @@ -100,6 +101,7 @@ struct tee_driver_ops { struct tee_ioctl_open_session_arg *arg, struct tee_param *param); int (*close_session)(struct tee_context *ctx, u32 session); + int (*system_session)(struct tee_context *ctx, u32 session); int (*invoke_func)(struct tee_context *ctx, struct tee_ioctl_invoke_arg *arg, struct tee_param *param); @@ -429,6 +431,20 @@ int tee_client_open_session(struct tee_context *ctx, */ int tee_client_close_session(struct tee_context *ctx, u32 session); +/** + * tee_client_system_session() - Declare session as a system session + * @ctx: TEE Context + * @session: Session id + * + * This function requests TEE to provision an entry context ready to use for + * that session only. The provisioned entry context is used for command + * invocation and session closure, not for command cancelling requests. + * TEE releases the provisioned context upon session closure. + * + * Return < 0 on error else 0 if an entry context has been provisioned. + */ +int tee_client_system_session(struct tee_context *ctx, u32 session); + /** * tee_client_invoke_func() - Invoke a function in a Trusted Application * @ctx: TEE Context From patchwork Tue May 16 12:51:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Etienne Carriere X-Patchwork-Id: 13243126 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A03EBC7EE2A for ; Tue, 16 May 2023 12:54:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=3OuuwdsY4iIk0zKh8dW9rQgqETwafBWOhU3BgHoceNo=; b=JpIYTmdqx1HPc7 SwYkkodP5CcfkiMsuXC8itZhTU9qGwMZ9EpdGi99dt2AbUIFJnONMA1jsm4ORkr2bfKEole1KDAO4 hIxLFlGUdxYaU1CtcXR0IoLrStSk/v/OCGXJBwaheyjUJ19RXbJi4hkbTfBoDidiUDYXLYK7qBHZd HDaT0IOlJYrI3ojzAHaPuf03OsOL/bJMHIV1723X0wb5i1Gc52I234ji/8MRLGgyoMgfkp8vEupxW hT0i4PoKTVRz5Frey5+Ie76dZMV2GEK4tFhavIubSDKcmJ1mE/KdyJihSjd8Fkh5ihI5Siln2wOcV DhKaTOR1xzgjqG7vqZ9A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pyuC3-005nHl-12; Tue, 16 May 2023 12:54:19 +0000 Received: from mail-wr1-x430.google.com ([2a00:1450:4864:20::430]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pyuBx-005nES-0f for linux-arm-kernel@lists.infradead.org; Tue, 16 May 2023 12:54:15 +0000 Received: by mail-wr1-x430.google.com with SMTP id ffacd0b85a97d-307c040797bso4735125f8f.3 for ; Tue, 16 May 2023 05:54:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684241651; x=1686833651; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BCSO+arfjZcxZvXd+bSKUDwehORRd/U6EmwHqbbosEM=; b=U7Ytn8MibfgWAJ2/0PSslBrHKvZFRbWUNXUl7QKfBYc8KAonBbHIU+XXjSr/PB51bG zG7+zuIhfc1psToH7QNaykAZtM8ZxeAbpPoWT2PBRzD+xY5Mx9Em8pkjc+kLd3BTB2L9 epZ5SphlJs8hMgTUYgg94z3oqE4iCNWRLLTiJ5B9DFx/quSvf9BUu89sxqxQ667cLNd8 87q1IuhOEJqc5HCvRv6nxEkB6crsl7+3cVfdgTWFlYs5e660sJIWMg+WVmiiyjx9vcEQ X46Mw+HvIjj3iAUYEHnCPnQ387AlkAfaEypBT8X+rEoGcEUBqbrVPLRurqRLEYq+WE2v 2Vtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684241651; x=1686833651; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BCSO+arfjZcxZvXd+bSKUDwehORRd/U6EmwHqbbosEM=; b=MwFtwJNyNcUHzJbqkj8DoIpC81N+/gv2ynYM/zDVUD6UppxwjJ4thcKtTN+IGfiMpK mI34JWPF8eFK240yDGQrwWqoE0cWySR+zpLHTFOh6MTQC8PFU+6K099z08NRG0PXSVaA pkbDKtgXex9XSRZfBN4Ktf/NS1CiQnt5H59uNf+hprvkWUqjGXZmCvI4lDt3LRRfzG2N p+zINjDv47hgf7Cw+/ZsDnzovzkNhCnfR12MvpZB5eLKQ6sXJllxfElDq1pF6chUFO/L wGVtc55hlarp89+3mnwxVlEWU+cmbQ4AtuGm88DzIaye3v6xpoC9pLeYxKFgM80otOKd uAnw== X-Gm-Message-State: AC+VfDxrPES0MIFK3+d1uRQZeUOSDkuZ5EGhggcR1N59XF9VF1RDWrut CZCSYvj6oymwBLl40+IfUzMjxw== X-Google-Smtp-Source: ACHHUZ5Gtc4sOx1RRsg0N4XFumIUpCnMfW8EGyXyHicv976jyuAgawtSWEF0Ek2FELxlRyNYWetZAQ== X-Received: by 2002:a5d:55c2:0:b0:306:5149:3aa8 with SMTP id i2-20020a5d55c2000000b0030651493aa8mr26129259wrw.24.1684241650960; Tue, 16 May 2023 05:54:10 -0700 (PDT) Received: from lmecxl1178.lme.st.com (87-88-164-239.abo.bbox.fr. [87.88.164.239]) by smtp.gmail.com with ESMTPSA id r18-20020a056000015200b003047d5b8817sm2521963wrx.80.2023.05.16.05.54.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 05:54:10 -0700 (PDT) From: Etienne Carriere To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, op-tee@lists.trustedfirmware.org, Sumit Garg , Jens Wiklander , Sudeep Holla , Cristian Marussi , Vincent Guittot , Etienne Carriere Subject: [PATCH v8 3/4] tee: optee: support tracking system threads Date: Tue, 16 May 2023 14:51:10 +0200 Message-Id: <20230516125111.2690204-4-etienne.carriere@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230516125111.2690204-1-etienne.carriere@linaro.org> References: <20230516125111.2690204-1-etienne.carriere@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230516_055413_247911_F6DD9DA1 X-CRM114-Status: GOOD ( 36.09 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Adds support in the OP-TEE driver to keep track of reserved system threads. The optee_cq_*() functions are updated to handle this if enabled. The SMC ABI part of the driver enables this tracking, but the FF-A ABI part does not. The call queue list optee_call_queue::waiters is replaced by 2 lists, one for regular sessions and one system sessions. The struct also stores count of TEE thread contexts provisioned/used/reserved to track use of TEE resources. The logic allows at most 1 OP-TEE entry can be reserved to TEE system sessions. For sake of simplicity, initialization of call queue management is factorized into new helper function optee_cq_init(). Co-developed-by: Jens Wiklander Signed-off-by: Jens Wiklander Signed-off-by: Etienne Carriere --- Changes since v7: - Changes the logic to reserve at most 1 call entry for system sessions as per patches v6 and v7 discussion threads (the 2 below bullets) and updates commit message accordingly. - Field optee_call_queue::res_sys_thread_count is replaced with 2 fields: sys_thread_req_count and boolean sys_thread_in_use. - Field optee_call_waiter::sys_thread is replaced with 2 fields: sys_thread_req and sys_thread_used. - Adds inline description comments for struct optee_call_queue and struct optee_call_waiter. Changes since v6: - Moved out changes related to adding boolean system thread attribute into optee driver call queue and SMC/FF-A ABIs API functions. These changes were squashed into patch 1/4 of this patch v7 series. - Comment about adding a specific commit for call queue refactoring was not addressed such a patch would only introduce function optee_cq_init() with very little content in (mutex & list init). - Added Co-developed-by tag for Jens contribution as he's not responsible for the changes I made in this patch v7. No change since v5 Changes since v4: - New change that supersedes implementation proposed in PATCH v4 (tee: system invocation"). Thanks to Jens implementation we don't need the new OP-TEE services that my previous patch versions introduced to monitor system threads entry. Now, Linux optee SMC ABI driver gets TEE provisioned thread contexts count once and monitors thread entries in OP-TEE on that basis and the system thread capability of the related tee session. By the way, I dropped the WARN_ONCE() call I suggested on tee thread exhaustion as it does not provides useful information. --- drivers/tee/optee/call.c | 131 ++++++++++++++++++++++++++++-- drivers/tee/optee/ffa_abi.c | 3 +- drivers/tee/optee/optee_private.h | 30 ++++++- drivers/tee/optee/smc_abi.c | 16 +++- 4 files changed, 167 insertions(+), 13 deletions(-) diff --git a/drivers/tee/optee/call.c b/drivers/tee/optee/call.c index 152ae9bb1785..7aec64f0f7e0 100644 --- a/drivers/tee/optee/call.c +++ b/drivers/tee/optee/call.c @@ -39,9 +39,28 @@ struct optee_shm_arg_entry { DECLARE_BITMAP(map, MAX_ARG_COUNT_PER_ENTRY); }; +void optee_cq_init(struct optee_call_queue *cq, int thread_count) +{ + mutex_init(&cq->mutex); + INIT_LIST_HEAD(&cq->normal_waiters); + INIT_LIST_HEAD(&cq->sys_waiters); + /* + * If cq->total_thread_count is 0 then we're not trying to keep + * track of how many free threads we have, instead we're relying on + * the secure world to tell us when we're out of thread and have to + * wait for another thread to become available. + */ + cq->total_thread_count = thread_count; + cq->free_normal_thread_count = thread_count; +} + void optee_cq_wait_init(struct optee_call_queue *cq, struct optee_call_waiter *w, bool sys_thread) { + bool need_wait = false; + + memset(w, 0, sizeof(*w)); + /* * We're preparing to make a call to secure world. In case we can't * allocate a thread in secure world we'll end up waiting in @@ -53,15 +72,46 @@ void optee_cq_wait_init(struct optee_call_queue *cq, mutex_lock(&cq->mutex); /* - * We add ourselves to the queue, but we don't wait. This - * guarantees that we don't lose a completion if secure world - * returns busy and another thread just exited and try to complete - * someone. + * We add ourselves to a queue, but we don't wait. This guarantees + * that we don't lose a completion if secure world returns busy and + * another thread just exited and try to complete someone. */ init_completion(&w->c); - list_add_tail(&w->list_node, &cq->waiters); + w->sys_thread_req = sys_thread; + if (sys_thread) { + list_add_tail(&w->list_node, &cq->sys_waiters); + if (cq->sys_thread_in_use) { + need_wait = true; + } else { + cq->sys_thread_in_use = true; + w->sys_thread_used = true; + } + } else { + list_add_tail(&w->list_node, &cq->normal_waiters); + if (cq->total_thread_count) { + /* + * Claim a normal thread if one is available, else + * we'll need to wait for a normal thread to be + * released. + */ + if (cq->free_normal_thread_count > 0) + cq->free_normal_thread_count--; + else + need_wait = true; + } + } mutex_unlock(&cq->mutex); + + while (need_wait) { + optee_cq_wait_for_completion(cq, w); + mutex_lock(&cq->mutex); + if (cq->free_normal_thread_count > 0) { + cq->free_normal_thread_count--; + need_wait = false; + } + mutex_unlock(&cq->mutex); + } } void optee_cq_wait_for_completion(struct optee_call_queue *cq, @@ -74,7 +124,10 @@ void optee_cq_wait_for_completion(struct optee_call_queue *cq, /* Move to end of list to get out of the way for other waiters */ list_del(&w->list_node); reinit_completion(&w->c); - list_add_tail(&w->list_node, &cq->waiters); + if (w->sys_thread_req) + list_add_tail(&w->list_node, &cq->sys_waiters); + else + list_add_tail(&w->list_node, &cq->normal_waiters); mutex_unlock(&cq->mutex); } @@ -83,10 +136,19 @@ static void optee_cq_complete_one(struct optee_call_queue *cq) { struct optee_call_waiter *w; - list_for_each_entry(w, &cq->waiters, list_node) { + list_for_each_entry(w, &cq->sys_waiters, list_node) { if (!completion_done(&w->c)) { complete(&w->c); - break; + return; + } + } + + if (!cq->total_thread_count || cq->free_normal_thread_count > 0) { + list_for_each_entry(w, &cq->normal_waiters, list_node) { + if (!completion_done(&w->c)) { + complete(&w->c); + break; + } } } } @@ -104,6 +166,11 @@ void optee_cq_wait_final(struct optee_call_queue *cq, /* Get out of the list */ list_del(&w->list_node); + if (w->sys_thread_used) + cq->sys_thread_in_use = false; + else + cq->free_normal_thread_count++; /* Release a normal thread */ + /* Wake up one eventual waiting task */ optee_cq_complete_one(cq); @@ -119,6 +186,31 @@ void optee_cq_wait_final(struct optee_call_queue *cq, mutex_unlock(&cq->mutex); } +bool optee_cq_inc_sys_thread_count(struct optee_call_queue *cq) +{ + if (cq->total_thread_count <= 1) + return false; + + mutex_lock(&cq->mutex); + if (!cq->sys_thread_req_count) + cq->free_normal_thread_count--; + cq->sys_thread_req_count++; + mutex_unlock(&cq->mutex); + + return true; +} + +void optee_cq_dec_sys_thread_count(struct optee_call_queue *cq) +{ + mutex_lock(&cq->mutex); + cq->sys_thread_req_count--; + if (!cq->sys_thread_req_count) + cq->free_normal_thread_count++; + /* If there's someone waiting, let it resume */ + optee_cq_complete_one(cq); + mutex_unlock(&cq->mutex); +} + /* Requires the filpstate mutex to be held */ static struct optee_session *find_session(struct optee_context_data *ctxdata, u32 session_id) @@ -361,6 +453,27 @@ int optee_open_session(struct tee_context *ctx, return rc; } +int optee_system_session(struct tee_context *ctx, u32 session) +{ + struct optee_context_data *ctxdata = ctx->data; + struct optee *optee = tee_get_drvdata(ctx->teedev); + struct optee_session *sess; + int rc = -EINVAL; + + mutex_lock(&ctxdata->mutex); + + sess = find_session(ctxdata, session); + if (sess && !sess->use_sys_thread && + optee_cq_inc_sys_thread_count(&optee->call_queue)) { + rc = 0; + sess->use_sys_thread = true; + } + + mutex_unlock(&ctxdata->mutex); + + return rc; +} + int optee_close_session_helper(struct tee_context *ctx, u32 session, bool system_thread) { @@ -378,6 +491,8 @@ int optee_close_session_helper(struct tee_context *ctx, u32 session, msg_arg->session = session; optee->ops->do_call_with_arg(ctx, shm, offs, system_thread); + if (system_thread) + optee_cq_dec_sys_thread_count(&optee->call_queue); optee_free_msg_arg(ctx, entry, offs); return 0; diff --git a/drivers/tee/optee/ffa_abi.c b/drivers/tee/optee/ffa_abi.c index 5fde9d4100e3..0c9055691343 100644 --- a/drivers/tee/optee/ffa_abi.c +++ b/drivers/tee/optee/ffa_abi.c @@ -852,8 +852,7 @@ static int optee_ffa_probe(struct ffa_device *ffa_dev) if (rc) goto err_unreg_supp_teedev; mutex_init(&optee->ffa.mutex); - mutex_init(&optee->call_queue.mutex); - INIT_LIST_HEAD(&optee->call_queue.waiters); + optee_cq_init(&optee->call_queue, 0); optee_supp_init(&optee->supp); optee_shm_arg_cache_init(optee, arg_cache_flags); ffa_dev_set_drvdata(ffa_dev, optee); diff --git a/drivers/tee/optee/optee_private.h b/drivers/tee/optee/optee_private.h index b68273051454..b8f22d7b34a6 100644 --- a/drivers/tee/optee/optee_private.h +++ b/drivers/tee/optee/optee_private.h @@ -40,15 +40,39 @@ typedef void (optee_invoke_fn)(unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, struct arm_smccc_res *); +/* + * struct optee_call_waiter - TEE entry may need to wait for a free TEE thread + * @list_node Reference in waiters list + * @c Waiting completion reference + * @sys_thread_req True only if waiter belongs to a requested system thread + * @sys_thread_used True only if waiter is consuming the system thread + */ struct optee_call_waiter { struct list_head list_node; struct completion c; + bool sys_thread_req; + bool sys_thread_used; }; +/* + * struct optee_call_queue - OP-TEE call queue management + * @mutex Serializes access to this struct + * @normal_waiters List of normal threads waiting to enter OP-TEE + * @sys_waiters List of system threads waiting to enter OP-TEE + * @total_thread_count Overall number of thread context in OP-TEE or 0 + * @free_normal_thread_count Number of normal threads context free in OP-TEE + * @sys_thread_req_count Number of registered system thread requests + * @sys_thread_in_use True only if reserved system entry is consumed + */ struct optee_call_queue { /* Serializes access to this struct */ struct mutex mutex; - struct list_head waiters; + struct list_head normal_waiters; + struct list_head sys_waiters; + int total_thread_count; + int free_normal_thread_count; + int sys_thread_req_count; + bool sys_thread_in_use; }; struct optee_notif { @@ -254,6 +278,7 @@ int optee_supp_send(struct tee_context *ctx, u32 ret, u32 num_params, int optee_open_session(struct tee_context *ctx, struct tee_ioctl_open_session_arg *arg, struct tee_param *param); +int optee_system_session(struct tee_context *ctx, u32 session); int optee_close_session_helper(struct tee_context *ctx, u32 session, bool system_thread); int optee_close_session(struct tee_context *ctx, u32 session); @@ -303,6 +328,9 @@ static inline void optee_to_msg_param_value(struct optee_msg_param *mp, mp->u.value.c = p->u.value.c; } +void optee_cq_init(struct optee_call_queue *cq, int thread_count); +bool optee_cq_inc_sys_thread_count(struct optee_call_queue *cq); +void optee_cq_dec_sys_thread_count(struct optee_call_queue *cq); void optee_cq_wait_init(struct optee_call_queue *cq, struct optee_call_waiter *w, bool sys_thread); void optee_cq_wait_for_completion(struct optee_call_queue *cq, diff --git a/drivers/tee/optee/smc_abi.c b/drivers/tee/optee/smc_abi.c index e2763cdcf111..3314ffeb91c8 100644 --- a/drivers/tee/optee/smc_abi.c +++ b/drivers/tee/optee/smc_abi.c @@ -1209,6 +1209,7 @@ static const struct tee_driver_ops optee_clnt_ops = { .release = optee_release, .open_session = optee_open_session, .close_session = optee_close_session, + .system_session = optee_system_session, .invoke_func = optee_invoke_func, .cancel_req = optee_cancel_req, .shm_register = optee_shm_register, @@ -1356,6 +1357,16 @@ static bool optee_msg_exchange_capabilities(optee_invoke_fn *invoke_fn, return true; } +static unsigned int optee_msg_get_thread_count(optee_invoke_fn *invoke_fn) +{ + struct arm_smccc_res res; + + invoke_fn(OPTEE_SMC_GET_THREAD_COUNT, 0, 0, 0, 0, 0, 0, 0, &res); + if (res.a0) + return 0; + return res.a1; +} + static struct tee_shm_pool * optee_config_shm_memremap(optee_invoke_fn *invoke_fn, void **memremaped_shm) { @@ -1609,6 +1620,7 @@ static int optee_probe(struct platform_device *pdev) struct optee *optee = NULL; void *memremaped_shm = NULL; unsigned int rpc_param_count; + unsigned int thread_count; struct tee_device *teedev; struct tee_context *ctx; u32 max_notif_value; @@ -1636,6 +1648,7 @@ static int optee_probe(struct platform_device *pdev) return -EINVAL; } + thread_count = optee_msg_get_thread_count(invoke_fn); if (!optee_msg_exchange_capabilities(invoke_fn, &sec_caps, &max_notif_value, &rpc_param_count)) { @@ -1725,8 +1738,7 @@ static int optee_probe(struct platform_device *pdev) if (rc) goto err_unreg_supp_teedev; - mutex_init(&optee->call_queue.mutex); - INIT_LIST_HEAD(&optee->call_queue.waiters); + optee_cq_init(&optee->call_queue, thread_count); optee_supp_init(&optee->supp); optee->smc.memremaped_shm = memremaped_shm; optee->pool = pool; From patchwork Tue May 16 12:51:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Etienne Carriere X-Patchwork-Id: 13243127 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D47EBC77B7A for ; Tue, 16 May 2023 12:54:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=5iOls1SRgSLM+6nksX0/r64usHMJY9ZvBcbrErHX30Y=; b=BkefPp7jOKTfr1 7mSbxZWkgAQRz0KBMJ2iip/YmEBeDO78RPRgQ8yoO4WNwTDRO8KLKQ7L7RFSvogFfP6srQWYH/3zX E2LZ16aUc4bsK4vfEIOqEf5CBPoH+7Xds1knCBiH5sH2TuJU1Z5hBcO+MXu5w/M2RUEb/sJvfykWK NHNMQhFpJTJmViUP3sQPvNFRbP5Dqs/Bu6Vi/EBWGTdBxJi/ZJZXKyewawMmaRmt1mb7NpeYyKKIK 4XKe1Qg94x1jIhnQ1aD9xlMIhkT8MMVQ7Xediial8MOXLT+mfqMx+yuy6H8Pch6W3RPfaYhO5FxlL anWmnYwmVy1fVtgYkKSw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pyuC3-005nIJ-2n; Tue, 16 May 2023 12:54:19 +0000 Received: from mail-wm1-x333.google.com ([2a00:1450:4864:20::333]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pyuBy-005nEq-1F for linux-arm-kernel@lists.infradead.org; Tue, 16 May 2023 12:54:15 +0000 Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-3f4ad71b00eso45516585e9.2 for ; Tue, 16 May 2023 05:54:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684241652; x=1686833652; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7B0ppPlpnqNlvLrDWw6A691nD5+RAL8Y7RlKrfxQcUs=; b=i8DkuUNO7jqjnmiTGpWShOI4F0dtKSFavowIVOFBQrBm5+jxryTOBAS8rlEesuGMuy Qil9RJA8Lfy+5jM+359EgQtVOYhrrusDTtOdtYLYuPcWl7wC4WJv87bE5WczG86Mpmsa jE3AUTGKfvtldAqz53Zx9pWEuGZdP4tsvoE+B2htbfcQS1egDBkBKz4YehQRJBapA6Jc rdrIvOyYpZJyftD0L5JxenLTaB/rrMcHeZW1H7D7kfLj81+/V/bXnCKF/RCU7Wt/NQlh dOc1Dr46Swod5TiQLagjJk9D4Xqrp2yqOW2z5jaFnmUddGH/PmRGx4AABMvrE4XJoXhj wRfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684241652; x=1686833652; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7B0ppPlpnqNlvLrDWw6A691nD5+RAL8Y7RlKrfxQcUs=; b=EhGn2nZDQ27Df5edBlPh6TC48zz0kWjQ7nGPUwrvqS8KP94Pm0uKVJwmpM/9HMpXUX PPAUPHox9IDHhwvMfbUnrwAzJo9HmmvfGvZPkyTz112EAsBbGFifbrZAMii3RyE6mMaH 2gZOK39qeBEL/z9mEVl+9m2equSwSWvjf1D5mj+b2Qrt01009FY/uUgdg84g8sOELKre acMnCQYjTsgcSDcVy8IN0eY7xv3IzltEAvXzQS7uPBkGEy+uZ3wuOHRVbWYN/J3c8HP8 /81s+vx5NgT83Kws/OqvZi5h27e//bhnrwHcVJ5ECSUXKhe2TtQtcKzbxTVUsTl/0h6U 5KEg== X-Gm-Message-State: AC+VfDwWwqVUe+wkvIoWGNUXBJHaWXa3sBxoEB8gty9cwBPR0JwWbWWC 0n6QsntphwPQY3AwTPUslydCRA== X-Google-Smtp-Source: ACHHUZ50KLhm96H8lf/YBH4BlkA2lKLNNueGedVaZlua9xQGiWfUfsr0vxjrKcq0uJq9oJPYm6tnmw== X-Received: by 2002:a5d:4684:0:b0:309:3698:7feb with SMTP id u4-20020a5d4684000000b0030936987febmr1024123wrq.46.1684241652102; Tue, 16 May 2023 05:54:12 -0700 (PDT) Received: from lmecxl1178.lme.st.com (87-88-164-239.abo.bbox.fr. [87.88.164.239]) by smtp.gmail.com with ESMTPSA id r18-20020a056000015200b003047d5b8817sm2521963wrx.80.2023.05.16.05.54.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 May 2023 05:54:11 -0700 (PDT) From: Etienne Carriere To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, op-tee@lists.trustedfirmware.org, Sumit Garg , Jens Wiklander , Sudeep Holla , Cristian Marussi , Vincent Guittot , Etienne Carriere Subject: [PATCH v8 4/4] firmware: arm_scmi: optee: use optee system invocation Date: Tue, 16 May 2023 14:51:11 +0200 Message-Id: <20230516125111.2690204-5-etienne.carriere@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230516125111.2690204-1-etienne.carriere@linaro.org> References: <20230516125111.2690204-1-etienne.carriere@linaro.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230516_055414_423872_6112BC15 X-CRM114-Status: GOOD ( 13.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Changes SCMI optee transport to call tee_client_system_session() to request optee driver to provision an entry context in OP-TEE for processing OP-TEE messages. This prevents possible deadlock in case OP-TEE threads are all consumed while these may be waiting for a clock or regulator to be enable which SCMI OP-TEE service which requires a free thread context to execute. Acked-by: Sudeep Holla Signed-off-by: Etienne Carriere Reviewed-by: Sumit Garg --- No change since v7 No change since v6 Changes since v5: - Applied Sudeep's review tag Changes since v4: - Updated to new API function tee_client_system_session() introduced in patch v5 2/3. No change since v3 Changes since v2: - Fixed syntax issues (missing ';' chars), reported by kernel test robot. Changes since v1: - Updated to use new tee API functions tee_client_request_system_context() and tee_client_release_system_context(). --- drivers/firmware/arm_scmi/optee.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/firmware/arm_scmi/optee.c b/drivers/firmware/arm_scmi/optee.c index e123de6e8c67..25bfb465484d 100644 --- a/drivers/firmware/arm_scmi/optee.c +++ b/drivers/firmware/arm_scmi/optee.c @@ -440,6 +440,10 @@ static int scmi_optee_chan_setup(struct scmi_chan_info *cinfo, struct device *de if (ret) goto err_free_shm; + ret = tee_client_system_session(scmi_optee_private->tee_ctx, channel->tee_session); + if (ret) + dev_warn(dev, "Could not switch to system session, do best effort\n"); + ret = get_channel(channel); if (ret) goto err_close_sess;