From patchwork Wed Feb 1 16:34:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florent Revest X-Patchwork-Id: 13124712 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1837AC38142 for ; Wed, 1 Feb 2023 16:35:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232514AbjBAQfs (ORCPT ); Wed, 1 Feb 2023 11:35:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41078 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232432AbjBAQfq (ORCPT ); Wed, 1 Feb 2023 11:35:46 -0500 Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A4A0B7A482 for ; Wed, 1 Feb 2023 08:35:29 -0800 (PST) Received: by mail-wm1-x32d.google.com with SMTP id bg26so7277106wmb.0 for ; Wed, 01 Feb 2023 08:35:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=p0B1nR0feNzt6iYs1fcCSNwhqATraAQZbyN0DagmRgs=; b=jcn6cTs4+XY1hUWmIDmmwPG44YIXWbCtGh35J88AOeps79NC6Aub1gKAPPv419PyFm HTPegDj2V+wQXQ7DcQvVkjHumtHWHz7verUSkgH7IJrJQQz1clTf1SwWQ3717K7f1cqs BiPYS75krdieSyLXCxa9QTp4cnoHXv5nKh5HI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p0B1nR0feNzt6iYs1fcCSNwhqATraAQZbyN0DagmRgs=; b=m4i19MW+nFOaduhAA7rwjWv0V2LHE6Dbxgn+oy/BqNtHUHmhvIzIo1a72OzkJwf4EZ ycWRkU+3miyxYPEo6piREfv4iZyoblhL1ZUF4XcNHfBK9n3f3cRG741EQkoM80FxHA5w R6ANNZguPTli7yjPFciG2j3XZCe0dA7vBemSFb7+oky0QpsXXAL29SfHapTDO+f28LJl Lqs/xY/xvjL19eZwbDowT8cjkStYuGzWvWKdmSOmGCl9eu1ZCuhZWuelVLdH2iNzzfQf m2o/XTH2q1TbLwVQvc2uz9d8GTisOztVsnOMWX+H9HMBUCoEm5djJd4eAP7Fd/r3Tyhq 1avw== X-Gm-Message-State: AO0yUKWDy+8ZFnaKOMGXCulAxHBsqE8O2oKTSXrxBujb4iAjjixDPuiz 8qrd7qhkkhnPPOo0yVdXiuD7Ug== X-Google-Smtp-Source: AK7set9sCy9RhmFzbGR11y6SAdcOcsShJEwbhl2h5tKcFqmeH9CrjXQjVjAY0i+Q+yzsszXf1HIeKQ== X-Received: by 2002:a05:600c:491c:b0:3d9:ee01:60a4 with SMTP id f28-20020a05600c491c00b003d9ee0160a4mr2827768wmp.1.1675269328196; Wed, 01 Feb 2023 08:35:28 -0800 (PST) Received: from revest.zrh.corp.google.com ([2a00:79e0:9d:6:4399:89a1:4a86:9630]) by smtp.gmail.com with ESMTPSA id r38-20020a05600c322600b003dd7edcc960sm2058522wmp.45.2023.02.01.08.35.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Feb 2023 08:35:27 -0800 (PST) From: Florent Revest To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org Cc: catalin.marinas@arm.com, will@kernel.org, rostedt@goodmis.org, mhiramat@kernel.org, mark.rutland@arm.com, ast@kernel.org, daniel@iogearbox.net, andrii@kernel.org, kpsingh@kernel.org, jolsa@kernel.org, xukuohai@huaweicloud.com, Florent Revest Subject: [PATCH 4/8] ftrace: Store direct called addresses in their ops Date: Wed, 1 Feb 2023 17:34:16 +0100 Message-Id: <20230201163420.1579014-5-revest@chromium.org> X-Mailer: git-send-email 2.39.1.519.gcb327c4b5f-goog In-Reply-To: <20230201163420.1579014-1-revest@chromium.org> References: <20230201163420.1579014-1-revest@chromium.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-trace-kernel@vger.kernel.org All direct calls are now registered using the register_ftrace_direct API so each ops can jump to only one direct-called trampoline. By storing the direct called trampoline address directly in the ops we can save one hashmap lookup in the direct call ops and implement arm64 direct calls on top of call ops. Signed-off-by: Florent Revest --- include/linux/ftrace.h | 3 +++ kernel/trace/ftrace.c | 6 ++++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h index a7dbd307c3a4..84f717f8959e 100644 --- a/include/linux/ftrace.h +++ b/include/linux/ftrace.h @@ -321,6 +321,9 @@ struct ftrace_ops { unsigned long trampoline_size; struct list_head list; ftrace_ops_func_t ops_func; +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS + unsigned long direct_call; +#endif #endif }; diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index cb77a0a208c7..b0426de11c45 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -2577,9 +2577,8 @@ ftrace_add_rec_direct(unsigned long ip, unsigned long addr, static void call_direct_funcs(unsigned long ip, unsigned long pip, struct ftrace_ops *ops, struct ftrace_regs *fregs) { - unsigned long addr; + unsigned long addr = ops->direct_call; - addr = ftrace_find_rec_direct(ip); if (!addr) return; @@ -5375,6 +5374,7 @@ int register_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) ops->func = call_direct_funcs; ops->flags = MULTI_FLAGS; ops->trampoline = FTRACE_REGS_ADDR; + ops->direct_call = addr; err = register_ftrace_function_nolock(ops); @@ -5445,6 +5445,7 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) /* Enable the tmp_ops to have the same functions as the direct ops */ ftrace_ops_init(&tmp_ops); tmp_ops.func_hash = ops->func_hash; + tmp_ops.direct_call = addr; err = register_ftrace_function_nolock(&tmp_ops); if (err) @@ -5466,6 +5467,7 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) entry->direct = addr; } } + ops->direct_call = addr; mutex_unlock(&ftrace_lock);