From patchwork Sat Aug 21 02:58:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Marchevsky X-Patchwork-Id: 12450783 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C48F0C4338F for ; Sat, 21 Aug 2021 02:59:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AD14C61186 for ; Sat, 21 Aug 2021 02:59:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240984AbhHUDAH (ORCPT ); Fri, 20 Aug 2021 23:00:07 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:27114 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S241015AbhHUDAH (ORCPT ); Fri, 20 Aug 2021 23:00:07 -0400 Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.16.0.43/8.16.0.43) with SMTP id 17L2wFeB009293 for ; Fri, 20 Aug 2021 19:59:28 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=GQ9Jk4cKjDo5n6s12a91EWYn3PSX8lc5zqf/wOLJ20A=; b=i8WWi80l0DgnENQcNAnrLiQk4YrA6Vv+Q2Uy9q2k6rYvXLQGLqWOjzRrhLaZWha4YmDW vV9ArOtlaWnb+Glv98y0MvWsFUifxN/Vmz10PIABx6Ji0VPsR7QcbDWqDOXCCasxUfg9 b8OuoH7+OgJaiGpi9ihJj5VbsLnXOunc3kY= Received: from mail.thefacebook.com ([163.114.132.120]) by m0001303.ppops.net with ESMTP id 3ajkg09gkf-16 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 Aug 2021 19:59:28 -0700 Received: from intmgw001.38.frc1.facebook.com (2620:10d:c085:208::11) by mail.thefacebook.com (2620:10d:c085:11d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Fri, 20 Aug 2021 19:59:02 -0700 Received: by devbig030.frc3.facebook.com (Postfix, from userid 158236) id 1D3AC5730093; Fri, 20 Aug 2021 19:58:58 -0700 (PDT) From: Dave Marchevsky To: CC: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Yonghong Song , Florent Revest , , , Dave Marchevsky Subject: [PATCH bpf-next 1/5] bpf: merge printk and seq_printf VARARG max macros Date: Fri, 20 Aug 2021 19:58:33 -0700 Message-ID: <20210821025837.1614098-2-davemarchevsky@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210821025837.1614098-1-davemarchevsky@fb.com> References: <20210821025837.1614098-1-davemarchevsky@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-FB-Source: Intern X-Proofpoint-GUID: 1mIPFd_BfhuKar1IPSjdAo9V5zjoINlL X-Proofpoint-ORIG-GUID: 1mIPFd_BfhuKar1IPSjdAo9V5zjoINlL X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.790 definitions=2021-08-20_11:2021-08-20,2021-08-20 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 malwarescore=0 phishscore=0 priorityscore=1501 bulkscore=0 lowpriorityscore=0 suspectscore=0 impostorscore=0 adultscore=0 clxscore=1015 mlxscore=0 spamscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108210017 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org MAX_SNPRINTF_VARARGS and MAX_SEQ_PRINTF_VARARGS are used by bpf helpers bpf_snprintf and bpf_seq_printf to limit their varargs. Both call into bpf_bprintf_prepare for print formatting logic and have convenience macros in libbpf (BPF_SNPRINTF, BPF_SEQ_PRINTF) which use the same helper macros to convert varargs to a byte array. Changing shared functionality to support more varargs for either bpf helper would affect the other as well, so let's combine the _VARARGS macros to make this more obvious. Signed-off-by: Dave Marchevsky Acked-by: Andrii Nakryiko --- include/linux/bpf.h | 2 ++ kernel/bpf/helpers.c | 4 +--- kernel/trace/bpf_trace.c | 4 +--- 3 files changed, 4 insertions(+), 6 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index f4c16f19f83e..be8d57e6e78a 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -2216,6 +2216,8 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type t, struct btf_id_set; bool btf_id_set_contains(const struct btf_id_set *set, u32 id); +#define MAX_BPRINTF_VARARGS 12 + int bpf_bprintf_prepare(char *fmt, u32 fmt_size, const u64 *raw_args, u32 **bin_buf, u32 num_args); void bpf_bprintf_cleanup(void); diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 4e8540716187..5ce19b376ef7 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -969,15 +969,13 @@ int bpf_bprintf_prepare(char *fmt, u32 fmt_size, const u64 *raw_args, return err; } -#define MAX_SNPRINTF_VARARGS 12 - BPF_CALL_5(bpf_snprintf, char *, str, u32, str_size, char *, fmt, const void *, data, u32, data_len) { int err, num_args; u32 *bin_args; - if (data_len % 8 || data_len > MAX_SNPRINTF_VARARGS * 8 || + if (data_len % 8 || data_len > MAX_BPRINTF_VARARGS * 8 || (data_len && !data)) return -EINVAL; num_args = data_len / 8; diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index cbc73c08c4a4..2cf4bfa1ab7b 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -414,15 +414,13 @@ const struct bpf_func_proto *bpf_get_trace_printk_proto(void) return &bpf_trace_printk_proto; } -#define MAX_SEQ_PRINTF_VARARGS 12 - BPF_CALL_5(bpf_seq_printf, struct seq_file *, m, char *, fmt, u32, fmt_size, const void *, data, u32, data_len) { int err, num_args; u32 *bin_args; - if (data_len & 7 || data_len > MAX_SEQ_PRINTF_VARARGS * 8 || + if (data_len & 7 || data_len > MAX_BPRINTF_VARARGS * 8 || (data_len && !data)) return -EINVAL; num_args = data_len / 8; From patchwork Sat Aug 21 02:58:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Marchevsky X-Patchwork-Id: 12450777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 063BFC4320E for ; Sat, 21 Aug 2021 02:59:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D9AF3610A3 for ; Sat, 21 Aug 2021 02:59:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231847AbhHUC7z (ORCPT ); Fri, 20 Aug 2021 22:59:55 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:32688 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240908AbhHUC7z (ORCPT ); Fri, 20 Aug 2021 22:59:55 -0400 Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17L2x2P4021820 for ; Fri, 20 Aug 2021 19:59:16 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=QPZzqQRKwoIbH95usX2efup9VFGOQGW0wxULxFuDsXc=; b=S+HfmKasR5WJfdfyvkbbh51P5Q/hdLhBXpZv4Xe+2B15lBqGaZU4qEfblovXCvG+pxof ZG7qeHbGIUjquxztdvTLmt4suqJ28tkuu7DIUjBlUyzaWseELEFhhiZkJFvcTISv/z+2 cGDP/O1c3xQMJpLsbDOpfEwQt2ukjz2ztLM= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com with ESMTP id 3ahxat0x3b-6 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 Aug 2021 19:59:16 -0700 Received: from intmgw002.25.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::f) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Fri, 20 Aug 2021 19:59:13 -0700 Received: by devbig030.frc3.facebook.com (Postfix, from userid 158236) id 05B1F57300AC; Fri, 20 Aug 2021 19:59:07 -0700 (PDT) From: Dave Marchevsky To: CC: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Yonghong Song , Florent Revest , , , Dave Marchevsky Subject: [PATCH bpf-next 2/5] bpf: add bpf_trace_vprintk helper Date: Fri, 20 Aug 2021 19:58:34 -0700 Message-ID: <20210821025837.1614098-3-davemarchevsky@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210821025837.1614098-1-davemarchevsky@fb.com> References: <20210821025837.1614098-1-davemarchevsky@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-FB-Source: Intern X-Proofpoint-GUID: J3ZbP5l1f1EYfNLe1KBagR84eeMt7z0E X-Proofpoint-ORIG-GUID: J3ZbP5l1f1EYfNLe1KBagR84eeMt7z0E X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.790 definitions=2021-08-20_11:2021-08-20,2021-08-20 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 spamscore=0 suspectscore=0 mlxlogscore=999 mlxscore=0 adultscore=0 phishscore=0 lowpriorityscore=0 impostorscore=0 clxscore=1015 malwarescore=0 priorityscore=1501 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108210017 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org This helper is meant to be "bpf_trace_printk, but with proper vararg support". Follow bpf_snprintf's example and take a u64 pseudo-vararg array. Write to dmesg using the same mechanism as bpf_trace_printk. Signed-off-by: Dave Marchevsky --- include/linux/bpf.h | 1 + include/uapi/linux/bpf.h | 23 +++++++++++++++ kernel/bpf/core.c | 5 ++++ kernel/bpf/helpers.c | 2 ++ kernel/trace/bpf_trace.c | 52 +++++++++++++++++++++++++++++++++- tools/include/uapi/linux/bpf.h | 23 +++++++++++++++ 6 files changed, 105 insertions(+), 1 deletion(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index be8d57e6e78a..b6c45a6cbbba 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1088,6 +1088,7 @@ bool bpf_prog_array_compatible(struct bpf_array *array, const struct bpf_prog *f int bpf_prog_calc_tag(struct bpf_prog *fp); const struct bpf_func_proto *bpf_get_trace_printk_proto(void); +const struct bpf_func_proto *bpf_get_trace_vprintk_proto(void); typedef unsigned long (*bpf_ctx_copy_t)(void *dst, const void *src, unsigned long off, unsigned long len); diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index c4f7892edb2b..899a2649d986 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -4871,6 +4871,28 @@ union bpf_attr { * Return * Value specified by user at BPF link creation/attachment time * or 0, if it was not specified. + * + * u64 bpf_trace_vprintk(const char *fmt, u32 fmt_size, const void *data, u32 data_len) + * Description + * Behaves like **bpf_trace_printk**\ () helper, but takes an array of u64 + * to format. Supports up to 12 arguments to print in this way. + * The *fmt* and *fmt_size* are for the format string itself. The *data* and + * *data_len* are format string arguments. + * + * Each format specifier in **fmt** corresponds to one u64 element + * in the **data** array. For strings and pointers where pointees + * are accessed, only the pointer values are stored in the *data* + * array. The *data_len* is the size of *data* in bytes. + * Formats **%s**, **%p{i,I}{4,6}** requires to read kernel memory. + * Reading kernel memory may fail due to either invalid address or + * valid address but requiring a major memory fault. If reading kernel memory + * fails, the string for **%s** will be an empty string, and the ip + * address for **%p{i,I}{4,6}** will be 0. Not returning error to + * bpf program is consistent with what **bpf_trace_printk**\ () does for now. + * + * Return + * The number of bytes written to the buffer, or a negative error + * in case of failure. */ #define __BPF_FUNC_MAPPER(FN) \ FN(unspec), \ @@ -5048,6 +5070,7 @@ union bpf_attr { FN(timer_cancel), \ FN(get_func_ip), \ FN(get_attach_cookie), \ + FN(trace_vprintk), \ /* */ /* integer value in 'imm' field of BPF_CALL instruction selects which helper diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 91f24c7b38a1..a137c550046c 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -2357,6 +2357,11 @@ const struct bpf_func_proto * __weak bpf_get_trace_printk_proto(void) return NULL; } +const struct bpf_func_proto * __weak bpf_get_trace_vprintk_proto(void) +{ + return NULL; +} + u64 __weak bpf_event_output(struct bpf_map *map, u64 flags, void *meta, u64 meta_size, void *ctx, u64 ctx_size, bpf_ctx_copy_t ctx_copy) diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 5ce19b376ef7..863e5ee68558 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -1419,6 +1419,8 @@ bpf_base_func_proto(enum bpf_func_id func_id) return &bpf_snprintf_btf_proto; case BPF_FUNC_snprintf: return &bpf_snprintf_proto; + case BPF_FUNC_trace_vprintk: + return bpf_get_trace_vprintk_proto(); default: return NULL; } diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index 2cf4bfa1ab7b..8b3f1ec9e082 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -398,7 +398,7 @@ static const struct bpf_func_proto bpf_trace_printk_proto = { .arg2_type = ARG_CONST_SIZE, }; -const struct bpf_func_proto *bpf_get_trace_printk_proto(void) +static __always_inline void __set_printk_clr_event(void) { /* * This program might be calling bpf_trace_printk, @@ -410,10 +410,58 @@ const struct bpf_func_proto *bpf_get_trace_printk_proto(void) */ if (trace_set_clr_event("bpf_trace", "bpf_trace_printk", 1)) pr_warn_ratelimited("could not enable bpf_trace_printk events"); +} +const struct bpf_func_proto *bpf_get_trace_printk_proto(void) +{ + __set_printk_clr_event(); return &bpf_trace_printk_proto; } +BPF_CALL_4(bpf_trace_vprintk, char *, fmt, u32, fmt_size, const void *, data, + u32, data_len) +{ + static char buf[BPF_TRACE_PRINTK_SIZE]; + unsigned long flags; + int ret, num_args; + u32 *bin_args; + + if (data_len & 7 || data_len > MAX_BPRINTF_VARARGS * 8 || + (data_len && !data)) + return -EINVAL; + num_args = data_len / 8; + + ret = bpf_bprintf_prepare(fmt, fmt_size, data, &bin_args, num_args); + if (ret < 0) + return ret; + + raw_spin_lock_irqsave(&trace_printk_lock, flags); + ret = bstr_printf(buf, sizeof(buf), fmt, bin_args); + + trace_bpf_trace_printk(buf); + raw_spin_unlock_irqrestore(&trace_printk_lock, flags); + + bpf_bprintf_cleanup(); + + return ret; +} + +static const struct bpf_func_proto bpf_trace_vprintk_proto = { + .func = bpf_trace_vprintk, + .gpl_only = true, + .ret_type = RET_INTEGER, + .arg1_type = ARG_PTR_TO_MEM, + .arg2_type = ARG_CONST_SIZE, + .arg3_type = ARG_PTR_TO_MEM_OR_NULL, + .arg4_type = ARG_CONST_SIZE_OR_ZERO, +}; + +const struct bpf_func_proto *bpf_get_trace_vprintk_proto(void) +{ + __set_printk_clr_event(); + return &bpf_trace_vprintk_proto; +} + BPF_CALL_5(bpf_seq_printf, struct seq_file *, m, char *, fmt, u32, fmt_size, const void *, data, u32, data_len) { @@ -1113,6 +1161,8 @@ bpf_tracing_func_proto(enum bpf_func_id func_id, const struct bpf_prog *prog) return &bpf_snprintf_proto; case BPF_FUNC_get_func_ip: return &bpf_get_func_ip_proto_tracing; + case BPF_FUNC_trace_vprintk: + return bpf_get_trace_vprintk_proto(); default: return bpf_base_func_proto(func_id); } diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index c4f7892edb2b..899a2649d986 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -4871,6 +4871,28 @@ union bpf_attr { * Return * Value specified by user at BPF link creation/attachment time * or 0, if it was not specified. + * + * u64 bpf_trace_vprintk(const char *fmt, u32 fmt_size, const void *data, u32 data_len) + * Description + * Behaves like **bpf_trace_printk**\ () helper, but takes an array of u64 + * to format. Supports up to 12 arguments to print in this way. + * The *fmt* and *fmt_size* are for the format string itself. The *data* and + * *data_len* are format string arguments. + * + * Each format specifier in **fmt** corresponds to one u64 element + * in the **data** array. For strings and pointers where pointees + * are accessed, only the pointer values are stored in the *data* + * array. The *data_len* is the size of *data* in bytes. + * Formats **%s**, **%p{i,I}{4,6}** requires to read kernel memory. + * Reading kernel memory may fail due to either invalid address or + * valid address but requiring a major memory fault. If reading kernel memory + * fails, the string for **%s** will be an empty string, and the ip + * address for **%p{i,I}{4,6}** will be 0. Not returning error to + * bpf program is consistent with what **bpf_trace_printk**\ () does for now. + * + * Return + * The number of bytes written to the buffer, or a negative error + * in case of failure. */ #define __BPF_FUNC_MAPPER(FN) \ FN(unspec), \ @@ -5048,6 +5070,7 @@ union bpf_attr { FN(timer_cancel), \ FN(get_func_ip), \ FN(get_attach_cookie), \ + FN(trace_vprintk), \ /* */ /* integer value in 'imm' field of BPF_CALL instruction selects which helper From patchwork Sat Aug 21 02:58:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Marchevsky X-Patchwork-Id: 12450785 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E4EECC432BE for ; Sat, 21 Aug 2021 02:59:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CC4A8611C4 for ; Sat, 21 Aug 2021 02:59:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240979AbhHUDAI (ORCPT ); Fri, 20 Aug 2021 23:00:08 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:18560 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241015AbhHUDAI (ORCPT ); Fri, 20 Aug 2021 23:00:08 -0400 Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17L2pAit011366 for ; Fri, 20 Aug 2021 19:59:30 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=kjxoTvvbtdKtm++ybFFEOg5xUoaDxGTFS2ZQmkmKJ70=; b=aPbHsuuDmjBPan4LhD/uJRbKYxnnRywY3faenm/vXjRmcxOu6F4+Q3OGxgyS0ely58Wc TGyEa0dc+nHhFx5ZCTMYLvcUzx40CMhDwgDMLJdcn9eg7iaLvPEKidCRRKXebmoMcekU qkPVsDczHAJkULN1KNfkIx4X20V6+KYmyzo= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com with ESMTP id 3ajfuh2x6a-18 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 Aug 2021 19:59:29 -0700 Received: from intmgw001.25.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::c) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Fri, 20 Aug 2021 19:59:14 -0700 Received: by devbig030.frc3.facebook.com (Postfix, from userid 158236) id 1421057300B2; Fri, 20 Aug 2021 19:59:10 -0700 (PDT) From: Dave Marchevsky To: CC: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Yonghong Song , Florent Revest , , , Dave Marchevsky Subject: [PATCH bpf-next 3/5] libbpf: Add bpf_vprintk convenience macro Date: Fri, 20 Aug 2021 19:58:35 -0700 Message-ID: <20210821025837.1614098-4-davemarchevsky@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210821025837.1614098-1-davemarchevsky@fb.com> References: <20210821025837.1614098-1-davemarchevsky@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-FB-Source: Intern X-Proofpoint-GUID: qqyZM2UU9Ea2lRi368TMyAf_883E3fiN X-Proofpoint-ORIG-GUID: qqyZM2UU9Ea2lRi368TMyAf_883E3fiN X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.790 definitions=2021-08-20_11:2021-08-20,2021-08-20 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 clxscore=1015 priorityscore=1501 mlxscore=0 suspectscore=0 lowpriorityscore=0 phishscore=0 impostorscore=0 mlxlogscore=982 adultscore=0 bulkscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108210016 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org bpf_vprintk functions similarly to BPF_SEQ_PRINTF and BPF_SNPRINTF macros elsewhere in the file - it allows use of bpf_trace_vprintk without manual conversion of varargs to u64 array. Like the bpf_printk macro, bpf_vprintk is meant to be the main interface to the bpf_trace_vprintk helper and thus is uncapitalized. Signed-off-by: Dave Marchevsky --- tools/lib/bpf/bpf_helpers.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/tools/lib/bpf/bpf_helpers.h b/tools/lib/bpf/bpf_helpers.h index b9987c3efa3c..43c8115956c3 100644 --- a/tools/lib/bpf/bpf_helpers.h +++ b/tools/lib/bpf/bpf_helpers.h @@ -224,4 +224,22 @@ enum libbpf_tristate { ___param, sizeof(___param)); \ }) +/* + * bpf_vprintk wraps the bpf_trace_printk helper with variadic arguments + * instead of an array of u64. + */ +#define bpf_vprintk(fmt, args...) \ +({ \ + static const char ___fmt[] = fmt; \ + unsigned long long ___param[___bpf_narg(args)]; \ + \ + _Pragma("GCC diagnostic push") \ + _Pragma("GCC diagnostic ignored \"-Wint-conversion\"") \ + ___bpf_fill(___param, args); \ + _Pragma("GCC diagnostic pop") \ + \ + bpf_trace_vprintk(___fmt, sizeof(___fmt), \ + ___param, sizeof(___param)); \ +}) + #endif From patchwork Sat Aug 21 02:58:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Marchevsky X-Patchwork-Id: 12450779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0364C4338F for ; Sat, 21 Aug 2021 02:59:19 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8A27D611C4 for ; Sat, 21 Aug 2021 02:59:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240908AbhHUC75 (ORCPT ); Fri, 20 Aug 2021 22:59:57 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:39798 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240917AbhHUC75 (ORCPT ); Fri, 20 Aug 2021 22:59:57 -0400 Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17L2pAim011366 for ; Fri, 20 Aug 2021 19:59:18 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=npoIg0hH/PLsSOOKa3cr7jFeeSikgJuTg/witRzt/oE=; b=ZAxh42V2NqblcLmt3CYkFjmaQ8DO8ymbx7pou+4EUnhBjorn6LX8sv/SQZsPX6sqlzd5 yUk+fEt0Cc7bxr7tpvEr+ulSHVftQBy+P0ALSFWa75YM8YA6b0Ve/WPA7XwC8rpStibI xprNv/+tvKs/m41Zs0nwMcvFeKchVI7IKU8= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com with ESMTP id 3ajfuh2x6a-11 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 Aug 2021 19:59:18 -0700 Received: from intmgw002.25.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::c) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Fri, 20 Aug 2021 19:59:13 -0700 Received: by devbig030.frc3.facebook.com (Postfix, from userid 158236) id BD27057300B7; Fri, 20 Aug 2021 19:59:11 -0700 (PDT) From: Dave Marchevsky To: CC: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Yonghong Song , Florent Revest , , , Dave Marchevsky Subject: [PATCH bpf-next 4/5] bpftool: only probe trace_vprintk feature in 'full' mode Date: Fri, 20 Aug 2021 19:58:36 -0700 Message-ID: <20210821025837.1614098-5-davemarchevsky@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210821025837.1614098-1-davemarchevsky@fb.com> References: <20210821025837.1614098-1-davemarchevsky@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-FB-Source: Intern X-Proofpoint-GUID: iMn05UMNWNVfAXRAo4loDXNCkXXYW-Pp X-Proofpoint-ORIG-GUID: iMn05UMNWNVfAXRAo4loDXNCkXXYW-Pp X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.790 definitions=2021-08-20_11:2021-08-20,2021-08-20 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 clxscore=1015 priorityscore=1501 mlxscore=0 suspectscore=0 lowpriorityscore=0 phishscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 bulkscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108210016 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Since commit 368cb0e7cdb5e ("bpftool: Make probes which emit dmesg warnings optional"), some helpers aren't probed by bpftool unless `full` arg is added to `bpftool feature probe`. bpf_trace_vprintk can emit dmesg warnings when probed, so include it. Signed-off-by: Dave Marchevsky --- tools/bpf/bpftool/feature.c | 1 + tools/testing/selftests/bpf/test_bpftool.py | 22 +++++++++------------ 2 files changed, 10 insertions(+), 13 deletions(-) diff --git a/tools/bpf/bpftool/feature.c b/tools/bpf/bpftool/feature.c index 7f36385aa9e2..ade44577688e 100644 --- a/tools/bpf/bpftool/feature.c +++ b/tools/bpf/bpftool/feature.c @@ -624,6 +624,7 @@ probe_helpers_for_progtype(enum bpf_prog_type prog_type, bool supported_type, */ switch (id) { case BPF_FUNC_trace_printk: + case BPF_FUNC_trace_vprintk: case BPF_FUNC_probe_write_user: if (!full_mode) continue; diff --git a/tools/testing/selftests/bpf/test_bpftool.py b/tools/testing/selftests/bpf/test_bpftool.py index 4fed2dc25c0a..1c2408ee1f5d 100644 --- a/tools/testing/selftests/bpf/test_bpftool.py +++ b/tools/testing/selftests/bpf/test_bpftool.py @@ -57,6 +57,11 @@ def default_iface(f): return f(*args, iface, **kwargs) return wrapper +DMESG_EMITTING_HELPERS = [ + "bpf_probe_write_user", + "bpf_trace_printk", + "bpf_trace_vprintk", + ] class TestBpftool(unittest.TestCase): @classmethod @@ -67,10 +72,7 @@ class TestBpftool(unittest.TestCase): @default_iface def test_feature_dev_json(self, iface): - unexpected_helpers = [ - "bpf_probe_write_user", - "bpf_trace_printk", - ] + unexpected_helpers = DMESG_EMITTING_HELPERS expected_keys = [ "syscall_config", "program_types", @@ -94,10 +96,7 @@ class TestBpftool(unittest.TestCase): bpftool_json(["feature", "probe"]), bpftool_json(["feature"]), ] - unexpected_helpers = [ - "bpf_probe_write_user", - "bpf_trace_printk", - ] + unexpected_helpers = DMESG_EMITTING_HELPERS expected_keys = [ "syscall_config", "system_config", @@ -121,10 +120,7 @@ class TestBpftool(unittest.TestCase): bpftool_json(["feature", "probe", "kernel", "full"]), bpftool_json(["feature", "probe", "full"]), ] - expected_helpers = [ - "bpf_probe_write_user", - "bpf_trace_printk", - ] + expected_helpers = DMESG_EMITTING_HELPERS for tc in test_cases: # Check if expected helpers are included at least once in any @@ -157,7 +153,7 @@ class TestBpftool(unittest.TestCase): not_full_set.add(helper) self.assertCountEqual(full_set - not_full_set, - {"bpf_probe_write_user", "bpf_trace_printk"}) + set(DMESG_EMITTING_HELPERS)) self.assertCountEqual(not_full_set - full_set, set()) def test_feature_macros(self): From patchwork Sat Aug 21 02:58:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Marchevsky X-Patchwork-Id: 12450781 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, UNWANTED_LANGUAGE_BODY,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 329DBC4320A for ; Sat, 21 Aug 2021 02:59:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1BDD4611C4 for ; Sat, 21 Aug 2021 02:59:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240977AbhHUDAD (ORCPT ); Fri, 20 Aug 2021 23:00:03 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:59504 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240917AbhHUDAC (ORCPT ); Fri, 20 Aug 2021 23:00:02 -0400 Received: from pps.filterd (m0109333.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 17L2ns2r025227 for ; Fri, 20 Aug 2021 19:59:24 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=x5MH0LICWFGXZzJKV/mKS2xu1y75l/9W0mB962bIBEM=; b=ceSYUczT0w2mdn3HZXyt998+Bz/FHls5ymLYEJ1swMyl8BParCUlrB2p838iVPuwVqHF nVERAqpeMdLmWJwJMZvD2jBXUT3PSZ6tg7dhzo8DO2hwyD30qO9FRaDQbuY8hbLRkFM7 kucWQl0WeNByDKVQ0mDwBZrf0SXClhS1w3A= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com with ESMTP id 3aj09u07wr-7 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 20 Aug 2021 19:59:23 -0700 Received: from intmgw001.25.frc3.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Fri, 20 Aug 2021 19:59:19 -0700 Received: by devbig030.frc3.facebook.com (Postfix, from userid 158236) id 4D7C457300BA; Fri, 20 Aug 2021 19:59:14 -0700 (PDT) From: Dave Marchevsky To: CC: Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Yonghong Song , Florent Revest , , , Dave Marchevsky Subject: [PATCH bpf-next 5/5] selftests/bpf: add trace_vprintk test prog Date: Fri, 20 Aug 2021 19:58:37 -0700 Message-ID: <20210821025837.1614098-6-davemarchevsky@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210821025837.1614098-1-davemarchevsky@fb.com> References: <20210821025837.1614098-1-davemarchevsky@fb.com> MIME-Version: 1.0 X-FB-Internal: Safe X-FB-Source: Intern X-Proofpoint-ORIG-GUID: EhQFYmUrr31Q9RTVR3qTYJEY0oMi2KbB X-Proofpoint-GUID: EhQFYmUrr31Q9RTVR3qTYJEY0oMi2KbB X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.790 definitions=2021-08-20_11:2021-08-20,2021-08-20 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 spamscore=0 bulkscore=0 impostorscore=0 malwarescore=0 adultscore=0 priorityscore=1501 clxscore=1015 lowpriorityscore=0 suspectscore=0 mlxlogscore=999 phishscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2108210016 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org This commit adds a test prog for vprintk which confirms that: * bpf_trace_vprintk is writing to dmesg * bpf_vprintk convenience macro works as expected * >3 args are printed Approach and code are borrowed from trace_printk test. Signed-off-by: Dave Marchevsky --- tools/testing/selftests/bpf/Makefile | 3 +- .../selftests/bpf/prog_tests/trace_vprintk.c | 75 +++++++++++++++++++ .../selftests/bpf/progs/trace_vprintk.c | 25 +++++++ 3 files changed, 102 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/trace_vprintk.c create mode 100644 tools/testing/selftests/bpf/progs/trace_vprintk.c diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 2a58b7b5aea4..af5e7a1e9a7c 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -313,7 +313,8 @@ LINKED_SKELS := test_static_linked.skel.h linked_funcs.skel.h \ linked_vars.skel.h linked_maps.skel.h LSKELS := kfunc_call_test.c fentry_test.c fexit_test.c fexit_sleep.c \ - test_ksyms_module.c test_ringbuf.c atomics.c trace_printk.c + test_ksyms_module.c test_ringbuf.c atomics.c trace_printk.c \ + trace_vprintk.c SKEL_BLACKLIST += $$(LSKELS) test_static_linked.skel.h-deps := test_static_linked1.o test_static_linked2.o diff --git a/tools/testing/selftests/bpf/prog_tests/trace_vprintk.c b/tools/testing/selftests/bpf/prog_tests/trace_vprintk.c new file mode 100644 index 000000000000..fd70427d2918 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/trace_vprintk.c @@ -0,0 +1,75 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2021 Facebook */ + +#include + +#include "trace_vprintk.lskel.h" + +#define TRACEBUF "/sys/kernel/debug/tracing/trace_pipe" +#define SEARCHMSG "1,2,3,4,5,6,7,8,9,10" + +void test_trace_vprintk(void) +{ + int err, iter = 0, duration = 0, found = 0; + struct trace_vprintk__bss *bss; + struct trace_vprintk *skel; + char *buf = NULL; + FILE *fp = NULL; + size_t buflen; + + skel = trace_vprintk__open(); + if (CHECK(!skel, "skel_open", "failed to open skeleton\n")) + return; + + err = trace_vprintk__load(skel); + if (CHECK(err, "skel_load", "failed to load skeleton: %d\n", err)) + goto cleanup; + + bss = skel->bss; + + err = trace_vprintk__attach(skel); + if (CHECK(err, "skel_attach", "skeleton attach failed: %d\n", err)) + goto cleanup; + + fp = fopen(TRACEBUF, "r"); + if (CHECK(fp == NULL, "could not open trace buffer", + "error %d opening %s", errno, TRACEBUF)) + goto cleanup; + + /* We do not want to wait forever if this test fails... */ + fcntl(fileno(fp), F_SETFL, O_NONBLOCK); + + /* wait for tracepoint to trigger */ + usleep(1); + trace_vprintk__detach(skel); + + if (CHECK(bss->trace_vprintk_ran == 0, + "bpf_trace_vprintk never ran", + "ran == %d", bss->trace_vprintk_ran)) + goto cleanup; + + if (CHECK(bss->trace_vprintk_ret <= 0, + "bpf_trace_vprintk returned <= 0 value", + "got %d", bss->trace_vprintk_ret)) + goto cleanup; + + /* verify our search string is in the trace buffer */ + while (getline(&buf, &buflen, fp) >= 0 || errno == EAGAIN) { + if (strstr(buf, SEARCHMSG) != NULL) + found++; + if (found == bss->trace_vprintk_ran) + break; + if (++iter > 1000) + break; + } + + if (CHECK(!found, "message from bpf_trace_vprintk not found", + "no instance of %s in %s", SEARCHMSG, TRACEBUF)) + goto cleanup; + +cleanup: + trace_vprintk__destroy(skel); + free(buf); + if (fp) + fclose(fp); +} diff --git a/tools/testing/selftests/bpf/progs/trace_vprintk.c b/tools/testing/selftests/bpf/progs/trace_vprintk.c new file mode 100644 index 000000000000..993439a19e1e --- /dev/null +++ b/tools/testing/selftests/bpf/progs/trace_vprintk.c @@ -0,0 +1,25 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2021 Facebook */ + +#include "vmlinux.h" +#include +#include + +char _license[] SEC("license") = "GPL"; + +int trace_vprintk_ret = 0; +int trace_vprintk_ran = 0; + +SEC("fentry/__x64_sys_nanosleep") +int sys_enter(void *ctx) +{ + static const char one[] = "1"; + static const char three[] = "3"; + static const char five[] = "5"; + static const char seven[] = "7"; + static const char nine[] = "9"; + + trace_vprintk_ret = bpf_vprintk("%s,%d,%s,%d,%s,%d,%s,%d,%s,%d %d\n", + one, 2, three, 4, five, 6, seven, 8, nine, 10, ++trace_vprintk_ran); + return 0; +}