From patchwork Tue Sep 21 01:31:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martin KaFai Lau X-Patchwork-Id: 12506705 X-Patchwork-Delegate: bpf@iogearbox.net Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18B93C433F5 for ; Tue, 21 Sep 2021 01:31:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DE308604D1 for ; Tue, 21 Sep 2021 01:31:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229790AbhIUBdL (ORCPT ); Mon, 20 Sep 2021 21:33:11 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:7244 "EHLO mx0b-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230183AbhIUBcz (ORCPT ); Mon, 20 Sep 2021 21:32:55 -0400 Received: from pps.filterd (m0109331.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18KHwWqF006002 for ; Mon, 20 Sep 2021 18:31:27 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : content-transfer-encoding : mime-version; s=facebook; bh=X36ME8e7f0R2qzKkpq3fXg6QJ01K7Au7LTLQ9SxZJTA=; b=atnnFFvetAQUbjohXdXIelYApk9h9lqNwVdbZTOynJ+BH4VlemoolyMoCYuKGUW+pL+w w1yBgm937BaT6IyH21cI6wECcCPm+cyPO7c6ekugNnmiQkYyVhvpVP+8pPsR+2JYf0X+ iRrXU9IdkiTBc+KCeLOO0k8SS+rx00L3LS8= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com with ESMTP id 3b6v27uns2-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Mon, 20 Sep 2021 18:31:27 -0700 Received: from intmgw001.37.frc1.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:82::f) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.14; Mon, 20 Sep 2021 18:31:25 -0700 Received: by devbig005.ftw2.facebook.com (Postfix, from userid 6611) id 5C6E32940D2A; Mon, 20 Sep 2021 18:31:22 -0700 (PDT) From: Martin KaFai Lau To: CC: Alexei Starovoitov , Andrii Nakryiko , Daniel Borkmann , , Yonghong Song Subject: [PATCH bpf-next 3/4] bpf: selftest: A bpf prog that has a 32bit scalar spill Date: Mon, 20 Sep 2021 18:31:22 -0700 Message-ID: <20210921013122.1037548-1-kafai@fb.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210921013102.1035356-1-kafai@fb.com> References: <20210921013102.1035356-1-kafai@fb.com> X-FB-Internal: Safe X-FB-Source: Intern X-Proofpoint-GUID: uLcatKMzTcyorLCL0Re2JZSN6Z9sR4cK X-Proofpoint-ORIG-GUID: uLcatKMzTcyorLCL0Re2JZSN6Z9sR4cK X-Proofpoint-UnRewURL: 0 URL was un-rewritten MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475 definitions=2021-09-20_11,2021-09-20_01,2020-04-07_01 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 clxscore=1015 mlxscore=0 phishscore=0 malwarescore=0 priorityscore=1501 impostorscore=0 lowpriorityscore=0 spamscore=0 bulkscore=0 mlxlogscore=999 adultscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109030001 definitions=main-2109210006 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-Delegate: bpf@iogearbox.net It is a simplified example that can trigger a 32bit scalar spill. The const scalar is refilled and added to a skb->data later. Since the reg state of the 32bit scalar spill is not saved now, adding the refilled reg to skb->data and then comparing it with skb->data_end cannot verify the skb->data access. With the earlier verifier patch and the llvm patch [1]. The verifier can correctly verify the bpf prog. Here is the snippet of the verifier log that leads to verifier conclusion that the packet data is unsafe to read. 67: R0=inv1 R1=inv17 R2=pkt(id=0,off=62,r=102,imm=0) R3=invP2 R4=pkt(id=0,off=68,r=102,imm=0) R5=inv1 R6=pkt(id=0,off=0,r=102,imm=0) R7=pkt_end(id=0,off=0,imm=0) R8=inv17 R9=inv102 R10=fp0 67: (63) *(u32 *)(r10 -16) = r9 68: R0=inv1 R1=inv17 R2=pkt(id=0,off=62,r=102,imm=0) R3=invP2 R4=pkt(id=0,off=68,r=102,imm=0) R5=inv1 R6=pkt(id=0,off=0,r=102,imm=0) R7=pkt_end(id=0,off=0,imm=0) R8=inv17 R9=inv102 R10=fp0 fp-16=????mmmm ... 74: R0_w=map_value_or_null(id=2,off=0,ks=16,vs=1,imm=0) R5_w=inv1 R6=pkt(id=0,off=0,r=102,imm=0) R7=pkt_end(id=0,off=0,imm=0) R8=inv17 R9_w=inv1 R10=fp0 fp-16=????mmmm 74: (61) r9 = *(u32 *)(r10 -16) 75: R0_w=map_value_or_null(id=2,off=0,ks=16,vs=1,imm=0) R5_w=inv1 R6=pkt(id=0,off=0,r=102,imm=0) R7=pkt_end(id=0,off=0,imm=0) R8=inv17 R9_w=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R10=fp0 fp-16=????mmmm 75: (bc) w1 = w9 76: R0_w=map_value_or_null(id=2,off=0,ks=16,vs=1,imm=0) R1_w=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R5_w=inv1 R6=pkt(id=0,off=0,r=102,imm=0) R7=pkt_end(id=0,off=0,imm=0) R8=inv17 R9_w=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R10=fp0 fp-16=????mmmm 76: (0f) r6 += r1 last_idx 76 first_idx 67 regs=2 stack=0 before 75: (bc) w1 = w9 regs=200 stack=0 before 74: (61) r9 = *(u32 *)(r10 -16) 77: R0_w=map_value_or_null(id=2,off=0,ks=16,vs=1,imm=0) R1_w=invP(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R5_w=inv1 R6_w=pkt(id=3,off=0,r=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R7=pkt_end(id=0,off=0,imm=0) R8=inv17 R9_w=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R10=fp0 fp-16=????mmmm ... 99: R0=inv1 R1=invP(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R5=inv1 R6=pkt(id=3,off=0,r=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R7=pkt_end(id=0,off=0,imm=0) R8_w=invP17 R9=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R10=fp0 fp-16=????mmmm 99: (bf) r1 = r6 100: R0=inv1 R1_w=pkt(id=3,off=0,r=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R5=inv1 R6=pkt(id=3,off=0,r=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R7=pkt_end(id=0,off=0,imm=0) R8_w=invP17 R9=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R10=fp0 fp-16=????mmmm 100: (07) r1 += 8 101: R0=inv1 R1_w=pkt(id=3,off=8,r=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R5=inv1 R6=pkt(id=3,off=0,r=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R7=pkt_end(id=0,off=0,imm=0) R8_w=invP17 R9=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R10=fp0 fp-16=????mmmm 101: (b4) w0 = 1 102: R0_w=inv1 R1_w=pkt(id=3,off=8,r=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R5=inv1 R6=pkt(id=3,off=0,r=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R7=pkt_end(id=0,off=0,imm=0) R8_w=invP17 R9=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R10=fp0 fp-16=????mmmm 102: (2d) if r1 > r7 goto pc-38 R0_w=inv1 R1_w=pkt(id=3,off=8,r=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R5=inv1 R6=pkt(id=3,off=0,r=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R7=pkt_end(id=0,off=0,imm=0) R8_w=invP17 R9=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R10=fp0 fp-16=????mmmm 103: R0_w=inv1 R1_w=pkt(id=3,off=8,r=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R5=inv1 R6=pkt(id=3,off=0,r=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R7=pkt_end(id=0,off=0,imm=0) R8_w=invP17 R9=inv(id=0,umax_value=4294967295,var_off=(0x0; 0xffffffff)) R10=fp0 fp-16=????mmmm 103: (69) r7 = *(u16 *)(r6 +0) invalid access to packet, off=0 size=2, R6(id=3,off=0,r=0) R6 offset is outside of the packet [1]: https://reviews.llvm.org/D109073 Signed-off-by: Martin KaFai Lau --- .../selftests/bpf/prog_tests/xdpwall.c | 15 + tools/testing/selftests/bpf/progs/xdpwall.c | 302 ++++++++++++++++++ 2 files changed, 317 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/xdpwall.c create mode 100644 tools/testing/selftests/bpf/progs/xdpwall.c diff --git a/tools/testing/selftests/bpf/prog_tests/xdpwall.c b/tools/testing/selftests/bpf/prog_tests/xdpwall.c new file mode 100644 index 000000000000..d263fd0a7860 --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/xdpwall.c @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2021 Facebook */ + +#include "test_progs.h" +#include "xdpwall.skel.h" + +void test_xdpwall(void) +{ + struct xdpwall *skel; + + skel = xdpwall__open_and_load(); + ASSERT_OK_PTR(skel, "skel"); + + xdpwall__destroy(skel); +} diff --git a/tools/testing/selftests/bpf/progs/xdpwall.c b/tools/testing/selftests/bpf/progs/xdpwall.c new file mode 100644 index 000000000000..314a8d6beff0 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/xdpwall.c @@ -0,0 +1,302 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2021 Facebook */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +enum pkt_parse_err { + NO_ERR, + BAD_IP6_HDR, + BAD_IP4GUE_HDR, + BAD_IP6GUE_HDR, +}; + +enum pkt_flag { + TUNNEL = 0x1, + TCP_SYN = 0x2, + QUIC_INITIAL_FLAG = 0x4, + TCP_ACK = 0x8, + TCP_RST = 0x10 +}; + +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(max_entries, 16); + __type(key, struct in6_addr); + __type(value, bool); +} v6_addr_map SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 16); + __type(key, int); + __type(value, __u8); +} tcp_port_map SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 16); + __type(key, int); + __type(value, __u16); +} udp_port_map SEC(".maps"); + +enum ip_type { V4 = 1, V6 = 2 }; + +struct fw_match_info { + __u8 v4_src_ip_match; + __u8 v6_src_ip_match; + __u8 v4_src_prefix_match; + __u8 v4_dst_prefix_match; + __u8 v6_src_prefix_match; + __u8 tcp_dp_match; + __u16 udp_sp_match; + __u16 udp_dp_match; + bool is_tcp; + bool is_tcp_syn; + bool is_link_local_v6; +}; + +struct pkt_info { + enum ip_type type; + union { + struct iphdr *ipv4; + struct ipv6hdr *ipv6; + } ip; + int sport; + int dport; + __u16 trans_hdr_offset; + __u8 proto; + __u8 flags; +}; + +static __always_inline struct ethhdr *parse_ethhdr(void *data, void *data_end) +{ + struct ethhdr *eth = data; + + if (eth + 1 > data_end) + return NULL; + + return eth; +} + +static __always_inline void +filter_src_ip6(struct pkt_info *info, struct fw_match_info *match_info) +{ + if (info->type == V6 && + bpf_map_lookup_elem(&v6_addr_map, &info->ip.ipv6->saddr)) + match_info->v6_src_ip_match = true; +} + +static __always_inline void * +get_transport_hdr(__u16 offset, void *data, void *data_end) +{ + if (offset > 255 || data + offset > data_end) + return NULL; + + return data + offset; +} + +static __always_inline bool tcphdr_only_contains_flag(struct tcphdr *tcp, + __u32 FLAG) +{ + return (tcp_flag_word(tcp) & + (TCP_FLAG_ACK | TCP_FLAG_RST | TCP_FLAG_SYN | TCP_FLAG_FIN)) == FLAG; +} + +static __always_inline void set_tcp_flags(struct pkt_info *info, + struct tcphdr *tcp) { + if (tcphdr_only_contains_flag(tcp, TCP_FLAG_SYN)) + info->flags |= TCP_SYN; + else if (tcphdr_only_contains_flag(tcp, TCP_FLAG_ACK)) + info->flags |= TCP_ACK; + else if (tcphdr_only_contains_flag(tcp, TCP_FLAG_RST)) + info->flags |= TCP_RST; +} + +static __always_inline bool +parse_tcp(struct pkt_info *info, void *transport_hdr, void *data_end) +{ + struct tcphdr *tcp = transport_hdr; + + if (tcp + 1 > data_end) + return false; + + info->sport = bpf_ntohs(tcp->source); + info->dport = bpf_ntohs(tcp->dest); + set_tcp_flags(info, tcp); + + return true; +} + +static __always_inline bool +parse_udp(struct pkt_info *info, void *transport_hdr, void *data_end) +{ + struct udphdr *udp = transport_hdr; + + if (udp + 1 > data_end) + return false; + + info->sport = bpf_ntohs(udp->source); + info->dport = bpf_ntohs(udp->dest); + + return true; +} + +static __always_inline __u8 filter_tcp_port(int port) +{ + __u8 *leaf = bpf_map_lookup_elem(&tcp_port_map, &port); + + return leaf ? *leaf : 0; +} + +static __always_inline __u16 filter_udp_port(int port) +{ + __u16 *leaf = bpf_map_lookup_elem(&udp_port_map, &port); + + return leaf ? *leaf : 0; +} + +static __always_inline bool +filter_transport_hdr(void *transport_hdr, void *data_end, + struct pkt_info *info, struct fw_match_info *match_info) +{ + if (info->proto == IPPROTO_TCP) { + if (!parse_tcp(info, transport_hdr, data_end)) + return false; + + match_info->is_tcp = true; + match_info->is_tcp_syn = (info->flags & TCP_SYN) > 0; + + match_info->tcp_dp_match = filter_tcp_port(info->dport); + } else if (info->proto == IPPROTO_UDP) { + if (!parse_udp(info, transport_hdr, data_end)) + return false; + + match_info->udp_dp_match = filter_udp_port(info->dport); + match_info->udp_sp_match = filter_udp_port(info->sport); + } + + return true; +} + +static __always_inline __u8 +parse_gue_v6(struct pkt_info *info, struct ipv6hdr *ip6h, void *data_end) +{ + struct udphdr *udp = (struct udphdr *)(ip6h + 1); + void *encap_data = udp + 1; + + if (udp + 1 > data_end) + return BAD_IP6_HDR; + + if (udp->dest != bpf_htons(6666)) + return NO_ERR; + + info->flags |= TUNNEL; + + if (encap_data + 1 > data_end) + return BAD_IP6GUE_HDR; + + if (*(__u8 *)encap_data & 0x30) { + struct ipv6hdr *inner_ip6h = encap_data; + + if (inner_ip6h + 1 > data_end) + return BAD_IP6GUE_HDR; + + info->type = V6; + info->proto = inner_ip6h->nexthdr; + info->ip.ipv6 = inner_ip6h; + info->trans_hdr_offset += sizeof(struct ipv6hdr) + sizeof(struct udphdr); + } else { + struct iphdr *inner_ip4h = encap_data; + + if (inner_ip4h + 1 > data_end) + return BAD_IP6GUE_HDR; + + info->type = V4; + info->proto = inner_ip4h->protocol; + info->ip.ipv4 = inner_ip4h; + info->trans_hdr_offset += sizeof(struct iphdr) + sizeof(struct udphdr); + } + + return NO_ERR; +} + +static __always_inline __u8 parse_ipv6_gue(struct pkt_info *info, + void *data, void *data_end) +{ + struct ipv6hdr *ip6h = data + sizeof(struct ethhdr); + + if (ip6h + 1 > data_end) + return BAD_IP6_HDR; + + info->proto = ip6h->nexthdr; + info->ip.ipv6 = ip6h; + info->type = V6; + info->trans_hdr_offset = sizeof(struct ethhdr) + sizeof(struct ipv6hdr); + + if (info->proto == IPPROTO_UDP) + return parse_gue_v6(info, ip6h, data_end); + + return NO_ERR; +} + +SEC("xdp") +int edgewall(struct xdp_md *ctx) +{ + void *data_end = (void *)(long)(ctx->data_end); + void *data = (void *)(long)(ctx->data); + struct fw_match_info match_info = {}; + struct pkt_info info = {}; + __u8 parse_err = NO_ERR; + void *transport_hdr; + struct ethhdr *eth; + bool filter_res; + __u32 proto; + + eth = parse_ethhdr(data, data_end); + if (!eth) + return XDP_DROP; + + proto = eth->h_proto; + if (proto != bpf_htons(ETH_P_IPV6)) + return XDP_DROP; + + if (parse_ipv6_gue(&info, data, data_end)) + return XDP_DROP; + + if (info.proto == IPPROTO_ICMPV6) + return XDP_PASS; + + if (info.proto != IPPROTO_TCP && info.proto != IPPROTO_UDP) + return XDP_DROP; + + filter_src_ip6(&info, &match_info); + + transport_hdr = get_transport_hdr(info.trans_hdr_offset, data, + data_end); + if (!transport_hdr) + return XDP_DROP; + + filter_res = filter_transport_hdr(transport_hdr, data_end, + &info, &match_info); + if (!filter_res) + return XDP_DROP; + + if (match_info.is_tcp && !match_info.is_tcp_syn) + return XDP_PASS; + + return XDP_DROP; +} + +char LICENSE[] SEC("license") = "GPL";