| Message ID | 20240723182439.1434795-1-thinker.li@gmail.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | monitor network traffic for flaky test cases | expand |
On 7/23/24 11:24, Kui-Feng Lee wrote: > Capture packets in the background for flaky test cases related to > network features. > > We have some flaky test cases that are difficult to debug without > knowing what the traffic looks like. Capturing packets, the CI log and > packet files may help developers to fix these flaky test cases. > > This patch set monitors a few test cases. Recently, they have been > showing flaky behavior. > > IPv4 TCP packet: 127.0.0.1:48165 -> 127.0.0.1:36707, len 68, ifindex 1, SYN > IPv4 TCP packet: 127.0.0.1:36707 -> 127.0.0.1:48165, len 60, ifindex 1, SYN, ACK > IPv4 TCP packet: 127.0.0.1:48165 -> 127.0.0.1:36707, len 60, ifindex 1, ACK > IPv4 TCP packet: 127.0.0.1:36707 -> 127.0.0.1:48165, len 52, ifindex 1, ACK > IPv4 TCP packet: 127.0.0.1:48165 -> 127.0.0.1:36707, len 52, ifindex 1, FIN, ACK > IPv4 TCP packet: 127.0.0.1:36707 -> 127.0.0.1:48165, len 52, ifindex 1, RST, ACK > Packet file: packets-2172-86.log > #280/87 select_reuseport/sockhash IPv4/TCP LOOPBACK test_detach_bpf:OK > > The above block is the log of a test case. It shows every packets of a > connection. The captured packets are stored in the file called > packets-2172-86.log. > > The following block is an example that monitors the network traffic of > a test case. This test is running in the network namespace > "testns". You can pass NULL to traffic_monitor_start() if the entire > test, from traffic_monitor_start() to traffic_monitor_stop(), is > running in the same namespace. > > struct tmonitor_ctx *tmon; > > ... > tmon = traffic_monitor_start("testns"); > ASSERT_TRUE(tmon, "traffic_monitor_start"); > > ... test ... > > traffic_monitor_stop(tmon); > > traffic_monitor_start() may fail, but we just ignore it since the > failure doesn't affect the following main test. > > This feature is enabled only if BPF selftests are built with > TRAFFIC_MONITOR variable being defined. For example, > > make TRAFFIC_MONITOR=1 -C tools/testing/selftests/bpf > > This command will enable traffic monitoring for BPF selftests. That > means we have to turn it on to get the log at CI. > > --- > > Changes from v1: > > - Initialize log_fd in traffic_monitor_start(). > > - Remove redundant including. Sorry for not updating changes correctly. No more tcpdump, it moves to use call pcap directly in a background thread. Packets are wrote to a packet file. In the log, it prints parsed information of TCP or UDP packets. For other packets, just print a string "Packet captured" to indicate a packet has been captured. > > v1: https://lore.kernel.org/all/20240713055552.2482367-5-thinker.li@gmail.com/ > > Kui-Feng Lee (4): > selftests/bpf: Add traffic monitor functions. > selftests/bpf: Monitor traffic for tc_redirect/tc_redirect_dtime. > selftests/bpf: Monitor traffic for sockmap_listen. > selftests/bpf: Monitor traffic for select_reuseport. > > tools/testing/selftests/bpf/Makefile | 5 + > tools/testing/selftests/bpf/network_helpers.c | 382 ++++++++++++++++++ > tools/testing/selftests/bpf/network_helpers.h | 16 + > .../bpf/prog_tests/select_reuseport.c | 7 + > .../selftests/bpf/prog_tests/sockmap_listen.c | 8 + > .../selftests/bpf/prog_tests/tc_redirect.c | 5 + > 6 files changed, 423 insertions(+) >
Capture packets in the background for flaky test cases related to network features. We have some flaky test cases that are difficult to debug without knowing what the traffic looks like. Capturing packets, the CI log and packet files may help developers to fix these flaky test cases. This patch set monitors a few test cases. Recently, they have been showing flaky behavior. IPv4 TCP packet: 127.0.0.1:48165 -> 127.0.0.1:36707, len 68, ifindex 1, SYN IPv4 TCP packet: 127.0.0.1:36707 -> 127.0.0.1:48165, len 60, ifindex 1, SYN, ACK IPv4 TCP packet: 127.0.0.1:48165 -> 127.0.0.1:36707, len 60, ifindex 1, ACK IPv4 TCP packet: 127.0.0.1:36707 -> 127.0.0.1:48165, len 52, ifindex 1, ACK IPv4 TCP packet: 127.0.0.1:48165 -> 127.0.0.1:36707, len 52, ifindex 1, FIN, ACK IPv4 TCP packet: 127.0.0.1:36707 -> 127.0.0.1:48165, len 52, ifindex 1, RST, ACK Packet file: packets-2172-86.log #280/87 select_reuseport/sockhash IPv4/TCP LOOPBACK test_detach_bpf:OK The above block is the log of a test case. It shows every packets of a connection. The captured packets are stored in the file called packets-2172-86.log. The following block is an example that monitors the network traffic of a test case. This test is running in the network namespace "testns". You can pass NULL to traffic_monitor_start() if the entire test, from traffic_monitor_start() to traffic_monitor_stop(), is running in the same namespace. struct tmonitor_ctx *tmon; ... tmon = traffic_monitor_start("testns"); ASSERT_TRUE(tmon, "traffic_monitor_start"); ... test ... traffic_monitor_stop(tmon); traffic_monitor_start() may fail, but we just ignore it since the failure doesn't affect the following main test. This feature is enabled only if BPF selftests are built with TRAFFIC_MONITOR variable being defined. For example, make TRAFFIC_MONITOR=1 -C tools/testing/selftests/bpf This command will enable traffic monitoring for BPF selftests. That means we have to turn it on to get the log at CI. --- Changes from v1: - Initialize log_fd in traffic_monitor_start(). - Remove redundant including. v1: https://lore.kernel.org/all/20240713055552.2482367-5-thinker.li@gmail.com/ Kui-Feng Lee (4): selftests/bpf: Add traffic monitor functions. selftests/bpf: Monitor traffic for tc_redirect/tc_redirect_dtime. selftests/bpf: Monitor traffic for sockmap_listen. selftests/bpf: Monitor traffic for select_reuseport. tools/testing/selftests/bpf/Makefile | 5 + tools/testing/selftests/bpf/network_helpers.c | 382 ++++++++++++++++++ tools/testing/selftests/bpf/network_helpers.h | 16 + .../bpf/prog_tests/select_reuseport.c | 7 + .../selftests/bpf/prog_tests/sockmap_listen.c | 8 + .../selftests/bpf/prog_tests/tc_redirect.c | 5 + 6 files changed, 423 insertions(+)