推荐一款轻量级 eBPF 前端工具 ply (jgsun.github.io)
https://jgsun.github.io/2020/10/11/13-57-49-ply-intro/
推荐一款轻量级 eBPF 前端工具 ply
2020-10-11 jgsun debug eBPF Dynamic_Tracer Embedded_LinuxOverview
ply 是 eBPF 的 front-end 前端工具之一，专为 embedded Linux systems 开发，采用 C 语言编写，只需 libc 和内核支持 BPF 就可以运行，不需要外部 kernel 模块，不需要 LLVM，不需要 python。
ply 由瑞典工程师 Tobias Waldekranz 开发，其项目主页是 PLY Light-weight Dynamic Tracer for Linux 。
使用非常灵活和轻量级，编辑一种类 C 语言的脚本，然后利用内核 eBPF 来收集和探测内核数据，比如打印出内核函数的调用栈，获取内核变量等，是学习内核，进行嵌入式 Linux 系统开发调试的利器！
本文记录 ply 的编译及使用过程，实验例程源码都已上传到 https://github.com/jgsun/buildroot。
内核配置
注意： 为使用 ply，内核需要支持 eBPF，选择如下配置：
CONFIG_KPROBES=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_DYNAMIC_FTRACE_WITH_REGS=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_FTRACE=y
CONFIG_DYNAMIC_FTRACE=y
CONFIG_BPF=y
CONFIG_BPF_SYSCALL=y
ply 编译
交叉编译
交叉编译步骤如下：
git clone https://github.com/wkz/ply
export PATH=$PATH:/home/jgsun/repo/buildroot-arm64/output/host/opt/ext-toolchain/bin
./autogen.sh
./configure --host=aarch64-none-linux-gnu --prefix=/home/jgsun/usr
make
make install
查看安装目录 /home/jgsun/usr：
jgsun@VirtualBox:~/usr$ tree
.
|-- include
| `-- ply
| |-- arch.h
| |-- buffer.h
| |-- func.h
| |-- internal.h
| |-- ir.h
| |-- kallsyms.h
| |-- node.h
| |-- perf_event.h
| |-- ply.h
| |-- printxf.h
| |-- provider.h
| |-- sym.h
| |-- syscall.h
| |-- type.h
| `-- utils.h
|-- lib
| |-- libply.a
| |-- libply.la
| |-- libply.so -> libply.so.0.0.0
| |-- libply.so.0 -> libply.so.0.0.0
| `-- libply.so.0.0.0
|-- sbin
| `-- ply
`-- share
`-- doc
`-- ply
|-- COPYING
`-- README.md
7 directories, 23 files
将lib目录的库文件和 sbin 目录的可执行文件 ply 拷贝到 target 板卡的文件系统即可使用。
jgsun@VirtualBox:~/usr$ scp -P 22 lib/* [email protected]:/lib
jgsun@VirtualBox:~/usr/=$ scp -P 22 sbin/ply [email protected]:~/bin
root@~/bin# ./ply -v
ply 2.1.1-11-g6aabe5f (linux-version:267277~4.20.13)
root@~/bin# uname -a
Linux qemu-aarch64 5.8.4 #4 SMP Fri Oct 16 11:05:52 CST 2020 aarch64 GNU/Linux
使用buildroot编译
ply 项目采用 GNU’s autotools build 系统，非常容易集成到 buildroot，首先在 buildroot 添加 ply package，然后执行 make ply 编译即可。
可以从 buildroot/package/ply 下载，也可以参考下述 patch 修改：
diff --git a/package/Config.in b/package/Config.in
index cb6d8e0e01..7dd278242e 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -2321,6 +2321,7 @@ menu "System tools"
source "package/openrc/Config.in"
source "package/openvmtools/Config.in"
source "package/pamtester/Config.in"
+ source "package/ply/Config.in"
source "package/polkit/Config.in"
source "package/powerpc-utils/Config.in"
source "package/procps-ng/Config.in"
diff --git a/package/ply/.Config.in.swp b/package/ply/.Config.in.swp
new file mode 100644
index 0000000000..1193f698fb
Binary files /dev/null and b/package/ply/.Config.in.swp differ
diff --git a/package/ply/Config.in b/package/ply/Config.in
new file mode 100644
index 0000000000..258a59b6fd
--- /dev/null
+++ b/package/ply/Config.in
@@ -0,0 +1,11 @@
+config BR2_PACKAGE_PLY
+ bool "ply"
+ depends on BR2_x86_64 || BR2_aarch64 || BR2_arm || BR2_ppc # needs <cpuid.h>
+ depends on BR2_TOOLCHAIN_USES_UCLIBC || BR2_TOOLCHAIN_USES_GLIBC
+ help
+ ply dynamically instruments the running kernel to aggregate and
+ extract user-defined data. It compiles an input program to one or
+ more Linux bpf(2) binaries and attaches them to arbitrary points
+ in the kernel using kprobes and tracepoints.
+
+ https://wkz.github.io/ply/
diff --git a/package/ply/ply.mk b/package/ply/ply.mk
new file mode 100644
index 0000000000..298ea4acf2
--- /dev/null
+++ b/package/ply/ply.mk
@@ -0,0 +1,15 @@
+################################################################################
+#
+# ply
+#
+################################################################################
+
+PLY_VERSION = 2.1.1
+PLY_SITE = https://github.com/wkz/ply/releases/download/$(PLY_VERSION)
+# fetched from Github, with no configure script
+PLY_AUTORECONF = YES
+PLY_DEPENDENCIES = host-bison host-flex
+PLY_LICENSE = GPL-2.0+
+PLY_LICENSE_FILES = COPYING
+
+$(eval $(autotools-package))
ply 示例
网址 root/ply 有部分示例。clone 编译可直接使用：
$ git clone https://github.com/jgsun/buildroot
$ cd buildroot && make qemu_aarch64_virt-fun_defconfig && make
$ qemu-system-aarch64 -M virt \
-cpu cortex-a57 -nographic -smp 4 -m 512 \
-kernel output/images/Image \
-append "root=/dev/ram0 console=ttyAMA0 kmemleak=on loglevel=8" \
-netdev type=tap,ifname=tap0,id=eth0,script=board/qemu/scripts/qemu-ifup_virbr0,queues=2 \
-device virtio-net-pci,netdev=eth0,mac='00:00:00:01:00:01',vectors=6,mq=on
打印出内核函数的调用栈
本 ply 示例打印出函数 rtnetlink_rcv 的调用栈：
root@~# cat netlink.ply
#!/usr/bin/env ply
kprobe:rtnetlink_rcv
{
print(stack);
}
运行 sshnetlink.ply，然后在另外一个终端通过 ssh 登录 qemu aarch64 board，执行 ip addr show dev eth0 命令：
$ ssh -p 22 [email protected]
root@~# ip addr show dev eth0
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 00:00:00:01:00:01 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.46/24 brd 192.168.122.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::200:ff:fe01:1/64 scope link
valid_lft forever preferred_lft forever
netlink.ply 就会打印出 rtnetlink_rcv 的调用栈：
root@~# ./netlink.ply
ply: active
rtnetlink_rcv
netlink_sendmsg+408
____sys_sendmsg+592
___sys_sendmsg+136
__sys_sendmsg+112
__arm64_sys_sendmsg+40
el0_svc_common.constprop.3+144
do_el0_svc+116
el0_sync_handler+280
el0_sync+320
rtnetlink_rcv
netlink_sendmsg+408
____sys_sendmsg+592
___sys_sendmsg+136
__sys_sendmsg+112
__arm64_sys_sendmsg+40
el0_svc_common.constprop.3+144
do_el0_svc+116
el0_sync_handler+280
el0_sync+320
rtnetlink_rcv
netlink_sendmsg+408
__sys_sendto+224
__arm64_sys_sendto+44
el0_svc_common.constprop.3+144
do_el0_svc+116
el0_sync_handler+280
el0_sync+320
获取打开文件的进程信息
本 ply 示例打印出发起系统调用 do_sys_open 打开文件的进程名，进程 pid 和文件名（第一个参数）：
#!/usr/bin/env ply
kprobe:do_sys_open
{
printf("%v(%v): %s\n",
comm, pid, str(arg1));
}
运行结果：
root@~/ply# ./opensnoop.ply
ply: active
dropbear ( 128): /dev/urandom
dropbear ( 128): /proc/timer_list
syslogd ( 60): /var/log/messages
dropbear ( 128): /proc/interrupts
dropbear ( 128): /proc/loadavg
dropbear ( 128): /proc/sys/kernel/random/entropy_avail
dropbear ( 128): /proc/net/netstat
dropbear ( 128): /proc/net/dev
dropbear ( 128): /proc/net/tcp
dropbear ( 128): /proc/net/rt_cache