kselftest NET reuseport_bpf test cases newly added. Test case FAILED on x86. Test case PASS on Hikey, Juno and x15. Test case History link: https://qa-reports.linaro.org/lkft/linux-mainline-oe/tests/kselftest/reuseport_bpf Test case: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/net/reuseport_bpf.c LAVA job id: https://lkft.validation.linaro.org/scheduler/job/67061#L1303 Linux version 4.15.0-rc1 (buildslave@x86-64-07) (gcc version 6.2.1 20161016 (Linaro GCC 6.2-2016.11)) #1 SMP Mon Nov 27 00:33:41 UTC 2017 metadata: git branch: master git repo: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git git commit: 4fbd8d194f06c8a3fd2af1ce560ddb31f7ec8323 git describe: v4.15-rc1 make_kernelversion: "4.15.0-rc1" kernel-config: http://snapshots.linaro.org/openembedded/lkft/morty/intel-core2-32/rpb/linux-mainline/468/config kernel-defconfig: http://snapshots.linaro.org/openembedded/lkft/morty/intel-core2-32/rpb/linux-mainline/468/defconfig kselftest__url: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git kselftest__version: "4.14+gitAUTOINC+4fbd8d194f" kselftest__revision: "4fbd8d194f06c8a3fd2af1ce560ddb31f7ec8323" build-url: https://ci.linaro.org/job/openembedded-lkft-linux-mainline/DISTRO=rpb,MACHINE=intel-core2-32,label=docker-stretch-amd64/468/ build-location: http://snapshots.linaro.org/openembedded/lkft/morty/intel-core2-32/rpb/linux-mainline/468
@Naresh: Please run manually and confirm failure prior to reporting it upstream.
Failed on v4.9.68 and also v4.9.69-rc1. Here's the history of the test on mainline: https://qa-reports.linaro.org/lkft/linux-mainline-oe/tests/kselftest/reuseport_bpf Here's the history of the test on 4.9-rc: https://qa-reports.linaro.org/lkft/linux-stable-rc-4.9-oe/tests/kselftest/reuseport_bpf Here's the history of the test on 4.9: https://qa-reports.linaro.org/lkft/linux-stable-4.9-oe/tests/kselftest/reuseport_bpf
Additional investigation
Additional investigation, The independent test execution results as reuseport_bpf PASS on 4.15.0-rc3+, 4.14.6-rc1+ and LTS-4.9.69-rc1+. Test output: ------------ cd /opt/kselftests/mainline/net ./reuseport_bpf ---- IPv4 UDP ---- Testing EBPF mod 10... <snippet> SUCCESS reuseport_bpf:PASS Detailed test log on 4.15.0-rc3+, https://lkft.validation.linaro.org/scheduler/job/76122#L979 Detailed test log on 4.14.6-rc1+ https://lkft.validation.linaro.org/scheduler/job/76119#L977 Detailed test log on LTS-4.9.69-rc1+. https://lkft.validation.linaro.org/scheduler/job/76117#L962
I suspect some test before reuseport_bpf doesn't clean after itself.
Created attachment 898 [details] kselftest-full-run.log kselftest-full-run.log
One more experiment, The same test case "reuseport_bpf" run 10 times in a row. The first run pass and other runs failed. + cd /opt/kselftests/mainline/net + ./reuseport_bpf ---- IPv4 UDP ---- Testing EBPF mod 10... Reprograming, testing mod 5... Testing EBPF mod 20... Reprograming, testing mod 10... Testing CBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 20... Reprograming, testing mod 10... Testing too many filters... Testing filters on non-SO_REUSEPORT socket... ---- IPv6 UDP ---- Testing EBPF mod 10... Reprograming, testing mod 5... Testing EBPF mod 20... Reprograming, testing mod 10... Testing CBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 20... Reprograming, testing mod 10... Testing too many filters... Testing filters on non-SO_REUSEPORT socket... ---- IPv6 UDP w/ mapped IPv4 ---- Testing EBPF mod 20... Reprograming, testing mod 10... Testing EBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 20... Reprograming, testing mod 10... ---- IPv4 TCP ---- Testing EBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 10... Reprograming, testing mod 5... Testing too many filters... Testing filters on non-SO_REUSEPORT socket... ---- IPv6 TCP ---- Testing EBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 10... Reprograming, testing mod 5... Testing too many filters... Testing filters on non-SO_REUSEPORT socket... ---- IPv6 TCP w/ mapped IPv4 ---- Testing EBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 10... Reprograming, testing mod 5... Testing filter add without bind... SUCCESS + echo PASS PASS + ./reuseport_bpf ---- IPv4 UDP ---- Testing EBPF mod 10... Reprograming, testing mod 5... ./reuseport_bpf: ebpf error. log: 0: (bf) r6 = r1 1: (20) r0 = *(u32 *)skb[0] 2: (97) r0 %= 10 3: (95) exit processed 4 insns : Operation not permitted + echo FAIL FAIL + ./reuseport_bpf ---- IPv4 UDP ---- Testing EBPF mod 10... Reprograming, testing mod 5... ./reuseport_bpf: ebpf error. log: 0: (bf) r6 = r1 1: (20) r0 = *(u32 *)skb[0] 2: (97) r0 %= 10 3: (95) exit processed 4 insns : Operation not permitted + echo FAIL FAIL + ./reuseport_bpf ---- IPv4 UDP ---- Testing EBPF mod 10... Reprograming, testing mod 5... Testing EBPF mod 20... Reprograming, testing mod 10... Testing CBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 20... Reprograming, testing mod 10... Testing too many filters... Testing filters on non-SO_REUSEPORT socket... ---- IPv6 UDP ---- Testing EBPF mod 10... Reprograming, testing mod 5... Testing EBPF mod 20... Reprograming, testing mod 10... Testing CBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 20... Reprograming, testing mod 10... Testing too many filters... Testing filters on non-SO_REUSEPORT socket... ---- IPv6 UDP w/ mapped IPv4 ---- Testing EBPF mod 20... Reprograming, testing mod 10... Testing EBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 20... Reprograming, testing mod 10... ---- IPv4 TCP ---- Testing EBPF mod 10... ./reuseport_bpf: failed to bind send socket: Address already in use + echo FAIL FAIL + ./reuseport_bpf ---- IPv4 UDP ---- Testing EBPF mod 10... Reprograming, testing mod 5... Testing EBPF mod 20... Reprograming, testing mod 10... Testing CBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 20... Reprograming, testing mod 10... Testing too many filters... Testing filters on non-SO_REUSEPORT socket... ---- IPv6 UDP ---- Testing EBPF mod 10... Reprograming, testing mod 5... Testing EBPF mod 20... Reprograming, testing mod 10... Testing CBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 20... Reprograming, testing mod 10... Testing too many filters... Testing filters on non-SO_REUSEPORT socket... ---- IPv6 UDP w/ mapped IPv4 ---- Testing EBPF mod 20... Reprograming, testing mod 10... Testing EBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 20... Reprograming, testing mod 10... ---- IPv4 TCP ---- Testing EBPF mod 10... ./reuseport_bpf: failed to bind send socket: Address already in use + echo FAIL FAIL + ./reuseport_bpf ---- IPv4 UDP ---- Testing EBPF mod 10... Reprograming, testing mod 5... Testing EBPF mod 20... Reprograming, testing mod 10... Testing CBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 20... Reprograming, testing mod 10... Testing too many filters... Testing filters on non-SO_REUSEPORT socket... ---- IPv6 UDP ---- Testing EBPF mod 10... Reprograming, testing mod 5... Testing EBPF mod 20... Reprograming, testing mod 10... Testing CBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 20... Reprograming, testing mod 10... Testing too many filters... Testing filters on non-SO_REUSEPORT socket... ---- IPv6 UDP w/ mapped IPv4 ---- Testing EBPF mod 20... Reprograming, testing mod 10... Testing EBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 20... Reprograming, testing mod 10... ---- IPv4 TCP ---- Testing EBPF mod 10... ./reuseport_bpf: failed to bind send socket: Address already in use + echo FAIL FAIL + ./reuseport_bpf ---- IPv4 UDP ---- Testing EBPF mod 10... Reprograming, testing mod 5... Testing EBPF mod 20... Reprograming, testing mod 10... Testing CBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 20... Reprograming, testing mod 10... Testing too many filters... Testing filters on non-SO_REUSEPORT socket... ---- IPv6 UDP ---- Testing EBPF mod 10... Reprograming, testing mod 5... Testing EBPF mod 20... Reprograming, testing mod 10... Testing CBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 20... Reprograming, testing mod 10... Testing too many filters... Testing filters on non-SO_REUSEPORT socket... ---- IPv6 UDP w/ mapped IPv4 ---- Testing EBPF mod 20... Reprograming, testing mod 10... Testing EBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 20... Reprograming, testing mod 10... ---- IPv4 TCP ---- Testing EBPF mod 10... ./reuseport_bpf: failed to bind send socket: Address already in use + echo FAIL FAIL + ./reuseport_bpf ---- IPv4 UDP ---- Testing EBPF mod 10... Reprograming, testing mod 5... Testing EBPF mod 20... Reprograming, testing mod 10... Testing CBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 20... Reprograming, testing mod 10... Testing too many filters... Testing filters on non-SO_REUSEPORT socket... ---- IPv6 UDP ---- Testing EBPF mod 10... Reprograming, testing mod 5... Testing EBPF mod 20... Reprograming, testing mod 10... Testing CBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 20... Reprograming, testing mod 10... Testing too many filters... Testing filters on non-SO_REUSEPORT socket... ---- IPv6 UDP w/ mapped IPv4 ---- Testing EBPF mod 20... Reprograming, testing mod 10... Testing EBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 10... Reprograming, testing mod 5... Testing CBPF mod 20... Reprograming, testing mod 10... ---- IPv4 TCP ---- Testing EBPF mod 10... ./reuseport_bpf: failed to bind send socket: Address already in use + echo FAIL FAIL
In the full log we do not have verbose or debug information to trace what failed. Running tests in net ======================================== selftests: reuseport_bpf [FAIL] selftests: reuseport_bpf_cpu [PASS] selftests: reuseport_bpf_numa [PASS] selftests: reuseport_dualstack [PASS] selftests: reuseaddr_conflict [PASS] selftests: run_netsocktests [PASS] selftests: run_afpackettests [PASS]
reuseport_bpf test failed on x15 running 4.9.70-rc1-00028-gcc35c65be8bb-dirty. Full run failed and independent execution pass. LAVA job id: https://lkft.validation.linaro.org/scheduler/job/78380#L3149 Comparison history is here, https://qa-reports.linaro.org/lkft/linux-stable-rc-4.9-oe/tests/kselftest/reuseport_bpf
@Naresh, please report this to bpf maintainers as separate issue.
[Daniel Borkmann wrote on 9 Feb 2018] Below should fix these two issues. (For checking the bpf selftests, we should probably adapt similar scheme with ctor/dtor to act more graceful.) tools/testing/selftests/net/reuseport_bpf.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/net/reuseport_bpf.c b/tools/testing/selftests/net/reuseport_bpf.c index 4a82174..cad14cd 100644 --- a/tools/testing/selftests/net/reuseport_bpf.c +++ b/tools/testing/selftests/net/reuseport_bpf.c @@ -21,6 +21,7 @@ #include <sys/epoll.h> #include <sys/types.h> #include <sys/socket.h> +#include <sys/resource.h> #include <unistd.h> #ifndef ARRAY_SIZE @@ -190,11 +191,14 @@ static void send_from(struct test_params p, uint16_t sport, char *buf, struct sockaddr * const saddr = new_any_sockaddr(p.send_family, sport); struct sockaddr * const daddr = new_loopback_sockaddr(p.send_family, p.recv_port); - const int fd = socket(p.send_family, p.protocol, 0); + const int fd = socket(p.send_family, p.protocol, 0), one = 1; if (fd < 0) error(1, errno, "failed to create send socket"); + if (setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &one, sizeof(one))) + error(1, errno, "failed to set reuseaddr"); + if (bind(fd, saddr, sockaddr_size())) error(1, errno, "failed to bind send socket"); @@ -433,6 +437,21 @@ void enable_fastopen(void) } } +static struct rlimit rlim_old, rlim_new; + +static __attribute__((constructor)) void main_ctor(void) +{ + getrlimit(RLIMIT_MEMLOCK, &rlim_old); + rlim_new.rlim_cur = rlim_old.rlim_cur + (1UL << 20); + rlim_new.rlim_max = rlim_old.rlim_max + (1UL << 20); + setrlimit(RLIMIT_MEMLOCK, &rlim_new); +} + +static __attribute__((destructor)) void main_dtor(void) +{ + setrlimit(RLIMIT_MEMLOCK, &rlim_old); +} + int main(void) { fprintf(stderr, "---- IPv4 UDP ----\n");
Test PASS on x86_64 running mainline kernel. <> SUCCESS selftests: reuseport_bpf [PASS] https://lkft.validation.linaro.org/scheduler/job/221513#L6100 Linux mainline kernel version: Linux version 4.17.0-rc5 (buildslave@x86-64-09) (gcc version 6.2.1 20161016 (Linaro GCC 6.2-2016.11)) #1 SMP Fri May 18 19:36:49 UTC 2018 History of test case results, Staging results of mainline shows test PASS. https://qa-reports.linaro.org/staging-lkft/linux-mainline/tests/kselftest/reuseport_bpf Staging results of 4.16 (v4.16.3) shows test PASS. https://qa-reports.linaro.org/staging-lkft/linux-stable-4.16/tests/kselftest/reuseport_bpf Staging results of 4.14 (v4.14.35) shows test PASS. https://qa-reports.linaro.org/staging-lkft/linux-stable-4.14/tests/kselftest/reuseport_bpf