Bug 3502 - LKFT: linux-mainline: x86: kselftest NET reuseport_bpf failed (new test)
Summary: LKFT: linux-mainline: x86: kselftest NET reuseport_bpf failed (new test)
Status: VERIFIED FIXED
Alias: None
Product: Kernel Functional Testing
Classification: Unclassified
Component: kselftest (show other bugs)
Version: linux-mainline
Hardware: x86 OpenEmbedded
: --- normal
Target Milestone: ---
Assignee: Anders Roxell
URL:
Keywords: Kernel, LKFT
Depends on:
Blocks:
 
Reported: 2017-11-27 19:24 UTC by Naresh Kamboju
Modified: 2018-06-29 15:39 UTC (History)
7 users (show)

See Also:
Linux kernel version: Mainline


Attachments
kselftest-full-run.log (41.91 KB, text/x-log)
2017-12-15 05:52 UTC, Naresh Kamboju
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Naresh Kamboju 2017-11-27 19:24:30 UTC
kselftest NET reuseport_bpf test cases newly added.
Test case FAILED on x86.
Test case PASS on Hikey, Juno and x15.

Test case History link:
https://qa-reports.linaro.org/lkft/linux-mainline-oe/tests/kselftest/reuseport_bpf

Test case:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/testing/selftests/net/reuseport_bpf.c

LAVA job id:
https://lkft.validation.linaro.org/scheduler/job/67061#L1303

 Linux version 4.15.0-rc1 (buildslave@x86-64-07) (gcc version 6.2.1 20161016 (Linaro GCC 6.2-2016.11)) #1 SMP Mon Nov 27 00:33:41 UTC 2017


metadata:
  git branch: master
  git repo: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
  git commit: 4fbd8d194f06c8a3fd2af1ce560ddb31f7ec8323
  git describe: v4.15-rc1
  make_kernelversion: "4.15.0-rc1"
  kernel-config: http://snapshots.linaro.org/openembedded/lkft/morty/intel-core2-32/rpb/linux-mainline/468/config
  kernel-defconfig: http://snapshots.linaro.org/openembedded/lkft/morty/intel-core2-32/rpb/linux-mainline/468/defconfig
  kselftest__url: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
  kselftest__version: "4.14+gitAUTOINC+4fbd8d194f"
  kselftest__revision: "4fbd8d194f06c8a3fd2af1ce560ddb31f7ec8323"
  build-url: https://ci.linaro.org/job/openembedded-lkft-linux-mainline/DISTRO=rpb,MACHINE=intel-core2-32,label=docker-stretch-amd64/468/
  build-location: http://snapshots.linaro.org/openembedded/lkft/morty/intel-core2-32/rpb/linux-mainline/468
Comment 1 Anmar Oueja 2017-11-28 15:31:49 UTC
@Naresh: Please run manually and confirm failure prior to reporting it upstream.
Comment 2 Dan Rue 2017-12-13 02:37:43 UTC
Failed on v4.9.68 and also v4.9.69-rc1.

Here's the history of the test on mainline: https://qa-reports.linaro.org/lkft/linux-mainline-oe/tests/kselftest/reuseport_bpf
Here's the history of the test on 4.9-rc: https://qa-reports.linaro.org/lkft/linux-stable-rc-4.9-oe/tests/kselftest/reuseport_bpf
Here's the history of the test on 4.9: https://qa-reports.linaro.org/lkft/linux-stable-4.9-oe/tests/kselftest/reuseport_bpf
Comment 3 Naresh Kamboju 2017-12-13 07:51:26 UTC
Additional investigation
Comment 4 Naresh Kamboju 2017-12-13 08:31:05 UTC
Additional investigation,

The independent test execution results as reuseport_bpf PASS on 
4.15.0-rc3+, 4.14.6-rc1+ and LTS-4.9.69-rc1+.

Test output:
------------
cd /opt/kselftests/mainline/net
./reuseport_bpf
---- IPv4 UDP ----
Testing EBPF mod 10...
<snippet>
SUCCESS
reuseport_bpf:PASS

Detailed test log on 4.15.0-rc3+,
https://lkft.validation.linaro.org/scheduler/job/76122#L979

Detailed test log on 4.14.6-rc1+
https://lkft.validation.linaro.org/scheduler/job/76119#L977

Detailed test log on LTS-4.9.69-rc1+.
https://lkft.validation.linaro.org/scheduler/job/76117#L962
Comment 5 Milosz Wasilewski 2017-12-13 11:29:41 UTC
I suspect some test before reuseport_bpf doesn't clean after itself.
Comment 6 Naresh Kamboju 2017-12-15 05:52:33 UTC
Created attachment 898 [details]
kselftest-full-run.log

kselftest-full-run.log
Comment 7 Naresh Kamboju 2017-12-15 06:02:53 UTC
One more experiment,
The same test case "reuseport_bpf" run 10 times in a row.
The first run pass and other runs failed.

+ cd /opt/kselftests/mainline/net
+ ./reuseport_bpf
---- IPv4 UDP ----
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing EBPF mod 20...
Reprograming, testing mod 10...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 20...
Reprograming, testing mod 10...
Testing too many filters...
Testing filters on non-SO_REUSEPORT socket...
---- IPv6 UDP ----
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing EBPF mod 20...
Reprograming, testing mod 10...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 20...
Reprograming, testing mod 10...
Testing too many filters...
Testing filters on non-SO_REUSEPORT socket...
---- IPv6 UDP w/ mapped IPv4 ----
Testing EBPF mod 20...
Reprograming, testing mod 10...
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 20...
Reprograming, testing mod 10...
---- IPv4 TCP ----
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing too many filters...
Testing filters on non-SO_REUSEPORT socket...
---- IPv6 TCP ----
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing too many filters...
Testing filters on non-SO_REUSEPORT socket...
---- IPv6 TCP w/ mapped IPv4 ----
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing filter add without bind...
SUCCESS
+ echo PASS
PASS
+ ./reuseport_bpf
---- IPv4 UDP ----
Testing EBPF mod 10...
Reprograming, testing mod 5...
./reuseport_bpf: ebpf error. log:
0: (bf) r6 = r1
1: (20) r0 = *(u32 *)skb[0]
2: (97) r0 %= 10
3: (95) exit
processed 4 insns
: Operation not permitted
+ echo FAIL
FAIL
+ ./reuseport_bpf
---- IPv4 UDP ----
Testing EBPF mod 10...
Reprograming, testing mod 5...
./reuseport_bpf: ebpf error. log:
0: (bf) r6 = r1
1: (20) r0 = *(u32 *)skb[0]
2: (97) r0 %= 10
3: (95) exit
processed 4 insns
: Operation not permitted
+ echo FAIL
FAIL
+ ./reuseport_bpf
---- IPv4 UDP ----
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing EBPF mod 20...
Reprograming, testing mod 10...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 20...
Reprograming, testing mod 10...
Testing too many filters...
Testing filters on non-SO_REUSEPORT socket...
---- IPv6 UDP ----
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing EBPF mod 20...
Reprograming, testing mod 10...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 20...
Reprograming, testing mod 10...
Testing too many filters...
Testing filters on non-SO_REUSEPORT socket...
---- IPv6 UDP w/ mapped IPv4 ----
Testing EBPF mod 20...
Reprograming, testing mod 10...
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 20...
Reprograming, testing mod 10...
---- IPv4 TCP ----
Testing EBPF mod 10...
./reuseport_bpf: failed to bind send socket: Address already in use
+ echo FAIL
FAIL
+ ./reuseport_bpf
---- IPv4 UDP ----
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing EBPF mod 20...
Reprograming, testing mod 10...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 20...
Reprograming, testing mod 10...
Testing too many filters...
Testing filters on non-SO_REUSEPORT socket...
---- IPv6 UDP ----
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing EBPF mod 20...
Reprograming, testing mod 10...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 20...
Reprograming, testing mod 10...
Testing too many filters...
Testing filters on non-SO_REUSEPORT socket...
---- IPv6 UDP w/ mapped IPv4 ----
Testing EBPF mod 20...
Reprograming, testing mod 10...
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 20...
Reprograming, testing mod 10...
---- IPv4 TCP ----
Testing EBPF mod 10...
./reuseport_bpf: failed to bind send socket: Address already in use
+ echo FAIL
FAIL
+ ./reuseport_bpf
---- IPv4 UDP ----
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing EBPF mod 20...
Reprograming, testing mod 10...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 20...
Reprograming, testing mod 10...
Testing too many filters...
Testing filters on non-SO_REUSEPORT socket...
---- IPv6 UDP ----
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing EBPF mod 20...
Reprograming, testing mod 10...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 20...
Reprograming, testing mod 10...
Testing too many filters...
Testing filters on non-SO_REUSEPORT socket...
---- IPv6 UDP w/ mapped IPv4 ----
Testing EBPF mod 20...
Reprograming, testing mod 10...
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 20...
Reprograming, testing mod 10...
---- IPv4 TCP ----
Testing EBPF mod 10...
./reuseport_bpf: failed to bind send socket: Address already in use
+ echo FAIL
FAIL
+ ./reuseport_bpf
---- IPv4 UDP ----
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing EBPF mod 20...
Reprograming, testing mod 10...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 20...
Reprograming, testing mod 10...
Testing too many filters...
Testing filters on non-SO_REUSEPORT socket...
---- IPv6 UDP ----
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing EBPF mod 20...
Reprograming, testing mod 10...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 20...
Reprograming, testing mod 10...
Testing too many filters...
Testing filters on non-SO_REUSEPORT socket...
---- IPv6 UDP w/ mapped IPv4 ----
Testing EBPF mod 20...
Reprograming, testing mod 10...
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 20...
Reprograming, testing mod 10...
---- IPv4 TCP ----
Testing EBPF mod 10...
./reuseport_bpf: failed to bind send socket: Address already in use
+ echo FAIL
FAIL
+ ./reuseport_bpf
---- IPv4 UDP ----
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing EBPF mod 20...
Reprograming, testing mod 10...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 20...
Reprograming, testing mod 10...
Testing too many filters...
Testing filters on non-SO_REUSEPORT socket...
---- IPv6 UDP ----
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing EBPF mod 20...
Reprograming, testing mod 10...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 20...
Reprograming, testing mod 10...
Testing too many filters...
Testing filters on non-SO_REUSEPORT socket...
---- IPv6 UDP w/ mapped IPv4 ----
Testing EBPF mod 20...
Reprograming, testing mod 10...
Testing EBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 10...
Reprograming, testing mod 5...
Testing CBPF mod 20...
Reprograming, testing mod 10...
---- IPv4 TCP ----
Testing EBPF mod 10...
./reuseport_bpf: failed to bind send socket: Address already in use
+ echo FAIL
FAIL
Comment 8 Naresh Kamboju 2017-12-15 06:04:11 UTC
In the full log we do not have verbose or debug information to trace what failed.

Running tests in net
========================================
selftests: reuseport_bpf [FAIL]
selftests: reuseport_bpf_cpu [PASS]
selftests: reuseport_bpf_numa [PASS]
selftests: reuseport_dualstack [PASS]
selftests: reuseaddr_conflict [PASS]
selftests: run_netsocktests [PASS]
selftests: run_afpackettests [PASS]
Comment 9 Naresh Kamboju 2017-12-16 05:54:26 UTC
reuseport_bpf test failed on x15 running 4.9.70-rc1-00028-gcc35c65be8bb-dirty.
Full run failed and independent execution pass.

LAVA job id:
https://lkft.validation.linaro.org/scheduler/job/78380#L3149

Comparison history is here,
https://qa-reports.linaro.org/lkft/linux-stable-rc-4.9-oe/tests/kselftest/reuseport_bpf
Comment 10 Milosz Wasilewski 2018-02-06 15:18:14 UTC
@Naresh, please report this to bpf maintainers as separate issue.
Comment 11 Naresh Kamboju 2018-02-08 12:52:09 UTC
[Daniel Borkmann wrote on 9 Feb 2018]

Below should fix these two issues. (For checking the bpf selftests, we should
probably adapt similar scheme with ctor/dtor to act more graceful.)

 tools/testing/selftests/net/reuseport_bpf.c | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/net/reuseport_bpf.c b/tools/testing/selftests/net/reuseport_bpf.c
index 4a82174..cad14cd 100644
--- a/tools/testing/selftests/net/reuseport_bpf.c
+++ b/tools/testing/selftests/net/reuseport_bpf.c
@@ -21,6 +21,7 @@
 #include <sys/epoll.h>
 #include <sys/types.h>
 #include <sys/socket.h>
+#include <sys/resource.h>
 #include <unistd.h>

 #ifndef ARRAY_SIZE
@@ -190,11 +191,14 @@ static void send_from(struct test_params p, uint16_t sport, char *buf,
        struct sockaddr * const saddr = new_any_sockaddr(p.send_family, sport);
        struct sockaddr * const daddr =
                new_loopback_sockaddr(p.send_family, p.recv_port);
-       const int fd = socket(p.send_family, p.protocol, 0);
+       const int fd = socket(p.send_family, p.protocol, 0), one = 1;

        if (fd < 0)
                error(1, errno, "failed to create send socket");

+       if (setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &one, sizeof(one)))
+               error(1, errno, "failed to set reuseaddr");
+
        if (bind(fd, saddr, sockaddr_size()))
                error(1, errno, "failed to bind send socket");

@@ -433,6 +437,21 @@ void enable_fastopen(void)
        }
 }

+static struct rlimit rlim_old, rlim_new;
+
+static  __attribute__((constructor)) void main_ctor(void)
+{
+       getrlimit(RLIMIT_MEMLOCK, &rlim_old);
+       rlim_new.rlim_cur = rlim_old.rlim_cur + (1UL << 20);
+       rlim_new.rlim_max = rlim_old.rlim_max + (1UL << 20);
+       setrlimit(RLIMIT_MEMLOCK, &rlim_new);
+}
+
+static __attribute__((destructor)) void main_dtor(void)
+{
+       setrlimit(RLIMIT_MEMLOCK, &rlim_old);
+}
+
 int main(void)
 {
        fprintf(stderr, "---- IPv4 UDP ----\n");
Comment 12 Naresh Kamboju 2018-05-21 15:33:31 UTC
Test PASS on x86_64 running mainline kernel.
<>
SUCCESS
selftests: reuseport_bpf [PASS]

https://lkft.validation.linaro.org/scheduler/job/221513#L6100

Linux mainline kernel version:
Linux version 4.17.0-rc5 (buildslave@x86-64-09) (gcc version 6.2.1 20161016 
(Linaro GCC 6.2-2016.11)) #1 SMP Fri May 18 19:36:49 UTC 2018


History of test case results,

Staging results of mainline shows test PASS.
https://qa-reports.linaro.org/staging-lkft/linux-mainline/tests/kselftest/reuseport_bpf

Staging results of 4.16 (v4.16.3) shows test PASS.
https://qa-reports.linaro.org/staging-lkft/linux-stable-4.16/tests/kselftest/reuseport_bpf

Staging results of 4.14 (v4.14.35) shows test PASS.
https://qa-reports.linaro.org/staging-lkft/linux-stable-4.14/tests/kselftest/reuseport_bpf