tests: net: Run networking tests only for selected platforms

[jukkar]

Run networking tests for native_posix, qemu_x86 and qemu_cortex_m3
platforms only as the tests are generic enough so no need to run
them in real physical device.

Signed-off-by: Jukka Rissanen jukka.rissanen@linux.intel.com

[jukkar]

We could even lower the number of tests if we skip testing in qemu_x86, that would lower the test count from 147 to 103 for tests/net directory. If this would be ok, I can send a new version that does that.

[jukkar]

Added fixed bugs list in commit message.

[codecov-io]

Codecov Report

Merging #9679 into master will decrease coverage by 0.18%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #9679      +/-   ##
==========================================
- Coverage   52.31%   52.12%   -0.19%     
==========================================
  Files         212      212              
  Lines       25939    25939              
  Branches     5590     5590              
==========================================
- Hits        13569    13521      -48     
- Misses      10123    10177      +54     
+ Partials     2247     2241       -6

Impacted Files	Coverage Δ
subsys/net/lib/app/server.c	41.41% <0%> (-11.12%)	⬇️
subsys/net/ip/net_stats.h	55.83% <0%> (-10.01%)	⬇️
subsys/net/lib/http/http.c	23.58% <0%> (-3.78%)	⬇️
subsys/net/lib/app/net_app.c	41.07% <0%> (-2.81%)	⬇️
subsys/net/ip/icmpv4.c	33.08% <0%> (-2.21%)	⬇️
subsys/net/ip/icmpv6.c	32.53% <0%> (-1.2%)	⬇️
subsys/net/lib/http/http_server.c	55.48% <0%> (-0.9%)	⬇️
subsys/net/ip/connection.c	78.05% <0%> (-0.25%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6dae106...3706357. Read the comment docs.

[pfalcon]

We could even lower the number of tests if we skip testing in qemu_x86

I think that would go too far.

[pfalcon]

Trying this patch together with #7831 (in the hope that it'll be possible to merge it afterwards!), and it uncovers some missed samples: samples/net/telnet, samples/net/dhcpv4_client, samples/net/dns_resolve, samples/net/nats, samples/net/syslog_net

[jukkar]

This PR only checks things under tests/net directory which was the whole point of this PR. It probably does not make sense to try to prevent sanitychecker from using samples/net programs.

[pfalcon]

This PR only checks things under tests/net directory which was the whole point of this PR. It probably does not make sense to try to prevent sanitychecker from using samples/net programs.

So, I don't see any good points in this PR - it decreases build coverage, which is generally bad, but could work as a stopgap measure to address #9015 until better solution comes up. But it stops short of fully masking #9015 and being able to resolve #7831.

[jukkar]

This PR does not decrease build coverage, it merely prevents unnecessary builds for tests/net directory. Those unit tests can safely be run in couple of architectures only.

[pfalcon]

This PR does not decrease build coverage

What about network driver code coverage from platforms which get excluded?

Anyway, I guess the best thing for me is to approve this, even if this doesn't clear the way for #7831, it goes in the needed direction (even if needed direction is a stopgap measure, again). (Otherwise, I thought someone else approves it anyway, surprised nobody did yet.)

[nashif]

well, tbh I do not like the approach here, we need to do this better, but right now this is the only way I can see it done. I will try to streamline how this will be done in sanitycheck in testing in general after 1.13.