Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
156 commits
Select commit Hold shift + click to select a range
429e89f
Checkpointing.
jiceathome May 5, 2025
bad6172
Merge branch 'master' into router_raw_socket2
jiceathome May 5, 2025
05bf0b7
Checkpointing.
jiceathome May 5, 2025
0e75647
Checkpoint.
jiceathome May 6, 2025
532bbc1
Checkpoint. Presumably code-complete for IPv4. Need to add IPv6 support.
jiceathome May 7, 2025
a7f808f
Checkpoint.
jiceathome May 8, 2025
1a7216e
Removed leftover printks.
jiceathome May 8, 2025
ed60148
Sorted out libebpf objects life cycle issues.
jiceathome May 9, 2025
e6f1520
Give required capability in the router's container. Enable afpacket u…
jiceathome May 9, 2025
bb0c07e
Merge branch 'master' into router_raw_socket2
jiceathome May 19, 2025
515522a
Restored ability to run router_benchmark in three steps (setup/run/te…
jiceathome May 20, 2025
7fd83d9
Minor cleanup
jiceathome May 20, 2025
99dc941
Fix assorted trivial bugs.
jiceathome May 20, 2025
05a1cc7
Lintify etc.
jiceathome May 20, 2025
b2834e7
Clarify role of the Priority() method of the Underlay interface.
jiceathome May 21, 2025
14ef830
Trigger new build from scratch
jiceathome May 21, 2025
2665de6
Delete dummy comment
jiceathome May 22, 2025
1e779ba
Merge branch 'master' into router_raw_socket2
jiceathome May 22, 2025
7d70a22
Make test portfiter test a native_test.
jiceathome May 23, 2025
1f344c3
Lintify etc.
jiceathome May 23, 2025
bc49771
Attempt to work around new buildkite issue of unsifficient disk cleanup.
jiceathome May 23, 2025
a6f85b2
Fix bug in WithHeader. Was making all output packets 8K long.
jiceathome May 24, 2025
37dfdbc
Increase the router benchmark's packet blaster's batch size.
jiceathome May 24, 2025
3f4da2e
For now be less demanding interms of router saturation.
jiceathome May 24, 2025
2b5e781
Add ipv6 support to the afpudpip underlay.
jiceathome May 27, 2025
ef87482
Missing space.
jiceathome May 27, 2025
4cdefda
Cleanup. Split the afp underlay in 4.
jiceathome May 27, 2025
efea026
Added ARP support.
jiceathome May 29, 2025
a1b3c8c
Cleanup.
jiceathome May 30, 2025
1e1bd9c
Add ipv6 NDP support.
jiceathome Jun 3, 2025
f441cb2
Fixed filtering for v6 packets.
jiceathome Jun 3, 2025
3bfa830
Experiment with giving more cpus to the router in thr benchmark test.
jiceathome Jun 3, 2025
5d6f850
Experiment with a different number of cores.
jiceathome Jun 3, 2025
c21e108
Another CI test for MAX_CPUS
jiceathome Jun 3, 2025
b43a257
Reverting to the std cpu count.
jiceathome Jun 3, 2025
281c363
Improve dropped packets metrics and give one more cpu to the traffic …
jiceathome Jun 4, 2025
20df900
Make sure our raw sockets don't receive their own outgoing traffic.
jiceathome Jun 4, 2025
57d719b
Improve the router_multi test. Small fix in router underlay.
jiceathome Jun 5, 2025
e9b0f38
Finish arp handling in the router_multi test.
jiceathome Jun 5, 2025
b1b0d86
In router_multi honor the ignoreNoneMatching test flag in all relevan…
jiceathome Jun 5, 2025
30c2cbd
Try giving the necessary caps to the development router.
jiceathome Jun 6, 2025
59e3221
Fix some V6-specific bugs and enable supervisord tests.
jiceathome Jun 6, 2025
5e3aa5f
Simpleminded attempt at making this test less flaky.
jiceathome Jun 9, 2025
f6257b7
Add support for arp/ndp annoucements.
jiceathome Jun 9, 2025
f0ccdba
Fix arp annoucement in ptp links.
jiceathome Jun 9, 2025
626a2d2
Improved arp cache and moved to its own file.
jiceathome Jun 9, 2025
623424c
Take mutex while ticking the cache.
jiceathome Jun 9, 2025
a821592
lintified
jiceathome Jun 10, 2025
7db81ce
Factor more arp/ndp out to neighbors.go
jiceathome Jun 10, 2025
d9d6441
Don't forget to sleep between ticks.
jiceathome Jun 10, 2025
e77c8ed
Cleanup some docker-compose config.
jiceathome Jun 12, 2025
cce0a20
Arrange to give proper capabilities to routers installed as packages.
jiceathome Jun 13, 2025
7ffda73
Simplified neighbor management and made more permissive.
jiceathome Jun 13, 2025
fb828b0
Assortment of bug fixes.
jiceathome Jun 16, 2025
32fe60c
Fix NDP handling.
jiceathome Jun 17, 2025
91574fd
Lintify.
jiceathome Jun 17, 2025
a39416f
Tweak retries and timeouts.
jiceathome Jun 20, 2025
a022a22
Roundtriper unit test now expects SetReadDedline and SetWriteDeadline.
jiceathome Jun 21, 2025
d4c5783
Lintify and remove the writeDeadline in resolver.
jiceathome Jun 21, 2025
76991d7
Add trace for issue unique to the CI system.
jiceathome Jun 21, 2025
644c8b4
Improve port filtering.
jiceathome Jun 22, 2025
7a28063
Address resolution improvements.
jiceathome Jun 23, 2025
bb21f66
Lintify
jiceathome Jun 23, 2025
02b12cc
Assorted bug fixes and small improvements.
jiceathome Jun 25, 2025
d1c6a5f
delete reference to unused module.
jiceathome Jun 25, 2025
edd9344
Add IP address to kfilter key. Do not try to resolved v6 addrs on loo…
jiceathome Jun 26, 2025
99e3284
Finally implement a non-lossy ARP/NDP.
jiceathome Jul 3, 2025
c2d5945
Fix raw socket underlay latency.
jiceathome Jul 4, 2025
e82995c
Delete another unnecessary work-around.
jiceathome Jul 4, 2025
3b83d41
sysadmin cap isn't necessary.
jiceathome Jul 4, 2025
c75f880
Remove unused import
jiceathome Jul 4, 2025
5329dde
Further reduce latency. Do not populate cache with NDP dup-address de…
jiceathome Jul 4, 2025
cca2dcc
Add back a little slack to the hidden_paths test.
jiceathome Jul 4, 2025
614f22e
Slight cleanup of the neighbor management. Make the supervisor mode w…
jiceathome Jul 6, 2025
a6629b1
Try a cleaner way of setting systcl flags.
jiceathome Jul 6, 2025
f4593b1
Make the systcl command more robust.
jiceathome Jul 7, 2025
1389c77
Make address-based filtering more efficient.
jiceathome Jul 7, 2025
719fd15
Reduce GC pressure caused by passing src address to internal link.
jiceathome Jul 7, 2025
4fc7eff
Rolling back previous change. That was broken.
jiceathome Jul 7, 2025
4befb94
Try and make the supervisor-based tests not so slow on CI.
jiceathome Jul 7, 2025
1c97657
Reduce cost of link mapping and src address handling.
jiceathome Jul 7, 2025
b060040
Undo the tentative sysctl changes. It doesn't help a bit.
jiceathome Jul 8, 2025
ab11eb7
Cheaper link mapping. (and hopefully correct too).
jiceathome Jul 8, 2025
108d267
Experiment with docker configuratoin for the scion_integration test.
jiceathome Jul 8, 2025
51d3bf5
Fix the ping test so it doesn't spend a minimum of one second per rou…
jiceathome Jul 8, 2025
d8df7ad
Revert scion_integration test to using the supervisor.
jiceathome Jul 8, 2025
09a2ecf
Renounce the retries in the router_multi tests.
jiceathome Jul 8, 2025
fbc6dd2
Attempt to solve what appears to be a latency issue.
jiceathome Jul 8, 2025
ef9faac
Add an rtt check in the router benchmark.
jiceathome Jul 10, 2025
0602ebd
All of a sudden buildkit doesn't like "key:" after "group:".
jiceathome Jul 11, 2025
43644e3
Trying to start the bazel remote cache LAST (we exit if we can't).
jiceathome Jul 11, 2025
00860d9
Tracing the pipeline generation.
jiceathome Jul 11, 2025
32312b1
more traceing
jiceathome Jul 11, 2025
e925a62
Undo previous 3 comits. Got what I needed.
jiceathome Jul 11, 2025
04b7738
trace bazel's whereabouts.
jiceathome Jul 11, 2025
240b1cc
more bazel scrutiny.
jiceathome Jul 11, 2025
76b801b
Export PATH?
jiceathome Jul 11, 2025
f7b3f76
Move path setting to environment hook.
jiceathome Jul 11, 2025
e4b5189
more fiddling
jiceathome Jul 11, 2025
f0757fa
more grasping at strwas in the dark.
jiceathome Jul 11, 2025
c984e72
can't make that one work.
jiceathome Jul 11, 2025
051d85f
Reverting all my prior junk.
jiceathome Jul 11, 2025
bde899f
Give 20 minutes for the longuest end2end test to complete on CI.
jiceathome Jul 11, 2025
4d72ce2
Lintify.
jiceathome Jul 11, 2025
7854726
Merge branch 'master' into router_raw_socket2
jiceathome Jul 11, 2025
3609047
Delete useless diff.
jiceathome Jul 11, 2025
6f89e9a
Fix merge fail.
jiceathome Jul 11, 2025
3ab2086
Delet blank space changes and remove unnecessary cap_sys_admin capabi…
jiceathome Jul 11, 2025
840147b
Lintify.
jiceathome Jul 11, 2025
2146bbd
Roll back retry in dameon rpc.
jiceathome Jul 11, 2025
2e45295
Long-shot experiments to try and find the cause of the new failures.
jiceathome Jul 11, 2025
7f11f36
Spaces not tabs.
jiceathome Jul 11, 2025
6a2872a
Try and suppress connectrpc use.
jiceathome Jul 11, 2025
e614fad
Try and increase the happy-eyeballs delay.
jiceathome Jul 11, 2025
b06a52e
Restoring happy.go as it was.
jiceathome Jul 11, 2025
f0c171a
Fix the happy eyeballs error collection.
jiceathome Jul 12, 2025
449cad4
Merge branch 'master' into router_raw_socket2
jiceathome Jul 15, 2025
3785654
Merge branch 'master' into router_raw_socket2
jiceathome Jul 15, 2025
043bcec
Retrying with a slightly longer timeout
jiceathome Jul 16, 2025
b0ed723
Merge branch 'master' into router_raw_socket2
jiceathome Jul 17, 2025
9b1c232
Unbreak benchmark.py.
jiceathome Jul 18, 2025
6f1d1d7
Minor fixes to brload and benchmark.py.
jiceathome Jul 21, 2025
ca1d4e9
Fix the handling of the debug_run and log_level flags.
jiceathome Jul 22, 2025
aaab0ca
enable profiling while running the real-machine benchmark.
jiceathome Jul 22, 2025
81c62e2
Make sure we run the profiler in the background rather than Before th…
jiceathome Jul 22, 2025
09c1e05
Make it easier to get two kinds of profiling data when running the st…
jiceathome Jul 23, 2025
323a4be
Make packet sending asynchronous. Cleanup sendAll() errors handling.
jiceathome Jul 23, 2025
6531ec1
appease lint.
jiceathome Jul 23, 2025
4d3b3d8
Minor improvements to the benhcmark scripts.
jiceathome Jul 23, 2025
238480f
Make the traffic generator sender asynchronous too.
jiceathome Jul 23, 2025
c297d94
Replace the underlay priority by a name and a preference map.
jiceathome Jul 23, 2025
39fa33d
Bypass queueing discpline when sending with SEND_MMSG.
jiceathome Jul 24, 2025
1a57a4a
log.Warn -> log.Info
jiceathome Jul 24, 2025
1b09e97
Delete commented-out change.
jiceathome Jul 24, 2025
c59523c
Implement anumber of reviewrs suggestions.
jiceathome Jul 24, 2025
544b7b0
Fix naming of underlay interfaces. They were confusing.
jiceathome Jul 24, 2025
f6c01e3
Fix spelling.
jiceathome Jul 24, 2025
41bb676
Lintify
jiceathome Jul 24, 2025
47bac4a
Minor tweaks to brload and the standalone benchmark instructions.
jiceathome Jul 26, 2025
bed5de5
Minor tweak.
jiceathome Jul 26, 2025
27e65cb
Lintify
jiceathome Jul 26, 2025
182dc52
Merge branch 'master' into router_raw_socket2
jiceathome Jul 28, 2025
fdc963f
Unbreak the standalone benchmark for the regular inet router underlay.
jiceathome Jul 29, 2025
11987bc
Lintify
jiceathome Jul 29, 2025
2aaca25
Enable overriding the noral IP addresses of the benchmark.
jiceathome Jul 30, 2025
2421155
Handle IP overrides properly.
jiceathome Jul 30, 2025
cedc24a
Enable the IP overrides flags in the test harness.
jiceathome Jul 30, 2025
3270c16
Minor fix to benchmark to enable running on aws.
jiceathome Jul 31, 2025
2e0d28a
Supply new fields required by benchmarklib.
jiceathome Jul 31, 2025
8d632c1
Clear the UDP checksum *before* sending the first packet.
jiceathome Jul 31, 2025
b433b38
Remove the need to allocate src addresses in the afpacket underlay.
jiceathome Jul 31, 2025
2587025
Merge branch 'master' into router_raw_socket2
jiceathome Aug 12, 2025
b4cacff
Implement reviewers suggestion of additional link options.
jiceathome Aug 23, 2025
ed4096f
Address reviewer's comments.
jiceathome Oct 3, 2025
cc838cb
Merge branch 'master' into router_raw_socket2
jiceathome Oct 3, 2025
a9ec125
Merge branch 'master' into router_raw_socket2
katyatitkova Nov 4, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .buildkite/pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ steps:
- ./scion.sh topology -c topology/default.topo
- ./scion.sh run
- tools/await-connectivity
- sleep 5
- ./bin/scion_integration || ( echo "^^^ +++" && false )
- ./bin/end2end_integration || ( echo "^^^ +++" && false )
plugins: &scion-run-hooks
Expand All @@ -141,7 +142,7 @@ steps:
pre-exit: .buildkite/cleanup-leftovers.sh
artifact_paths: &scion-run-artifact-paths
- test-out.tar.gz
timeout_in_minutes: 15
timeout_in_minutes: 20
key: e2e_integration_tests_v2
retry: *automatic-retry
- label: "E2E: failing links :man_in_business_suit_levitating:"
Expand All @@ -152,6 +153,7 @@ steps:
- ./scion.sh topology -c topology/default-no-peers.topo
- ./scion.sh run
- tools/await-connectivity
- sleep 5
- ./bin/end2end_integration || ( echo "^^^ +++" && false )
- ./tools/integration/revocation_test.sh
plugins: *scion-run-hooks
Expand All @@ -167,6 +169,7 @@ steps:
- ./scion.sh topology -d
- ./scion.sh run
- tools/await-connectivity
- sleep 5
- echo "--- run tests"
- ./bin/end2end_integration -d || ( echo "^^^ +++" && false )
plugins: *scion-run-hooks
Expand Down
37 changes: 21 additions & 16 deletions .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,24 +41,29 @@ linters:
msg: spell trust root certificate as trc / TRC
goheader:
values:
regexp:
copyright-lines: |-
(Copyright 20[0-9][0-9] .*)(
Copyright 20[0-9][0-9] .*)*
template: |-
{{copyright-lines}}

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
const:
LICENSE_TEXT: |-
Licensed under the Apache License, Version 2.0 \(the "License"\);
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
SPDX_LINE: |-
SPDX-License-Identifier: Apache-2.0
regexp:
copyright_lines: |-
(Copyright 20[0-9][0-9] .*)
(Copyright 20[0-9][0-9] .*)*
license: |-
({{SPDX_LINE}})|({{LICENSE_TEXT}})
template: |-
{{copyright_lines}}{{license}}
lll:
line-length: 100
tab-width: 4
Expand Down
56 changes: 14 additions & 42 deletions MODULE.bazel.lock
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this file intentionally in the repository?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. For now that's the least bad solution. It's supposed to be in the repo to ensure the locking, but on the other hand it is updated automatically. It's all silly and nobody at bazel seems to know what the right thing to do is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. For now that's the least bad solution. It's supposed to be in the repo to ensure the locking, but on the other hand it is updated automatically. It's all silly and nobody at bazel seems to know what the right thing to do is.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,13 @@ build-dev:
tar -kxf bazel-bin/scion.tar -C bin
tar -kxf bazel-bin/scion-ci.tar -C bin
tar -kxf bazel-bin/scion-topo.tar -C bin
sudo setcap "cap_bpf=ep cap_net_admin=ep cap_net_raw=ep" bin/router

build:
rm -f bin/*
bazel build //:scion
tar -kxf bazel-bin/scion.tar -C bin
sudo setcap "cap_bpf=ep cap_net_admin=ep cap_net_raw=ep" bin/router

# BFLAGS is optional. It may contain additional command line flags for CI builds. Currently this is:
# "--file_name_version=$(tools/git-version)" to include the git version in the artifacts names.
Expand Down
2 changes: 1 addition & 1 deletion acceptance/common/raw.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ def raw_test(
py_binary(
name = "%s_teardown" % name,
srcs = [src],
args = ["teardown"],
args = ["teardown"] + args,
main = src,
deps = [":%s_lib" % name],
imports = imports,
Expand Down
2 changes: 1 addition & 1 deletion acceptance/common/topogen.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ def topogen_test(

py_test(
name = name,
size = "large",
size = "medium",
srcs = [src],
main = src,
args = args + common_args,
Expand Down
9 changes: 5 additions & 4 deletions acceptance/hidden_paths/test.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

import http.server
import threading
import time

from acceptance.common import base
from acceptance.common import scion
Expand Down Expand Up @@ -110,11 +111,11 @@ def setup_start(self):
server_thread = threading.Thread(target=configuration_server, args=[server])
server_thread.start()
self._server = server

super().setup_start()

self.await_connectivity()
self._server.shutdown() # by now configuration must have been downloaded everywhere
self.await_connectivity() # <- not very reliable
time.sleep(10) # <- ...so
self._server.shutdown() # by now configuration must have been downloaded everywhere

def _run(self):
# Group 3
Expand All @@ -141,7 +142,7 @@ def _showpaths_bidirectional(self, source: str, destination: str):

def _showpaths_run(self, source_as: str, destination_as: str):
print(self.execute_tester(ISD_AS(self._ases[source_as]),
"scion", "sp", self._ases[destination_as], "--timeout", "2s"))
"scion", "sp", self._ases[destination_as], "--timeout", "3s"))


def configuration_server(server):
Expand Down
6 changes: 1 addition & 5 deletions acceptance/router_benchmark/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -48,18 +48,14 @@ py_library(
srcs = ["benchmarklib.py"],
)

# To ensure that the linter runs over this. Cannot actually be run with
# bazel run; it is meant to be executed from the command line.
py_binary(
name = "benchmark",
srcs = ["benchmark.py"],
args = [
"--brload",
"$(location //acceptance/router_benchmark/brload:brload)",
],
data = [
"//acceptance/router_benchmark/brload",
],
data = data,
imports = ["."],
visibility = ["//visibility:public"],
deps = [
Expand Down
Loading
Loading