Skip to content

Conversation

@jiceathome
Copy link
Contributor

@jiceathome jiceathome commented May 5, 2025

the afpudpip underlay is interchangeable with the udpip underlay but uses af_packet sockets instead of udp sockets. The only expected difference is performance.

Summary:

Add new underlay implementation that uses raw sockets:

  • Implements point-to-point and open-ended UDP "connections"
  • Implements IPv4, IPv6, ARP, and NDP

Add support for the router to select between interchangeable underlays.
Move egress packet buffer disposition and metering to underlay (to support
asynchronous transmission after ARP/NDP resolution).

Other things there were required as a consequence or to enable debugging:

  • Added the capabilities cap_bpf, cap_net_admin and cap_net_raw to everything
    that executes the router and have the rpm/deb installers do likewise.
  • Make the latency-sensitive tests a little more lenient because our CI system
    runs on AWS which sacrifices network latency.
  • Enable the setup-run-teardown mode of operations for the router benchmark.
  • Add a debug-run (brief) mode to the router_benchmark.
  • Added a latency mesurement to the router benchmark.
  • Made the traffic generator of the router benchmark faster.
  • Improve the bpf filtering in private/underlay to include IP addresses in the
    filtering keys and to manage both a kernel side and a socket-side filter.
    (The kernel-side lets the kernel have only what the socket-side does not
    take).
  • Added some sysctl when starting supervisor-based configurations to enable
    the sending of raw packets through the loopback interface.
  • Made the braccept runner able to respond to ARP/NDP and to filter redundant
    traffic.

In passing:

  • At long last addressed the docker-compose warning about the deprecated
    version field.
  • Removed a useless ~1second delay after the last ping send in scion ping.
  • Added a v4 only default-like config for end2end tests.
  • Update goheader lint config to allow choice of license text or SPDX line.

Fixes: #4732

@jiceathome
Copy link
Contributor Author

This change is Reviewable

jiceathome and others added 7 commits May 5, 2025 17:20
Added another filter. Needed to ensure that the kernel does not
do any processing on the packets received by the raw socket.
Expensive stuff would otherwise result (e.g. icmp responses).
Also ditched the PacketSource and Packets() channel. It's too buggy even for
us in tests (crashes after closing the afp).

Added draining for afp once filter is set. Else we get stray packets, which
confuses the test. Did the same in the afp undelay.
@jiceathome jiceathome changed the title Checkpointing. router: afpacket udp underlay (WIP) May 9, 2025
jiceathome and others added 20 commits May 9, 2025 18:31
Comments, debug stmt and useless error return.
Now passes the benchmark.
Not quite sure what difference it makes, but it sure isn't an sh_test.
In passing added some facility for short debug runs.
This is a feable attempt at making it at least slightly faster than the router.
We used to require more than 3% packet drop. We are now content with 3% or more.
Had to rearrange the code a tiny bit.
In passing, attempting to add the needed capabilities to the router
in integration tests besides the router benchmark.
In passing. Fixed another missing space in router_multi test script.
In passing, further simplified the process of making debug runs of the router
benchmark.
jiceathome and others added 3 commits July 31, 2025 18:11
Copy link
Contributor Author

@jiceathome jiceathome left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 87 of 100 files reviewed, 1 unresolved discussion (waiting on @jeltevanbommel, @JordiSubira, @marcfrei, @oncilla, and @romshark)


router/dataplane.go line 534 at r5 (raw file):

Previously, jeltevanbommel (jelte) wrote…

Just checked, I did extend the link config with more fields, and then had a struct SendingOptions that was passed in to the underlays. That struct then had a bunch of withXYZ functions to build the options, for which the linkinfo data, amongst others, was used. Something similar could work here, but in any case, as long as there are arbitrary options that can be passed in, it's okay. What about having a composable Options struct, containing an IPOptions, that you create with the link.LocalAddr and link.Remote.Addr, and then later on e.g., an MPLSOptions could be added? Then you're not directly passing in the LinkInfo, but you can still arbitrarily extend it?

I am reluctant to add any kind of complex configuration. I am very concerned that we if try to define something like that is a structured way we will end-up complexifying it for every single new underlay that anyone comes up with. I think this is a compelling case for under-specification. Each underlay can have whatever configuration it wants; as long as it is willing to parse the string. So, I added an "options" field. If that proves insufficient, or inadequate once you are actively trying to add a new underlay, then you have my heartfelt blessing to change it.

Implemented it very minimally, though: one arbitrary options string.
Copy link
Contributor

@tzaeschke tzaeschke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, with the big caveat that I don´t really speak go and I know nothing about AF sockets. In short, the PR as such looks good to me, but I can't really judge technical correctness.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this file intentionally in the repository?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. For now that's the least bad solution. It's supposed to be in the repo to ensure the locking, but on the other hand it is updated automatically. It's all silly and nobody at bazel seems to know what the right thing to do is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. For now that's the least bad solution. It's supposed to be in the repo to ensure the locking, but on the other hand it is updated automatically. It's all silly and nobody at bazel seems to know what the right thing to do is.

metricsBegin := begin.Unix()

numPkt := 0
out:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't there a better way than using goto? On first sight it is unclear to me whether this loop will always terminate...
Is this normal "style" in go? Disclaimer: I am really not very good at reading go, I have written maybe 200 lines go in my life...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not a bare goto; it is a label to dissambiguate a break statement. Two loops are nested and we need to break out of both.

case outcome = <-listenerChan:
if outcome == 0 {
log.Error("Listener never saw a valid packet being forwarded")
log.Error("listener never saw a valid packet being forwarded")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use uppercase "Listener"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for some reason, the dominant go convention is to never write errors as full sentences.


// If we're going to send, we need to make sure we're not receiving our own stuff. The default
// behaviour is less than clear. The loopback doesn't work with veth, but likely does with
// else.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is "else"? Sentence?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

for {
// This will hog a core (as far as the Go scheduler is concerned) for the duration of the
// call as the Go run-time has no idea that this is a blocking write. This is perfectly fine
// for our use case. This can be made non-blocking if that helps sending faster.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds like making it non-blocking requires a code change. Why is the an if-clause below that deals with the non-blocking state? Am I mixing things up?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No you're not mixing things up. The if clause below is only useful if non-blocking is used. Otherwise, it does nothing and costs almost nothing. After experimenting with many, many variations, I found it convenient that I could switch to non-blocking mode by just changing the flag, but as of now, non-blocking brings no improvement.

@@ -0,0 +1,114 @@
// Copyright 2025 SCION Association
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use SPDX header?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -0,0 +1,401 @@
// Copyright 2025 SCION Association
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use SPDX header?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

"github.com/scionproto/scion/router"
)

// ARP cache parameters. The longuish TTL is because I suspect that linux rate limits responses,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"longish

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -0,0 +1,527 @@
// Copyright 2025 SCION Association
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use SPDX header?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -0,0 +1,470 @@
// Copyright 2025 SCION Association
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use SPDX header?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Among which: added lint check for *either* license text or SPDX line in
headers. Use the latter in all new files.
# MAX_CPUS: the total number of cpus that the test will try to harness. The standard for this
# test is 5: 2 for brload and 3 for the router. Any different number invalidates the performance
# index (which will be reported as 0).
MAX_CPUS = 5
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, cpu is the term used by the data source: lscpu. What we are selecting is [v]cpus, not cores.

# _run(). As a result, running setup, run, at teardown separately is not possible for
# this test. May be it would be possible to reconstruct the map without actually setup the
# interfaces, assuming brload isn't being changed in-between.
# _run(). As a result, running setup, run, at teardown separately is difficult.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed. FWIW, it is done. Updated.

metricsBegin := begin.Unix()

numPkt := 0
out:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not a bare goto; it is a label to dissambiguate a break statement. Two loops are nested and we need to break out of both.

case outcome = <-listenerChan:
if outcome == 0 {
log.Error("Listener never saw a valid packet being forwarded")
log.Error("listener never saw a valid packet being forwarded")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for some reason, the dominant go convention is to never write errors as full sentences.


// If we're going to send, we need to make sure we're not receiving our own stuff. The default
// behaviour is less than clear. The loopback doesn't work with veth, but likely does with
// else.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

for {
// This will hog a core (as far as the Go scheduler is concerned) for the duration of the
// call as the Go run-time has no idea that this is a blocking write. This is perfectly fine
// for our use case. This can be made non-blocking if that helps sending faster.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No you're not mixing things up. The if clause below is only useful if non-blocking is used. Otherwise, it does nothing and costs almost nothing. After experimenting with many, many variations, I found it convenient that I could switch to non-blocking mode by just changing the flag, but as of now, non-blocking brings no improvement.

__uint(max_entries, 64);
} k_map_flt SEC(".maps");

// The traffic that goes to the AF_PACKET socket, must not get to the regular kernel networking
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


// The traffic that goes to the AF_PACKET socket, must not get to the regular kernel networking
// stack; else it will expand resources processing it, generating ICMP responses AND sending them!
// That is the purpose of this program.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor Author

@jiceathome jiceathome left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 71 of 101 files reviewed, 25 unresolved discussions (waiting on @jeltevanbommel, @JordiSubira, @marcfrei, @oncilla, @romshark, and @tzaeschke)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. For now that's the least bad solution. It's supposed to be in the repo to ensure the locking, but on the other hand it is updated automatically. It's all silly and nobody at bazel seems to know what the right thing to do is.

// BufHead returns a slice of bytes of the requested size borrowed from the head of the packet.
// buffer. This space can be used safely by an underlay to store data on ingest and retrieve on
// egress; should the same underlay perform both operations. The data is protected against
// overwrites provided that n is included in the underlay's headroom requirements.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

dstIsLocal := (p.scionLayer.DstIA == p.d.localIA)
if p.ingressFromLink == 0 {
// Outbound
// In via internal or sibling (may only be outbound)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clarified

// Note: SCMP error messages triggered by the sibling router may use paths that
// don't start with the first hop.
if p.path.IsFirstHop() && !srcIsLocal {
// How did it get here?
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

}
return err
// We do not care if some BFD packets get bounced under high load. If it becomes a problem,
// the solution is do use BFD's demand-mode. To be considered in a future refactoring.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

"github.com/scionproto/scion/router"
)

// ARP cache parameters. The longuish TTL is because I suspect that linux rate limits responses,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -0,0 +1,527 @@
// Copyright 2025 SCION Association
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -0,0 +1,470 @@
// Copyright 2025 SCION Association
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

import "github.com/cilium/ebpf"

func loadPortfilter() (*ebpf.CollectionSpec, error) {
func loadKfilter() (*ebpf.CollectionSpec, error) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried and make the heading comments more explicit.

// traffic that we do want also gets to the kernel networking stack. Another
// filter (kfilter.c) has to drop it.
//
// This is far from ideal because I have yet to find a way to dispatch traffic
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well... I think that this is as good as it gets unless we use an XDP socket, which is the focus of a subsequent development. I just wanted to address the possible objection and explain why it is not horrible either.

Copy link
Contributor Author

@jiceathome jiceathome left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 71 of 101 files reviewed, 25 unresolved discussions (waiting on @jeltevanbommel, @JordiSubira, @marcfrei, @oncilla, @romshark, and @tzaeschke)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. For now that's the least bad solution. It's supposed to be in the repo to ensure the locking, but on the other hand it is updated automatically. It's all silly and nobody at bazel seems to know what the right thing to do is.

// BufHead returns a slice of bytes of the requested size borrowed from the head of the packet.
// buffer. This space can be used safely by an underlay to store data on ingest and retrieve on
// egress; should the same underlay perform both operations. The data is protected against
// overwrites provided that n is included in the underlay's headroom requirements.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

dstIsLocal := (p.scionLayer.DstIA == p.d.localIA)
if p.ingressFromLink == 0 {
// Outbound
// In via internal or sibling (may only be outbound)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clarified

// Note: SCMP error messages triggered by the sibling router may use paths that
// don't start with the first hop.
if p.path.IsFirstHop() && !srcIsLocal {
// How did it get here?
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

}
return err
// We do not care if some BFD packets get bounced under high load. If it becomes a problem,
// the solution is do use BFD's demand-mode. To be considered in a future refactoring.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

"github.com/scionproto/scion/router"
)

// ARP cache parameters. The longuish TTL is because I suspect that linux rate limits responses,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -0,0 +1,527 @@
// Copyright 2025 SCION Association
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -0,0 +1,470 @@
// Copyright 2025 SCION Association
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

import "github.com/cilium/ebpf"

func loadPortfilter() (*ebpf.CollectionSpec, error) {
func loadKfilter() (*ebpf.CollectionSpec, error) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried and make the heading comments more explicit.

// traffic that we do want also gets to the kernel networking stack. Another
// filter (kfilter.c) has to drop it.
//
// This is far from ideal because I have yet to find a way to dispatch traffic
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well... I think that this is as good as it gets unless we use an XDP socket, which is the focus of a subsequent development. I just wanted to address the possible objection and explain why it is not horrible either.

@katyatitkova katyatitkova self-requested a review November 4, 2025 12:11
@katyatitkova
Copy link
Contributor

@tzaeschke @marcfrei @oncilla @JordiSubira @jeltevanbommel Any objections against merging?

@tzaeschke
Copy link
Contributor

No objections, but I lack the technical understanding for this PR.

@marcfrei
Copy link
Contributor

marcfrei commented Nov 4, 2025

One of the goals of this PR was to "provide better (to some extent) performance for the udp/ip underlay with relatively low effort". Unfortunately, this turned out to require more effort than initially anticipated with a lot of changes (100 files, +4460 −616 lines). And as far as I know, we don't have benchmarks that show substantially better performance than the UDP/IP underlay. Given the scope of the changes, the question then is, whether we really want to take on the additional maintenance burden going forward. I'm hesitant, what do you think?

@katyatitkova
Copy link
Contributor

And as far as I know, we don't have benchmarks that show substantially better performance than the UDP/IP underlay.

I wasn't deeply involved into benchmarking, but I hope I remember correctly. There were benchmarks that JC and Roman ran on our existing benchmarking machine and on AWS (I think with this code as well, not the master version - but I'm not sure), and the conclusion was that we're hitting the kernel's single-threaded limits on packets per second. So the way to improve it is to use kernel bypass techniques. Our existing setup is suboptimal for benchmarking, and we already have new hardware, just didn't have time to fully assemble it yet. Roman will continue with it when he's back from vacation. So let's see what the benchmarks will show then...

@jiceathome
Copy link
Contributor Author

jiceathome commented Nov 4, 2025 via email

@marcfrei
Copy link
Contributor

marcfrei commented Nov 5, 2025

Thanks for the clarification, J-C! Having this PR as a stepping stone towards an XDP underlay is perfectly fine with me if the delta is indeed only about 10% and Roman is able to leverage all this work. In the ideal case we would then end up with just the existing UDP/IP underlay and a second XDP-based underlay that replaces af_packet, derived from this PR?

@jiceathome
Copy link
Contributor Author

jiceathome commented Nov 5, 2025 via email

Comment on lines +151 to +154
"protocol": "<udpip|other_underlay_protocol>"
"local": "<ip|hostname>:<port>", # or just ":<port>"
"remote": "<ip|hostname:port>",
"remote": "<ip|hostname>:<port>",
"options": "<options>", # optional, defined by protocol
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would specify the inter-AS underlay protocol for the given interface/link, right? How is the intra-AS underlay protocol specified?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change is also orthogonal to the af_packet implementation for the "udip" underlay. So maybe split it off into a separate PR?


.. object:: preferred_underlays

.. option:: udpip = <string>, default = "afpacket"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep "inet" as the default for now.

Comment on lines +240 to +241
In the absence of ``preferred_underlays``, "afpacket" is preferred; falling back to
"inet".
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above.

cfg.BFD.RequiredMinRxInterval = util.DurWrap{Duration: 200 * time.Millisecond}
}
if cfg.PreferredUnderlays == nil {
cfg.PreferredUnderlays = map[string]string{"udpip": "afpacket"}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"inet", see above.

// overwrites provided that the value of n is counted in the underlay's headroom requirements.
//
// n is the size of the slice to be borrowed from the head of the packet buffer.
func (p *Packet) BuffHead(n int) []byte {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'BufHead' in the doc string vs. 'BuffHead' in the method definition. I'd prefer 'HeadBytes'.

}
// if port is outside the configured port range we send to the fixed port.
if port < l.dispatchStart && port > l.dispatchEnd {
if port < l.dispatchStart || port > l.dispatchEnd {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😅

Resolve(p *Packet, dst addr.Host, port uint16) error
// Send queues the packet for sending over this link; discarding if the queue is full.
Send(p *Packet) bool
Send(p *Packet)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was looking at the PacketPool management and it's quite subtle. On start, each underlay gets its own copy of a PacketPool struct, but all copies share the same underlying channel, making the pool actually shared. I think it would be more explicit and easier to understand if we passed a pointer to a PacketPool struct to the Underlay.Start method.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Separate PR?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep the functions in this file package private to reduce the overall API surface.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

router: add a second underlay implementation that uses the AF_PACKET API

5 participants