How to make a FreeBSD vrf

[paigeadelethompson]

[paigeadelethompson]

This actually is a pretty terrible example considering I don’t use a loopback at all, rather to get from lan -> core I could use loopback just fine, but to get from lan -> core -> wan it needs something besides 127.0.0.1 as a src net.. maybe there is a better way than what I’m doing (I just use epair links between VRFs and the core VRF and it essentially lets me do a cute thing that vaguely resembles route leaking and basic policy / restriction using reject/blackhole which I really like, I love this. I want to do just slightly more than this, and setup VXLAN/EVPN with another server and get away from using VLANs .

I didn’t have a lot of time to mess around with lo to see if I could make it work for me instead of using epairs, but it’d be nice for sure but not a problem really.

[paigeadelethompson]

frr/lib/vrf.c

Line 45 in 2b5e48c

static enum vrf_backend_type vrf_backend = VRF_BACKEND_VRF_LITE;

frr/lib/vrf.h

Line 92 in 2b5e48c

enum vrf_backend_type {

[donaldsharp]

So from a FRR perspective someone needs to spend some time fixing up the FRR code to recognize this concept as a vrf under FreeBSD. Honestly I don't think it would be a bunch of work but it would be something. No-one has spent any time implementing this.

[paigeadelethompson]

I was actually thinking about taking a stab at it myself. I've taken a look through the FRR code a little and it doesn't look too terrible. If the change is welcome I'll certainly prioritize it then send you a PR :)

[paigeadelethompson]

Summary of Work that needs to be done

I'm planning to write this myself (I don't really trust the LLM to do a very good job at least there won't be anything in the patch that I haven't personally curated or written from scratch. This proposal is based on what work Gemini thinks needs to be done, and give or take my own understanding is approximately the same. My own understanding is also largely based on what I already know about FreeBSD's networking stack. I'm going to check all of these first but just in case I'm posting this here so that anybody interested in this can be looped in and provide feedback ahead of time.

This document outlines the roadmap for adapting FRR's VRF implementation to support FreeBSD FIBs (Routing Tables).

Technical Constraints & Standards

• Platform Gating: All FreeBSD-specific system calls (like SO_SETFIB and SIOCGIFFIB) and logic must be wrapped in #ifdef __FreeBSD__ blocks to prevent regressions or build failures on other BSDs or Linux.
• Feature Detection: Where possible, use existing FRR macros for BSD feature detection to maintain compatibility with different FreeBSD versions.

1. Zebra VRF & Interface Integration

• Interface-based FIB Discovery: Implement logic to retrieve the FIB ID of an interface upon discovery.
  • Strategy: Use ioctl() with SIOCGIFFIB during interface enumeration or update events.
  • Files: zebra/if_sysctl.c, zebra/if_ioctl.c
• Dynamic Table Mapping: Automatically associate a VRF with the FIB ID derived from its member interfaces using existing table-id fields. Update zebra_vrf_lookup_by_table() and ensure the mapping is dynamic as interfaces are added/removed from the VRF.
  • Files: zebra/zebra_vrf.c

2. Kernel Routing Socket (Writes)

• Multi-FIB Writes: Update rtm_write() to support multiple FIBs. On FreeBSD, use setsockopt(fd, IPPROTO_IP, SO_SETFIB, ...) on the routing socket before calling write().

  • Files: zebra/kernel_socket.c, zebra/kernel_socket.h

• Dataplane Integration: Ensure the kernel_route_update retrieves the correct table_id from the zebra_dplane_ctx and passes it to the socket logic.

  • Files: zebra/rt_socket.c

3. Kernel Message Reading (Reads)

• FIB Identification: When reading messages in rtm_read(), identify the FIB ID (e.g., via rtm_fib or socket options) to map it back to the correct VRF. FreeBSD uses a mib data structure together with sysctl to accomplish this, eg:

        mib[0] = CTL_NET;
        mib[1] = PF_ROUTE;
        mib[2] = 0;             /* protocol */
        mib[3] = AF_UNSPEC;
        mib[4] = NET_RT_DUMP;
        mib[5] = 0;             /* no flags */
        mib[6] = fib;
        if (sysctl(mib, nitems(mib), NULL, &needed, NULL, 0) < 0)

• https://github.com/Ravenhammer-Research/Stelleri/blob/612782898538e16019403657d12dea5a1f3741a0/src/system/freebsd/SystemRoutes.cpp#L34 also /usr/src/sbin/route/route.c for examples
  • Files: zebra/kernel_socket.c

4. Socket Options & VRF Bind

• vrf_bind(): Update for FreeBSD. Replace or supplement SO_BINDTODEVICE logic with SO_SETFIB for standard sockets created within a VRF context. FreeBSD uses SIOCSIFFIB to accomplish this: https://github.com/Ravenhammer-Research/Stelleri/blob/612782898538e16019403657d12dea5a1f3741a0/src/system/freebsd/SystemInterfaces.cpp#L591C21-L591C31
  • Files: lib/vrf.c

[paigeadelethompson]

There is one piece of this which stands out; the yang model which Gemini thought to suggest changing because it was unaware of the table-id leaf and wanted to propose adding it--pretty sure that's not necessary, but none of the FIB operations will work if net.fibs option is == 1, and so any specification for a VRF with a table ID of depends on net.fibs also being at least and the max might be 65536 but I'll need to confirm that.

[paigeadelethompson]

Another thing that might be interesting for another conversation is netns; it’s perhaps possible to accomplish this FreeBSD VNet jails

[paigeadelethompson]

In if_sysctl.c #if !defined(GNU_LINUX) && !defined(OPEN_BSD) I assume FreeBSD would use this branch but does anything else? In any case I’ll add a nested gate for now and if anybody wants that done another way I’ll change it later