userland DDP implementation

[rdmark]

Userland DDP implementation options

The following is an edited analyzis by GPT-5.5 after ingesting Netatalk code, docs, and the TailTalk DDP implementation.

Netatalk implements the AppleTalk protocols above the Datagram Delivery Protocol (DDP), but currently expects the host operating system kernel to provide DDP sockets, interface addressing, and packet routing.

Kernel AppleTalk support is no longer broadly available, so this issue evaluates three approaches for adding a userland DDP implementation.

Netatalk already contains a historical BSD kernel implementation under sys/netatalk/. It includes DDP socket control blocks, input and output processing, routing, EtherTalk Phase I and II handling, AARP, and interface address management. The Rust implementation in TailTalk provides a second, working userland proof of concept.

Current integration constraints

Netatalk does not currently have a complete DDP abstraction. netddp_open() creates a native AF_APPLETALK datagram socket, while netddp_sendto(), netddp_recvfrom(), and netddp_close() are thin wrappers around normal socket operations.

The rest of Netatalk consequently expects:

• synchronous sendto() and recvfrom() semantics;
• a real descriptor usable with select() and poll();
• dynamic and fixed DDP socket binding;
• the DDP protocol type as the first byte of the socket payload;
• kernel-compatible broadcast and wildcard address behavior;
• per-interface AppleTalk addresses, including late-bound addressing;
• routes learned by atalkd to affect DDP forwarding.

In addition, atalkd directly creates AppleTalk sockets and uses platform-specific interface, multicast, and routing operations. Replacing only netddp_open() is therefore insufficient.

Existing in-tree DDP implementation

The code under sys/netatalk/ already defines much of the behavior that a userland implementation must preserve:

• ddp_usrreq.c implements socket attachment, binding, dynamic port allocation, connection state, address validation, and local socket demultiplexing.
• ddp_input.c implements short and extended DDP input, datagram length checks, checksum verification, local delivery, forwarding, and hop-count handling.
• ddp_output.c implements extended DDP header generation, checksums, route selection, and Phase I encapsulation.
• aarp.c implements AARP probing, resolution, responses, caching, and Phase I/II link behavior.
• at_control.c implements interface addresses, network ranges, address probing, and directly connected routes.

This is kernel code rather than a dormant userland library. It depends on historical BSD facilities including mbuf, ifnet, socketvar, kernel route structures, interrupt priority functions, kernel timers, and sleep/wakeup operations. The Meson build currently installs its headers but does not compile these C sources.

Consequently, the implementation cannot simply be enabled in a userland build. Its protocol logic and data model can, however, be adapted behind modern userland interfaces. It should be treated as the primary reference for Netatalk-compatible semantics rather than designing those semantics from scratch.

Option 1: C wrapper around TailTalk

TailTalk demonstrates working userland EtherTalk, AARP, long and short DDP, and LocalTalk support. Reusing it could provide the shortest path to an initial prototype and an independent implementation against which the in-tree kernel logic can be tested.

However, its current interface differs substantially from Netatalk's expectations:

• TailTalk uses a Tokio actor API rather than synchronous socket operations.
• A TailTalk DDP socket is associated with a protocol type, while Netatalk includes the type byte in every datagram.
• TailTalk owns packet capture, address selection, and link-layer state inside the stack.
• Its socket registry is local to one stack instance.
• It does not implement an RTMP-driven forwarding table.
• It currently treats network zero as LocalTalk rather than implementing the complete DDP wildcard and directly-connected-network semantics expected by Netatalk.
• Its DDP protocol type is restricted to the known AppleTalk types rather than accepting an arbitrary 8-bit DDP type.
• Additional validation and failure handling would be needed for production use, including received checksum verification, strict length validation, queue overflow behavior, and removal of panic paths.

An in-process wrapper would create independent DDP stacks in afpd, papd, utilities, and other processes. Those instances would compete for packet capture, DDP ports, and AppleTalk addresses. A viable TailTalk integration would therefore need a single Rust daemon with a C IPC client. At that point, the design is effectively the separate-daemon approach described below rather than a simple language wrapper.

There is also a licensing issue to resolve. Netatalk is distributed under GPLv2, while TailTalk is GPLv3. Directly linking TailTalk into Netatalk, or translating its implementation into the Netatalk source tree, would require appropriate relicensing or another explicit licensing resolution.

Assessment

This is useful as a rapid prototype, an interoperability target, and a source of packet test vectors. It is not a good direct production integration in its current form. The in-tree kernel implementation is a better source for Netatalk's socket, routing, and interface semantics.

Option 2: Userland DDP implementation as a separate daemon and library

This approach would adapt the in-tree DDP and AARP logic into a portable userland core, then introduce a dedicated daemon, provisionally called ddpd, as the single owner of physical AppleTalk interfaces, AARP state, DDP sockets, and packet forwarding.

afpd / papd / tools / atalkd
             |
       libatalk DDP client
             | Unix-domain IPC
            ddpd
             |
     pcap / AF_PACKET / BPF / DLPI

The client side in libatalk would expose the existing netddp_* operations. Each virtual DDP socket would use a real Unix-domain descriptor so existing select(), poll(), and atp_fileno() users remain functional. The send and receive wrappers would carry the AppleTalk source or destination address alongside each payload.

Advantages

• A clear process and privilege boundary.
• One owner for link access, AARP caches, addresses, and DDP port allocation.
• A protocol core that can be tested independently from the rest of Netatalk.
• Existing in-tree socket, routing, forwarding, and AARP behavior can guide the implementation.
• Potential reuse by applications outside Netatalk.
• A daemon crash does not directly corrupt atalkd's in-process routing structures.

Disadvantages

atalkd currently learns routes through RTMP and installs them into the kernel. Once DDP forwarding moves to userland, ddpd must receive those route changes. A separate daemon therefore needs a control protocol covering at least:

• provisional and final interface addresses;
• EtherTalk phase and network ranges;
• route addition, deletion, replacement, and expiry;
• gateway reachability;
• zone multicast membership;
• interface state changes;
• routing enabled or disabled per interface.

This divides a single logical AppleTalk network state between two daemons and introduces ordering, reconnection, resynchronization, and failure-recovery requirements.

Assessment

This is the strongest choice when process isolation, independent DDP availability, or reuse outside Netatalk is a primary goal. The in-tree implementation substantially reduces protocol design uncertainty, but it does not remove the need to synchronize the routing and forwarding state held by atalkd and ddpd.

Option 3: Userland adaptation of the in-tree DDP implementation as part of atalkd

atalkd already owns the AppleTalk control plane:

• interface discovery and configuration;
• provisional and final address selection;
• RTMP route learning and expiry;
• ZIP network and zone information;
• NBP forwarding;
• routing policy between interfaces.

Putting the userland DDP data plane under the same ownership avoids synchronizing routes and interface state between two daemons. Existing service definitions also start atalkd before PAP, AFP over AppleTalk, MacIP, and related services.

The implementation should not be copied directly into atalkd's existing main.c, nor should the old kernel environment be emulated wholesale. The useful protocol and state-machine logic from sys/netatalk/ should be adapted into a modular internal library with explicit components for:

• DDP parsing, serialization, length validation, checksums, and hop handling;
• EtherTalk Phase I and Phase II framing;
• AARP probing, resolution, cache expiry, and conflict detection;
• DDP socket binding and datagram demultiplexing;
• interface and address state;
• route lookup and packet forwarding;
• platform-specific link backends;
• broker IPC for other Netatalk processes.

The following kernel concepts have straightforward userland replacements:

• mbuf chains become bounded packet buffers or buffer views.
• DDP protocol control blocks become broker-managed virtual socket objects.
• socket receive buffers and sorwakeup() become bounded per-client queues backed by pollable Unix-domain sockets.
• kernel route objects become lookups into the route table already maintained by atalkd.
• ifnet output callbacks become explicit link-backend operations.
• kernel timers and sleep/wakeup operations become event-loop timers and state transitions.

This preserves the established DDP behavior without retaining dependencies on historical kernel internals.

Other daemons and command-line tools would still need virtual DDP sockets. atalkd would therefore expose a Unix-domain socket broker, and libatalk would provide the client backend. A connection representing a DDP socket would remain a pollable descriptor, preserving the existing event-loop model.

The current native kernel backend should remain available. A backend abstraction would allow Netatalk to use either native AF_APPLETALK sockets or the userland broker without changing ATP, ASP, NBP, PAP, and AFP code unnecessarily.

Advantages

• Routing and DDP forwarding share one authoritative route table.
• No new synchronization protocol between routing and forwarding daemons.
• Fits the existing service lifecycle and dependency model.
• Reuses the design and behavior of the existing Netatalk DDP, AARP, and interface code.
• Allows the old upper-layer protocol implementations to remain largely unchanged.
• Can preserve kernel DDP as an alternative backend.

Disadvantages

• Increases atalkd's responsibility and failure impact.
• Requires careful modularization to avoid adding more complexity to legacy daemon code.
• Requires an IPC broker even though the protocol engine is local to atalkd.
• Link access and routing currently require elevated privileges, so privilege separation should be considered during the design.

Assessment

This provides the best ownership model for Netatalk because atalkd already maintains the state required to route DDP correctly. The existing sys/netatalk/ implementation lowers protocol design risk and provides concrete compatibility behavior. The legacy-code risk can be controlled by adapting its logic into a separate internal module with narrow interfaces rather than embedding kernel compatibility code in the existing event loop and routing source files.

Conclusion

The recommended direction is a C userland adaptation of the DDP, AARP, socket, and interface logic already present under sys/netatalk/. It should be implemented as a modular subsystem owned by atalkd, together with a Unix-domain socket broker and a libatalk client backend.

The in-tree implementation should be the primary source for Netatalk-compatible binding, wildcard address, routing, forwarding, checksum, and AARP behavior. It must be adapted rather than compiled unchanged because its execution and memory model is tied to historical BSD kernels.

TailTalk should be used as a working userland proof of concept, an independent interoperability target, and a source of packet-level test vectors. Direct integration should be limited to prototyping unless its API, routing model, production hardening, and licensing are addressed.

A standalone ddpd remains a credible alternative if independent reuse or process isolation is considered more important than keeping the RTMP control plane and DDP forwarding plane together.

Suggested implementation order:

1. Document the observable socket, routing, forwarding, interface, and AARP behavior implemented by sys/netatalk/.
2. Define a backend-neutral DDP socket contract while retaining the native kernel backend.
3. Add regression tests for malformed packets, checksums, broadcasts, wildcard addresses, dynamic ports, route selection, and descriptor readiness.
4. Adapt DDP packet processing, framing, checksums, and AARP into a userland core with offline packet tests.
5. Replace kernel protocol control blocks and receive buffers with virtual socket objects and bounded queues.
6. Add the Unix-domain broker and libatalk client backend.
7. Support a single EtherTalk Phase II interface without routing.
8. Redirect atalkd interface and route operations to the userland core.
9. Add multiple interfaces, forwarding, RTMP route expiry, Phase I, and zone multicast behavior.
10. Add privilege separation, resource limits, restart recovery, fuzzing, and interoperability tests.
11. Differential-test against TailTalk and known-working Linux or NetBSD kernel implementations.

[rdmark]

Follow-up: findings from the modern NetBSD implementation

GPT-5.5 compared the current NetBSD implementation under sys/netatalk/ with Netatalk's historical in-tree copy. The NetBSD code is a maintained descendant of the same implementation imported from Netatalk in 1997, rather than an independent implementation. It nevertheless contains later fixes and useful design changes that should be incorporated into a userland port.

Local unicast and broadcast loopback

Modern NetBSD explicitly redirects locally addressed DDP packets to loopback. It also sends a local copy of an outgoing broadcast while transmitting the original packet on the physical interface.

The broadcast behavior was added to fix AppleTalk name registration. A userland implementation must reproduce it explicitly: packet capture cannot be expected to receive frames sent by the same host, particularly when the capture filter excludes the local source MAC.

Route-cache invalidation

NetBSD retains a route cache per DDP socket but validates it before reuse. Interface or route changes invalidate stale cached routes.

The userland equivalent should associate cached routes with an atalkd route-table generation. Any route or relevant interface change increments the generation, forcing the next send to perform a new lookup.

Interface removal and state cleanup

The modern socket interface includes an explicit interface-purge operation. Removing an interface deletes its AppleTalk addresses and dependent routes rather than leaving cleanup to process shutdown.

The userland core therefore needs a complete interface-removal transition covering:

• directly connected routes;
• cached socket routes;
• AARP entries and unresolved packets;
• multicast membership;
• virtual sockets bound specifically to the removed address.

Compact representation of network ranges

NetBSD decomposes a Phase II network range into the minimum practical set of masked routes instead of creating one route for every network number.

The userland core does not need to reproduce kernel radix routes, but it should store directly connected ranges as ranges or prefixes. Expanding a large range into thousands of individual entries is unnecessary.

Bounded queues and statistics

Modern NetBSD uses bounded Phase I and Phase II ingress queues, bounded socket receive buffers, explicit overflow notification, and 64-bit per-CPU DDP counters.

For a single-threaded atalkd implementation, the useful equivalent is:

• one event-loop owner for DDP, routes, AARP, and socket state;
• bounded ingress, per-client receive, and unresolved-AARP queues;
• explicit overflow behavior rather than unbounded allocation;
• counters for short and extended packets, missing and invalid checksums, malformed lengths, forwarding, unreachable destinations, and socket queue overflow.

Port-range discrepancy

Modern NetBSD defines the valid port range as 1 through 254 and dynamically allocates from 128 through 254. Linux and Netatalk's in-tree header exclude port 254 on EtherTalk, describing it as legal only on LocalTalk. TailTalk currently allocates through 255.

This must be resolved explicitly. For an initial EtherTalk implementation, the conservative behavior is:

• privileged ports: 1-127;
• dynamic ports: 128-253;
• port 254 rejected on EtherTalk;
• port 255 always rejected.

ATADDR_LATENET is not a NetBSD requirement

The modern NetBSD documentation explicitly comments out ATADDR_LATENET. Binding requires ATADDR_ANYNET, which resolves to the primary configured address. Netatalk's current AppleTalk consumers do not reference ATADDR_LATENET directly.

Late-bound multi-interface sockets should therefore not block an initial implementation. They can be specified separately if another supported kernel implementation or application requires them.

Additional input validation is still required

The modern NetBSD code should not be ported without hardening:

• Its DDP input path does not enforce a minimum and maximum declared datagram length. Netatalk's in-tree copy has a newer fix for this.
• Its forwarding hop-limit check is disabled, allowing the four-bit hop field to wrap.
• Its AARP parser notes that hardware and protocol address lengths still need validation.

The userland implementation should require:

• declared DDP length of at least header size plus the type byte;
• declared DDP length no greater than header size plus DDP_MAXSZ;
• captured data at least as large as the declared length;
• trimming only after validating Ethernet padding;
• verification of every nonzero extended-DDP checksum;
• rejection before forwarding when the hop count is already 15;
• exact AARP hardware length, protocol length, opcode, and frame-size validation.

Updated use of reference implementations

The implementation references should be used as follows:

1. Netatalk's sys/netatalk/ for existing project semantics and its newer DDP length validation.
2. Modern NetBSD for local and broadcast loopback, route-cache lifecycle, interface purge, compact range routes, bounded queues, and statistics.
3. TailTalk as the independent userland interoperability implementation and link-I/O proof of concept.

These findings do not change the earlier recommendation. They strengthen the case for a modular userland adaptation under atalkd, while making clear that neither the historical Netatalk code nor the modern NetBSD descendant should be copied without reconciling their behavioral differences and known validation gaps.

[NJRoadfan]

Port 254 is a valid DDP port per Inside AppleTalk. Its likely that whoever wrote the Linux and Netatalk implementation were confusing this with nodes, of which node 254 isn't valid on Phase 2 networks and shouldn't be assigned to a client. DDP port 255 is not valid in all cases.

[NJRoadfan]

A good example of a userland DDP implimentation is the Localtalk-to-Ethernet bridge in the GSport emulator that @pneubauer wrote that utilizes pcap.

See: https://github.com/david-schmidt/gsport/tree/main/src/atbridge

aarp.c, elap.c, and helpers forms the core of what we need.

[pneubauer]

A good example of a userland DDP implimentation is the Localtalk-to-Ethernet bridge in the GSport emulator that @pneubauer wrote that utilizes pcap.

See: https://github.com/david-schmidt/gsport/tree/main/src/atbridge

aarp.c, elap.c, and helpers forms the core of what we need.

I did that GSPort implementation -- it works well enough for file sharing and netboot from a LocalTalk-only emulated Apple IIGS to Netatalk (on Ethertalk). I'm happy to help adapt that implementation.