Refactor server.py to use Asyncio and Non-blocking I/O for improved scalability

[from2001]

Problem Description

Currently, server.py utilizes the threading module combined with blocking ZeroMQ calls. While this works for a moderate number of clients, it faces scalability challenges due to Python's GIL (Global Interpreter Lock) and the overhead of OS-level context switching.

In high-load LBE (Location-Based Entertainment) scenarios—such as 100+ clients sending transforms at 30Hz—this architecture can lead to:

• Increased CPU usage due to thread contention.
• Latency (lag) caused by lock competition (_rooms_lock).
• Difficulty in integrating with modern async-native libraries (e.g., the FastAPI REST bridge currently runs in a separate thread).

Proposed Solution

Migrate the server architecture to an Asyncio-based event loop model using zmq.asyncio (native async support in pyzmq).

By moving to a single-threaded, non-blocking I/O model, we can handle thousands of concurrent connections efficiently, similar to Node.js or Nginx architectures.

Implementation Plan

• Replace Threading with Asyncio: Convert NetSyncServer to run on an asyncio event loop instead of spawning receive_thread and periodic_thread.
• Adopt Non-blocking ZMQ: Switch from standard zmq.Context to zmq.asyncio.Context and use await for socket operations.
• Refactor Main Loops:
  • Convert _receive_loop to an async def coroutine.
  • Convert _periodic_loop (cleanup/broadcasting) to an async def coroutine using asyncio.sleep().
• Simplify Locking: Reduce or eliminate _rooms_lock usage, as the single-threaded event loop naturally prevents many race conditions.
• Integrate REST Bridge: Run the FastAPI application within the same event loop (removing the need for a separate thread for rest_bridge.py).

Code Example (Concept)

Current (Blocking):

def _receive_loop(self):
    while self.running:
        if self.router.poll(100):
            msg = self.router.recv_multipart()
            self.process_message(msg)

Proposed (Async):

async def _receive_loop(self):
    while True:
        # Non-blocking await
        msg = await self.router.recv_multipart()
        # Process message as a task
        asyncio.create_task(self.process_message(msg))

Benefits

• High Scalability: Efficiently handles C10K-level concurrency.
• Reduced Latency: Minimizes overhead from thread switching.
• Code Simplicity: Reduces the complexity of thread synchronization and locking.

[from2001]

Conclusion: No need to implement this immediately

While this Issue makes some theoretically valid points, it appears to be a classic case of premature optimization. Here's my analysis:

Verifying the Issue's Claims
Claim: GIL is a bottleneck
Reality: ZeroMQ (pyzmq) is a C extension that releases the GIL during socket operations. GIL rarely becomes a major bottleneck for I/O-bound servers
────────────────────────────────────────
Claim: Thread context switching is problematic
Reality: Currently only 5-6 threads. Overhead is minimal unless you have thousands of threads
────────────────────────────────────────
Claim: Lock contention causes latency
Reality: No concrete measurement data provided. Unclear if _rooms_lock contention is actually a problem
Reasons Not to Implement

1. No evidence of the problem: The claim "issues occur with 100+ clients" lacks concrete benchmark results or latency measurements
2. Risk of large-scale refactoring: Completely rewriting working code carries a high risk of introducing new bugs
3. ZeroMQ is already highly efficient: ZeroMQ itself is designed to be event-driven, and performs well even with a thread-based architecture
4. Asyncio isn't necessarily superior: Single-threaded asyncio can suffer from blocking when CPU-bound processing is mixed in, and can actually be harder to debug

When to Consider Implementation

This should be considered when the following conditions are met:

1. Concrete benchmarks using client_simulator etc. confirm that the thread model is actually the bottleneck
2. Real operational issues occur with the current architecture (increased latency, 100% CPU usage, etc.)
3. Specific problems arise with REST bridge integration