Fix MT5 thread-safety issues in data_server.py

[wJubran]

Problem

The current MT5 data feed implementation in backend/data_server.py has known thread-safety issues. MT5's Python API is not thread-safe and must be accessed from the same thread it was initialized on. Under concurrent load, this can cause crashes or data inconsistencies.

Current Behavior

• MT5 calls are made from multiple threads without proper synchronization
• Intermittent crashes or disconnections under heavy load
• No dedicated MT5 thread or queue-based communication pattern

Expected Behavior

• MT5 interactions should be isolated to a single dedicated thread
• A thread-safe queue or event loop pattern should handle requests/responses
• The fix should not introduce latency in the data feed

Suggested Approach

• Create a dedicated MT5Thread class that owns the MT5 connection
• Use queue.Queue for inter-thread communication
• Replace direct MT5 calls from async handlers with queue-based requests

Files Involved

• backend/data_server.py
• backend/providers/ (MT5 provider)

Skills Needed

Python, Threading, asyncio, MetaTrader5 API

[NadirAliOfficial]

Hit this exact issue building production MT5 bots. The thread-safety problem is real — MT5's Python library must be initialized and called from the same thread.

Here's the pattern I use in production — a dedicated MT5 thread with a queue:

import threading
import queue
import MetaTrader5 as mt5

class MT5Thread(threading.Thread):
    def __init__(self):
        super().__init__(daemon=True)
        self._queue = queue.Queue()
        self._results = {}
        self._lock = threading.Lock()

    def run(self):
        mt5.initialize()
        while True:
            task_id, fn, args, kwargs = self._queue.get()
            try:
                result = fn(*args, **kwargs)
            except Exception as e:
                result = e
            with self._lock:
                self._results[task_id] = result
            self._queue.task_done()

    def call(self, fn, *args, **kwargs):
        task_id = id(fn) + id(args)
        self._queue.put((task_id, fn, args, kwargs))
        self._queue.join()
        with self._lock:
            return self._results.pop(task_id)

# Usage — all MT5 calls go through the thread
mt5_thread = MT5Thread()
mt5_thread.start()

# safe from any thread
rates = mt5_thread.call(mt5.copy_rates_from_pos, 'EURUSD', mt5.TIMEFRAME_M5, 0, 100)
tick = mt5_thread.call(mt5.symbol_info_tick, 'EURUSD')

Key points:

• mt5.initialize() is called inside the thread's run() — never from outside
• All callers submit tasks via queue.put() and block on queue.join()
• The dedicated thread serializes all MT5 access — no concurrent calls possible
• daemon=True ensures it shuts down cleanly with the main process

This eliminated all intermittent crashes in my setup under heavy concurrent data requests.

[wJubran]

Thanks — this is very helpful, and it lines up closely with the issue we are seeing in the current MT5 integration.

The dedicated MT5 thread + queue pattern makes sense here, especially the point that mt5.initialize() must happen inside the owning thread and that all MT5 access should be serialized through that thread.

If you're interested in contributing this, I’d be happy to review either:

1. a short implementation outline in this issue, or
2. a draft PR focused on the MT5 thread-safety fix.

A few things I’d want to preserve:

• keep the fix scoped mainly to backend/data_server.py and the MT5 provider path
• avoid introducing noticeable latency in the live data flow
• make shutdown / reconnect / error propagation behavior explicit
• keep the design clear enough for future contributors to extend

Thanks again — this is exactly the kind of practical input that is useful for this issue.

[miguelangelo78]

@wJubran I notice you've been dealing with mt5 thread safety, and it's great that @NadirAliOfficial helped out. I would like to help as well. What if you could get all of your data without MT5 at all? It would be a nice change to this project if you could get the ticks directly from the source. Let me know if this interests you, and ping me back.

[wJubran]

@miguelangelo78 Thanks — this is an interesting direction, and I appreciate you offering to help.

Getting tick data directly from the source instead of going through MT5 could be a meaningful architectural improvement, especially if it helps reduce the thread-safety and integration issues around the current MT5 path.

For this specific issue, though, I’d prefer to keep the scope focused on fixing the existing MT5 thread-safety problem in the current implementation.

If you’re interested in the direct-source approach, I’d be very open to reviewing either:

1. a short proposal describing what source/provider path you have in mind, or
2. a separate issue / draft PR scoped specifically to that architecture.

That way, we can keep this issue focused while still exploring your idea properly.

Thanks again — I’d be glad to hear more details.

[miguelangelo78]

@wJubran here is my proposal: #10