关于1.6.1版本docker-compose部署 提交代码卡Judging的问题

[wellwei]

按照官网的docker-compose一键部署命令部署：

git clone -b v1.6.1 https://github.com/QingdaoU/OnlineJudgeDeploy.git

部署后未更改任何设置，提交非CE代码时卡Judging（CE代码正常）
以下是 ~/data/backend/log/dramatiq.log的日志

[2024-10-18 04:06:50] - [DEBUG] - [dramatiq.worker.ConsumerThread(default):326]  - Pushing message '2e93045a-f01f-468b-9056-2e16a154173b' onto work queue.
[2024-10-18 04:06:50] - [DEBUG] - [dramatiq.worker.WorkerThread:479]  - Received message judge_task('f16d911230421eb40a133701b81a6750', 1) with id '2e93045a-f01f-468b-9056-2e16a154173b'.
[2024-10-18 04:06:50] - [ERROR] - [dramatiq.worker.WorkerThread:513]  - Failed to process message judge_task('f16d911230421eb40a133701b81a6750', 1) with unhandled exception.
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/dramatiq/worker.py", line 485, in process_message
    res = actor(*message.args, **message.kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/dramatiq/actor.py", line 182, in __call__
    return self.fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/judge/tasks.py", line 14, in judge_task
    JudgeDispatcher(submission_id, problem_id).judge()
  File "/app/judge/dispatcher.py", line 174, in judge
    self._compute_statistic_info(resp["data"])
  File "/app/judge/dispatcher.py", line 106, in _compute_statistic_info
    self.submission.statistic_info["time_cost"] = max([x["cpu_time"] for x in resp_data])
                                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: max() iterable argument is empty
[2024-10-18 04:06:50] - [WARNING] - [dramatiq.middleware.retries.Retries:103]  - Retries exceeded for message '2e93045a-f01f-468b-9056-2e16a154173b'.
[2024-10-18 04:06:50] - [DEBUG] - [dramatiq.worker.ConsumerThread(default):344]  - Rejecting message '2e93045a-f01f-468b-9056-2e16a154173b'.
[2024-10-18 04:06:50] - [ERROR] - [sentry.errors:686]  - Sentry responded with an error: module 'ssl' has no attribute 'wrap_socket' (url: https://sentry.io/api/263057/store/)
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/raven/transport/threaded.py", line 165, in send_sync
    super(ThreadedHTTPTransport, self).send(url, data, headers)
  File "/usr/local/lib/python3.12/site-packages/raven/transport/http.py", line 38, in send
    response = urlopen(
               ^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/raven/utils/http.py", line 66, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/urllib/request.py", line 515, in open
    response = self._open(req, data)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/urllib/request.py", line 532, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/urllib/request.py", line 492, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/raven/utils/http.py", line 46, in https_open
    return self.do_open(ValidHTTPSConnection, req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/urllib/request.py", line 1344, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/local/lib/python3.12/http/client.py", line 1331, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.12/http/client.py", line 1377, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.12/http/client.py", line 1326, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/lib/python3.12/http/client.py", line 1085, in _send_output
    self.send(msg)
  File "/usr/local/lib/python3.12/http/client.py", line 1029, in send
    self.connect()
  File "/usr/local/lib/python3.12/site-packages/raven/utils/http.py", line 37, in connect
    self.sock = ssl.wrap_socket(
                ^^^^^^^^^^^^^^^
AttributeError: module 'ssl' has no attribute 'wrap_socket'
b"Sentry responded with an error: module 'ssl' has no attribute 'wrap_socket' (url: https://sentry.io/api/263057/store/)"
[2024-10-18 04:06:50] - [ERROR] - [sentry.errors.uncaught:712]  - ["Failed to process message judge_task('f16d911230421eb40a133701b81a6750', 1) with unhandled exception.", '  File "dramatiq/worker.py", line 485, in process_message', '  File "dramatiq/actor.py", line 182, in __call__', '  File "judge/tasks.py", line 14, in judge_task', '  File "judge/dispatcher.py", line 174, in judge', '  File "judge/dispatcher.py", line 106, in _compute_statistic_info']
b'["Failed to process message judge_task(\'f16d911230421eb40a133701b81a6750\', 1) with unhandled exception.", \'  File "dramatiq/worker.py", line 485, in process_message\', \'  File "dramatiq/actor.py", line 182, in __call__\', \'  File "judge/tasks.py", line 14, in judge_task\', \'  File "judge/dispatcher.py", line 174, in judge\', \'  File "judge/dispatcher.py", line 106, in _compute_statistic_info\']'

询问AI可能是docker容器Python版本太高不兼容，不知如何解决

[wellwei]

以下日志是judge_server启动时，~/data/judge_server/log/judge_server.log中的报错信息

2024-10-18 12:00:19,853 ERROR HTTPConnectionPool(host='oj-backend', port=8000): Max retries exceeded with url: /api/judge_server_heartbeat/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x79c571698410>: Failed to establish a new connection: [Errno 111] Connection refused'))
Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/app/.venv/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 496, in _make_request
    conn.request(
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connection.py", line 400, in request
    self.endheaders()
  File "/usr/lib/python3.12/http/client.py", line 1326, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.12/http/client.py", line 1085, in _send_output
    self.send(msg)
  File "/usr/lib/python3.12/http/client.py", line 1029, in send
    self.connect()
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connection.py", line 238, in connect
    self.sock = self._new_conn()
                ^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connection.py", line 213, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x79c571698410>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='oj-backend', port=8000): Max retries exceeded with url: /api/judge_server_heartbeat/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x79c571698410>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/service.py", line 17, in _request
    resp = requests.post(self.backend_url, json=data,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='oj-backend', port=8000): Max retries exceeded with url: /api/judge_server_heartbeat/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x79c571698410>: Failed to establish a new connection: [Errno 111] Connection refused'))
2024-10-18 12:00:19,857 ERROR 
Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/app/.venv/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 496, in _make_request
    conn.request(
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connection.py", line 400, in request
    self.endheaders()
  File "/usr/lib/python3.12/http/client.py", line 1326, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.12/http/client.py", line 1085, in _send_output
    self.send(msg)
  File "/usr/lib/python3.12/http/client.py", line 1029, in send
    self.connect()
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connection.py", line 238, in connect
    self.sock = self._new_conn()
                ^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connection.py", line 213, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x79c571698410>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='oj-backend', port=8000): Max retries exceeded with url: /api/judge_server_heartbeat/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x79c571698410>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/service.py", line 17, in _request
    resp = requests.post(self.backend_url, json=data,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='oj-backend', port=8000): Max retries exceeded with url: /api/judge_server_heartbeat/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x79c571698410>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/service.py", line 42, in <module>
    service.heartbeat()
  File "/app/service.py", line 35, in heartbeat
    self._request(data)
  File "/app/service.py", line 22, in _request
    raise JudgeServiceError("Heartbeat request failed")
exception.JudgeServiceError

说未链接到后端，但是管理面板可以看到judge的状态正常

[mikucat0309]

後台的 judge server 狀態是透過檢測最後一次 judge server 往 web backend 送的 healthcheck 時間來判斷，從您提供的 log 來看很像是 judge server 沒法往 web backend 送 health check，應該是網路設定方面的問題

想確認一下您是用 Docker 與 Docker Compose 架設的嗎？有無添加自訂義的 network？
剛剛在 AWS 起了台 EC2 測試，建立一套 judge 的網路環境是會通的

[wellwei]

问题已解决，是因为导入fps格式题目时，测试数据没导入，导致评测结果返回空异常。

[kindle1126]

你是怎麼解決的?我一直卡Pending