[Bug]: Fail to startup RAGFlow

[peiqi00001]

Is there an existing issue for the same bug?

• I have checked the existing issues.

RAGFlow workspace code commit ID

1233

RAGFlow image version

5115122

Other environment information

启动时一直陷入以下循环

Actual behavior

  ____   ___    ______ ______ __               
   / __ \ /   |  / ____// ____// /____  _      __
  / /_/ // /| | / / __ / /_   / // __ \| | /| / /
 / _, _// ___ |/ /_/ // __/  / // /_/ /| |/ |/ / 
/_/ |_|/_/  |_|\____//_/    /_/ \____/ |__/|__/                             

2024-12-19 17:02:00,166 INFO 439 RAGFlow version: v0.14.1 full
2024-12-19 17:02:00,166 INFO 439 project base: /ragflow
2024-12-19 17:02:00,166 INFO 439 Current configs, from /ragflow/conf/service_conf.yaml:
ragflow: {'host': '0.0.0.0', 'http_port': 9380}
mysql: {'name': 'rag_flow', 'user': 'root', 'password': 'infini_rag_flow', 'host': 'mysql', 'port': 3306, 'max_connections': 100, 'stale_timeout': 30}
minio: {'user': 'rag_flow', 'password': 'infini_rag_flow', 'host': 'minio:9000'}
es: {'hosts': 'http://es01:9200', 'username': 'elastic', 'password': 'infini_rag_flow'}
infinity: {'uri': 'infinity:23817', 'db_name': 'default_db'}
redis: {'db': 1, 'password': 'infini_rag_flow', 'host': 'redis:6379'}
2024-12-19 17:02:00,166 INFO 439 Use Elasticsearch http://es01:9200 as the doc engine.
2024-12-19 17:02:00,176 INFO 439 GET http://es01:9200/ [status:200 duration:0.008s]
2024-12-19 17:02:00,180 INFO 439 HEAD http://es01:9200/ [status:200 duration:0.003s]
2024-12-19 17:02:00,180 INFO 439 Elasticsearch http://es01:9200 is healthy.
2024-12-19 17:02:00,188 WARNING 439 Load term.freq FAIL!
2024-12-19 17:02:00,194 WARNING 439 Realtime synonym is disabled, since no redis connection.
2024-12-19 17:02:00,201 WARNING 439 Load term.freq FAIL!
2024-12-19 17:02:00,206 WARNING 439 Realtime synonym is disabled, since no redis connection.
Traceback (most recent call last):
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3232, in connect
self._state.set_connection(self._connect())
File "/ragflow/.venv/lib/python3.10/site-packages/playhouse/pool.py", line 76, in inner
return fn(self, *args, **kwargs)
File "/ragflow/.venv/lib/python3.10/site-packages/playhouse/pool.py", line 169, in _connect
conn = super(PooledDatabase, self)._connect()
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 4201, in _connect
conn = mysql.connect(db=self.database, autocommit=True,
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 361, in init
self.connect()
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 669, in connect
self._request_authentication()
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 957, in _request_authentication
auth_packet = self._read_packet()
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 775, in _read_packet
packet.raise_for_error()
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/protocol.py", line 219, in raise_for_error
err.raise_mysql_exception(self._data)
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/err.py", line 150, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.OperationalError: (1524, "Plugin 'mysql_native_password' is not loaded")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/ragflow/api/ragflow_server.py", line 80, in
init_web_db()
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3127, in inner
with ConnectionContext(self.db):
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3122, in enter
self.db.connect()
File "/ragflow/.venv/lib/python3.10/site-packages/playhouse/pool.py", line 121, in connect
return super(PooledDatabase, self).connect(reuse_if_open)
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3231, in connect
with exception_wrapper:
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3059, in exit
reraise(new_type, new_type(exc_value, *exc_args), traceback)
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 192, in reraise
raise value.with_traceback(tb)
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3232, in connect
self._state.set_connection(self._connect())
File "/ragflow/.venv/lib/python3.10/site-packages/playhouse/pool.py", line 76, in inner
return fn(self, *args, **kwargs)
File "/ragflow/.venv/lib/python3.10/site-packages/playhouse/pool.py", line 169, in _connect
conn = super(PooledDatabase, self)._connect()
File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 4201, in _connect
conn = mysql.connect(db=self.database, autocommit=True,
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 361, in init
self.connect()
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 669, in connect
self._request_authentication()
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 957, in _request_authentication
auth_packet = self._read_packet()
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 775, in _read_packet
packet.raise_for_error()
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/protocol.py", line 219, in raise_for_error
err.raise_mysql_exception(self._data)
File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/err.py", line 150, in raise_mysql_exception
raise errorclass(errno, errval)
peewee.OperationalError: (1524, "Plugin 'mysql_native_password' is not loaded")
2024-12-19 17:02:03,651 INFO 578 ragflow_server log path: /ragflow/logs/ragflow_server.log
2024-12-19 17:02:10,858 INFO 578 init database on cluster mode successfully
2024-12-19 17:02:20,773 WARNING 20 RedisDB.queue_info rag_flow_svr_queue got exception: no such key
2024-12-19 17:02:20,774 INFO 20 task_consumer_0 reported heartbeat: {"name": "task_consumer_0", "now": "2024-12-19T17:02:20.773378", "boot_at": "2024-12-19T17:01:50.704858", "pending": 0, "lag": 0, "done": 0, "failed": 0, "current": null}
2024-12-19 17:02:28,724 INFO 578

Expected behavior

No response

Steps to reproduce

____   ___    ______ ______ __               
       / __ \ /   |  / ____// ____// /____  _      __
      / /_/ // /| | / / __ / /_   / // __ \| | /| / /
     / _, _// ___ |/ /_/ // __/  / // /_/ /| |/ |/ / 
    /_/ |_|/_/  |_|\____//_/    /_/ \____/ |__/|__/                             

    
2024-12-19 17:02:00,166 INFO     439 RAGFlow version: v0.14.1 full
2024-12-19 17:02:00,166 INFO     439 project base: /ragflow
2024-12-19 17:02:00,166 INFO     439 Current configs, from /ragflow/conf/service_conf.yaml:
        ragflow: {'host': '0.0.0.0', 'http_port': 9380}
        mysql: {'name': 'rag_flow', 'user': 'root', 'password': 'infini_rag_flow', 'host': 'mysql', 'port': 3306, 'max_connections': 100, 'stale_timeout': 30}
        minio: {'user': 'rag_flow', 'password': 'infini_rag_flow', 'host': 'minio:9000'}
        es: {'hosts': 'http://es01:9200', 'username': 'elastic', 'password': 'infini_rag_flow'}
        infinity: {'uri': 'infinity:23817', 'db_name': 'default_db'}
        redis: {'db': 1, 'password': 'infini_rag_flow', 'host': 'redis:6379'}
2024-12-19 17:02:00,166 INFO     439 Use Elasticsearch http://es01:9200 as the doc engine.
2024-12-19 17:02:00,176 INFO     439 GET http://es01:9200/ [status:200 duration:0.008s]
2024-12-19 17:02:00,180 INFO     439 HEAD http://es01:9200/ [status:200 duration:0.003s]
2024-12-19 17:02:00,180 INFO     439 Elasticsearch http://es01:9200 is healthy.
2024-12-19 17:02:00,188 WARNING  439 Load term.freq FAIL!
2024-12-19 17:02:00,194 WARNING  439 Realtime synonym is disabled, since no redis connection.
2024-12-19 17:02:00,201 WARNING  439 Load term.freq FAIL!
2024-12-19 17:02:00,206 WARNING  439 Realtime synonym is disabled, since no redis connection.
Traceback (most recent call last):
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3232, in connect
    self._state.set_connection(self._connect())
  File "/ragflow/.venv/lib/python3.10/site-packages/playhouse/pool.py", line 76, in inner
    return fn(self, *args, **kwargs)
  File "/ragflow/.venv/lib/python3.10/site-packages/playhouse/pool.py", line 169, in _connect
    conn = super(PooledDatabase, self)._connect()
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 4201, in _connect
    conn = mysql.connect(db=self.database, autocommit=True,
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 361, in __init__
    self.connect()
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 669, in connect
    self._request_authentication()
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 957, in _request_authentication
    auth_packet = self._read_packet()
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 775, in _read_packet
    packet.raise_for_error()
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/protocol.py", line 219, in raise_for_error
    err.raise_mysql_exception(self._data)
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/err.py", line 150, in raise_mysql_exception
    raise errorclass(errno, errval)
pymysql.err.OperationalError: (1524, "Plugin 'mysql_native_password' is not loaded")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/ragflow/api/ragflow_server.py", line 80, in <module>
    init_web_db()
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3127, in inner
    with ConnectionContext(self.db):
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3122, in __enter__
    self.db.connect()
  File "/ragflow/.venv/lib/python3.10/site-packages/playhouse/pool.py", line 121, in connect
    return super(PooledDatabase, self).connect(reuse_if_open)
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3231, in connect
    with __exception_wrapper__:
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3059, in __exit__
    reraise(new_type, new_type(exc_value, *exc_args), traceback)
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 192, in reraise
    raise value.with_traceback(tb)
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 3232, in connect
    self._state.set_connection(self._connect())
  File "/ragflow/.venv/lib/python3.10/site-packages/playhouse/pool.py", line 76, in inner
    return fn(self, *args, **kwargs)
  File "/ragflow/.venv/lib/python3.10/site-packages/playhouse/pool.py", line 169, in _connect
    conn = super(PooledDatabase, self)._connect()
  File "/ragflow/.venv/lib/python3.10/site-packages/peewee.py", line 4201, in _connect
    conn = mysql.connect(db=self.database, autocommit=True,
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 361, in __init__
    self.connect()
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 669, in connect
    self._request_authentication()
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 957, in _request_authentication
    auth_packet = self._read_packet()
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/connections.py", line 775, in _read_packet
    packet.raise_for_error()
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/protocol.py", line 219, in raise_for_error
    err.raise_mysql_exception(self._data)
  File "/ragflow/.venv/lib/python3.10/site-packages/pymysql/err.py", line 150, in raise_mysql_exception
    raise errorclass(errno, errval)
peewee.OperationalError: (1524, "Plugin 'mysql_native_password' is not loaded")
2024-12-19 17:02:03,651 INFO     578 ragflow_server log path: /ragflow/logs/ragflow_server.log
2024-12-19 17:02:10,858 INFO     578 init database on cluster mode successfully
2024-12-19 17:02:20,773 WARNING  20 RedisDB.queue_info rag_flow_svr_queue got exception: no such key
2024-12-19 17:02:20,774 INFO     20 task_consumer_0 reported heartbeat: {"name": "task_consumer_0", "now": "2024-12-19T17:02:20.773378", "boot_at": "2024-12-19T17:01:50.704858", "pending": 0, "lag": 0, "done": 0, "failed": 0, "current": null}
2024-12-19 17:02:28,724 INFO     578

Additional information

No response

[Snify89]

Can you please post your docker-compose file? Your MySQL container seems not to start I think.

[yuzhichang]

Upgrade to v0.15.0 at first.

[peiqi00001]

services:
es01:
container_name: ragflow-es-01
profiles:
- elasticsearch
image: docker.elastic.co/elasticsearch/elasticsearch:8.13.2
volumes:
- esdata01:/usr/share/elasticsearch/data
ports:
- ${ES_PORT}:9200
env_file: .env
environment:
- node.name=es01
- ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
- bootstrap.memory_lock=false
- discovery.type=single-node
- xpack.security.enabled=true
- xpack.security.http.ssl.enabled=false
- xpack.security.transport.ssl.enabled=false
- cluster.routing.allocation.disk.watermark.low=5gb
- cluster.routing.allocation.disk.watermark.high=3gb
- cluster.routing.allocation.disk.watermark.flood_stage=2gb
- TZ=${TIMEZONE}
mem_limit: ${MEM_LIMIT}
ulimits:
memlock:
soft: -1
hard: -1
healthcheck:
test: ["CMD-SHELL", "curl http://localhost:9200"]
interval: 10s
timeout: 10s
retries: 120
networks:
- ragflow
restart: on-failure

infinity:
container_name: ragflow-infinity
profiles:
- infinity
image: infiniflow/infinity:v0.5.0-dev6
volumes:
- infinity_data:/var/infinity
- ./infinity_conf.toml:/infinity_conf.toml
command: ["-f", "/infinity_conf.toml"]
ports:
- ${INFINITY_THRIFT_PORT}:23817
- ${INFINITY_HTTP_PORT}:23820
- ${INFINITY_PSQL_PORT}:5432
env_file: .env
environment:
- TZ=${TIMEZONE}
mem_limit: ${MEM_LIMIT}
ulimits:
nofile:
soft: 500000
hard: 500000
networks:
- ragflow
healthcheck:
test: ["CMD", "curl", "http://localhost:23820/admin/node/current"]
interval: 10s
timeout: 10s
retries: 120
restart: on-failure

mysql:
#--default-authentication-plugin=mysql_native_password
# mysql:5.7 linux/arm64 image is unavailable.
image: mysql:8.4
container_name: ragflow-mysql
env_file: .env
environment:
- MYSQL_ROOT_PASSWORD=${MYSQL_PASSWORD}
- TZ=${TIMEZONE}
command:
--max_connections=1000
--character-set-server=utf8mb4
--collation-server=utf8mb4_unicode_ci
--tls_version="TLSv1.2,TLSv1.3"
--init-file /data/application/init.sql
ports:
- ${MYSQL_PORT}:3306
volumes:
- mysql_data:/var/lib/mysql
- ./init.sql:/data/application/init.sql
networks:
- ragflow
healthcheck:
test: ["CMD", "mysqladmin" ,"ping", "-uroot", "-p${MYSQL_PASSWORD}"]
interval: 10s
timeout: 10s
retries: 3
restart: on-failure

minio:
image: minio/minio:RELEASE.2023-03-20T20-16-18Z
container_name: ragflow-minio
command: server --console-address ":9001" /data
ports:
- ${MINIO_PORT}:9000
- ${MINIO_CONSOLE_PORT}:9001
env_file: .env
environment:
- MINIO_ROOT_USER=${MINIO_USER}
- MINIO_ROOT_PASSWORD=${MINIO_PASSWORD}
- TZ=${TIMEZONE}
volumes:
- minio_data:/data
networks:
- ragflow
restart: on-failure

redis:
image: valkey/valkey:8
container_name: ragflow-redis
command: redis-server --requirepass ${REDIS_PASSWORD} --maxmemory 128mb --maxmemory-policy allkeys-lru
env_file: .env
ports:
- ${REDIS_PORT}:6379
volumes:
- redis_data:/data
networks:
- ragflow
restart: on-failure

volumes:
esdata01:
driver: local
infinity_data:
driver: local
mysql_data:
driver: local
minio_data:
driver: local
redis_data:
driver: local

networks:
ragflow:
driver: bridge

[KevinHuSh]

Is it on MAC?
You could alter the compose file and switch it to mariaDB. And set MACOS=1 in docker/.env.

[peiqi00001]

是在 MAC 上吗？ 你可以修改 Compose 文件并将其切换到 mariaDB。然后MACOS=1设置docker/.env。
linux 服务器

[peiqi00001]

2024-12-20 15:16:50+08:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.39-1.el9 started.
2024-12-20 15:16:50+08:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2024-12-20 15:16:50+08:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 8.0.39-1.el9 started.
'/var/lib/mysql/mysql.sock' -> '/var/run/mysqld/mysqld.sock'
2024-12-20T07:16:50.997744Z 0 [Warning] [MY-011068] [Server] The syntax '--skip-host-cache' is deprecated and will be removed in a future release. Please use SET GLOBAL host_cache_size=0 instead.
2024-12-20T07:16:50.999829Z 0 [Warning] [MY-010918] [Server] 'default_authentication_plugin' is deprecated and will be removed in a future release. Please use authentication_policy instead.
2024-12-20T07:16:50.999858Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.39) starting as process 1
2024-12-20T07:16:51.007146Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2024-12-20T07:16:51.241936Z 1 [ERROR] [MY-014061] [InnoDB] Invalid MySQL server downgrade: Cannot downgrade from 80403 to 80039. Downgrade is only permitted between patch releases.
mysqld: Can't open file: 'mysql.ibd' (errno: 0 - )
2024-12-20T07:16:51.598283Z 1 [ERROR] [MY-010334] [Server] Failed to initialize DD Storage Engine
2024-12-20T07:16:51.598468Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
2024-12-20T07:16:51.598490Z 0 [ERROR] [MY-010119] [Server] Aborting
2024-12-20T07:16:51.599047Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.39) MySQL Community Server - GPL.

[peiqi00001]

这是什么原因

[KevinHuSh]

Is it out of free space on disk?

[peiqi00001]

/dev/mapper/cl-AIDATA 2.0T 32G 2.0T 2% /AIDATA

[peiqi00001]

很足

[peiqi00001]

The problem has been solved.
The reason is that ragflow-server startup error
Solution:
The early versions of RAGFlow used MySQL 5.7, but the recent versions have started using 8.0. Therefore, if you are upgrading from an earlier version, you may encounter MySQL compatibility issues. There are two solutions to address this problem:

Look for where the MySQL data is stored and delete all of it. This way, there will be no compatibility issues as everything will be new. However, all the data previously imported into RAGFlow, created users, etc., will be lost.

Investigate how to upgrade the MySQL data from version 5.7 to 8.0.