Skip to content

Commit

Permalink
Merge pull request #575 from atishamte/feature/FlowiseMultiDatabaseSu…
Browse files Browse the repository at this point in the history 
…pport

Feature/flowise multi database support
  • Loading branch information
@HenryHengZJ
HenryHengZJ authored Jul 19, 2023
2 parents 59a9651 + 7f5acf3 commit 0de108c
Show file tree
Hide file tree
Showing 9 changed files with 149 additions and 35 deletions.
21 changes: 14 additions & 7 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -138,18 +138,25 @@ FLOWISE_PASSWORD=1234
Flowise support different environment variables to configure your instance. You can specify the following variables in the `.env` file inside `packages/server` folder. Read [more](https://docs.flowiseai.com/environment-variables)
| Variable | Description | Type | Default |
| -------------------------- | ---------------------------------------------------------------- | ------------------------------------------------ | ----------------------------------- |
| ---------------- | ---------------------------------------------------------------- | ------------------------------------------------ | ----------------------------------- |
| PORT | The HTTP port Flowise runs on | Number | 3000 |
| FLOWISE_USERNAME | Username to login | String |
| FLOWISE_PASSWORD | Password to login | String |
| DEBUG | Print logs onto terminal/console | Boolean |
| LOG_PATH | Location where log files are stored | String | `your-path/Flowise/packages/server` |
| LOG_LEVEL | Different log levels for loggers to be saved | Enum String: `error`, `info`, `verbose`, `debug` | `info` |
| DATABASE_PATH | Location where database is saved | String | `your-home-dir/.flowise` |
| FLOWISE_USERNAME | Username to login | String | |
| FLOWISE_PASSWORD | Password to login | String | |
| DEBUG | Print logs from components | Boolean | |
| LOG_PATH | Location where log files are stored | String | `your-path/Flowise/logs` |
| LOG_LEVEL | Different levels of logs | Enum String: `error`, `info`, `verbose`, `debug` | `info` |
| APIKEY_PATH | Location where api keys are saved | String | `your-path/Flowise/packages/server` |
| EXECUTION_MODE | Whether predictions run in their own process or the main process | Enum String: `child`, `main` | `main` |
| TOOL_FUNCTION_BUILTIN_DEP | NodeJS built-in modules to be used for Tool Function | String | |
| TOOL_FUNCTION_EXTERNAL_DEP | External modules to be used for Tool Function | String | |
| OVERRIDE_DATABASE | Override current database with default | Enum String: `true`, `false` | `true` |
| DATABASE_TYPE | Type of database to store the flowise data | Enum String: `sqlite`, `mysql`, `postgres` | `sqlite` |
| DATABASE_PATH | Location where database is saved (When DATABASE_TYPE is sqlite) | String | `your-home-dir/.flowise` |
| DATABASE_HOST | Host URL or IP address (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_PORT | Database port (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_USERNAME | Database username (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_PASSWORD | Database password (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_NAME | Database name (When DATABASE_TYPE is not sqlite) | String | |
You can also specify the env variables when using `npx`. For example:
Expand Down
15 changes: 14 additions & 1 deletion packages/server/.env.example
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,23 @@ PORT=3000
# FLOWISE_USERNAME=user
# FLOWISE_PASSWORD=1234
# DEBUG=true
# DATABASE_PATH=/your_database_path/.flowise
# APIKEY_PATH=/your_api_key_path/.flowise
# LOG_PATH=/your_log_path/.flowise/logs
# LOG_LEVEL=debug (error | warn | info | verbose | debug)
# EXECUTION_MODE=main (child | main)

OVERRIDE_DATABASE="true"
DATABASE_TYPE="sqlite" # sqlite, mysql, postgres

# When database is sqlite
# DATABASE_PATH=/your_database_path/.flowise

# When database is not sqlite
# DATABASE_PORT=""
# DATABASE_HOST=""
# DATABASE_NAME="flowise"
# DATABASE_USER=""
# DATABASE_PASSWORD=""

# TOOL_FUNCTION_BUILTIN_DEP=crypto,fs
# TOOL_FUNCTION_EXTERNAL_DEP=moment,lodash
21 changes: 14 additions & 7 deletions packages/server/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,18 +34,25 @@ FLOWISE_PASSWORD=1234
Flowise support different environment variables to configure your instance. You can specify the following variables in the `.env` file inside `packages/server` folder. Read [more](https://docs.flowiseai.com/environment-variables)
| Variable | Description | Type | Default |
| -------------------------- | ---------------------------------------------------------------- | ------------------------------------------------ | ----------------------------------- |
| ---------------- | ---------------------------------------------------------------- | ------------------------------------------------ | ----------------------------------- |
| PORT | The HTTP port Flowise runs on | Number | 3000 |
| FLOWISE_USERNAME | Username to login | String |
| FLOWISE_PASSWORD | Password to login | String |
| DEBUG | Print logs onto terminal/console | Boolean |
| LOG_PATH | Location where log files are stored | String | `your-path/Flowise/packages/server` |
| LOG_LEVEL | Different log levels for loggers to be saved | Enum String: `error`, `info`, `verbose`, `debug` | `info` |
| DATABASE_PATH | Location where database is saved | String | `your-home-dir/.flowise` |
| FLOWISE_USERNAME | Username to login | String | |
| FLOWISE_PASSWORD | Password to login | String | |
| DEBUG | Print logs from components | Boolean | |
| LOG_PATH | Location where log files are stored | String | `your-path/Flowise/logs` |
| LOG_LEVEL | Different levels of logs | Enum String: `error`, `info`, `verbose`, `debug` | `info` |
| APIKEY_PATH | Location where api keys are saved | String | `your-path/Flowise/packages/server` |
| EXECUTION_MODE | Whether predictions run in their own process or the main process | Enum String: `child`, `main` | `main` |
| TOOL_FUNCTION_BUILTIN_DEP | NodeJS built-in modules to be used for Tool Function | String | |
| TOOL_FUNCTION_EXTERNAL_DEP | External modules to be used for Tool Function | String | |
| OVERRIDE_DATABASE | Override current database with default | Enum String: `true`, `false` | `true` |
| DATABASE_TYPE | Type of database to store the flowise data | Enum String: `sqlite`, `mysql`, `postgres` | `sqlite` |
| DATABASE_PATH | Location where database is saved (When DATABASE_TYPE is sqlite) | String | `your-home-dir/.flowise` |
| DATABASE_HOST | Host URL or IP address (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_PORT | Database port (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_USERNAME | Database username (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_PASSWORD | Database password (When DATABASE_TYPE is not sqlite) | String | |
| DATABASE_NAME | Database name (When DATABASE_TYPE is not sqlite) | String | |
You can also specify the env variables when using `npx`. For example:
Expand Down
2 changes: 2 additions & 0 deletions packages/server/package.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@
"flowise-ui": "*",
"moment-timezone": "^0.5.34",
"multer": "^1.4.5-lts.1",
"mysql": "^2.18.1",
"pg": "^8.11.1",
"reflect-metadata": "^0.1.13",
"socket.io": "^4.6.1",
"sqlite3": "^5.1.6",
Expand Down
60 changes: 52 additions & 8 deletions packages/server/src/ChildProcess.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -142,14 +142,58 @@ export class ChildProcess {
* @returns {DataSource}
*/
async function initDB() {
const homePath = process.env.DATABASE_PATH ?? path.join(getUserHome(), '.flowise')
const childAppDataSource = new DataSource({
type: 'sqlite',
database: path.resolve(homePath, 'database.sqlite'),
synchronize: true,
entities: [ChatFlow, ChatMessage, Tool],
migrations: []
})
let childAppDataSource
let homePath
switch (process.env.DATABASE_TYPE) {
case 'sqlite':
homePath = process.env.DATABASE_PATH ?? path.join(getUserHome(), '.flowise')
childAppDataSource = new DataSource({
type: 'sqlite',
database: path.resolve(homePath, 'database.sqlite'),
synchronize: process.env.OVERRIDE_DATABASE == 'true',
entities: [ChatFlow, ChatMessage, Tool],
migrations: []
})
break
case 'mysql':
childAppDataSource = new DataSource({
type: 'mysql',
host: process.env.DATABASE_HOST,
port: parseInt(process.env.DATABASE_PORT || '3306'),
username: process.env.DATABASE_USER,
password: process.env.DATABASE_PASSWORD,
database: process.env.DATABASE_NAME,
charset: 'utf8mb4',
synchronize: process.env.OVERRIDE_DATABASE == 'true',
entities: [ChatFlow, ChatMessage, Tool],
migrations: []
})
break
case 'postgres':
childAppDataSource = new DataSource({
type: 'postgres',
host: process.env.DATABASE_HOST,
port: parseInt(process.env.DATABASE_PORT || '5432'),
username: process.env.DATABASE_USER,
password: process.env.DATABASE_PASSWORD,
database: process.env.DATABASE_NAME,
synchronize: process.env.OVERRIDE_DATABASE == 'true',
entities: [ChatFlow, ChatMessage, Tool],
migrations: []
})
break
default:
homePath = process.env.DATABASE_PATH ?? path.join(getUserHome(), '.flowise')
childAppDataSource = new DataSource({
type: 'sqlite',
database: path.resolve(homePath, 'database.sqlite'),
synchronize: process.env.OVERRIDE_DATABASE == 'true',
entities: [ChatFlow, ChatMessage, Tool],
migrations: []
})
break
}

return await childAppDataSource.initialize()
}

Expand Down
59 changes: 50 additions & 9 deletions packages/server/src/DataSource.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,56 @@ import { getUserHome } from './utils'
let appDataSource: DataSource

export const init = async (): Promise<void> => {
const homePath = process.env.DATABASE_PATH ?? path.join(getUserHome(), '.flowise')

appDataSource = new DataSource({
type: 'sqlite',
database: path.resolve(homePath, 'database.sqlite'),
synchronize: true,
entities: [ChatFlow, ChatMessage, Tool],
migrations: []
})
let homePath
switch (process.env.DATABASE_TYPE) {
case 'sqlite':
homePath = process.env.DATABASE_PATH ?? path.join(getUserHome(), '.flowise')
appDataSource = new DataSource({
type: 'sqlite',
database: path.resolve(homePath, 'database.sqlite'),
synchronize: process.env.OVERRIDE_DATABASE == 'true',
entities: [ChatFlow, ChatMessage, Tool],
migrations: []
})
break
case 'mysql':
appDataSource = new DataSource({
type: 'mysql',
host: process.env.DATABASE_HOST,
port: parseInt(process.env.DATABASE_PORT || '3306'),
username: process.env.DATABASE_USER,
password: process.env.DATABASE_PASSWORD,
database: process.env.DATABASE_NAME,
charset: 'utf8mb4',
synchronize: process.env.OVERRIDE_DATABASE == 'true',
entities: [ChatFlow, ChatMessage, Tool],
migrations: []
})
break
case 'postgres':
appDataSource = new DataSource({
type: 'postgres',
host: process.env.DATABASE_HOST,
port: parseInt(process.env.DATABASE_PORT || '5432'),
username: process.env.DATABASE_USER,
password: process.env.DATABASE_PASSWORD,
database: process.env.DATABASE_NAME,
synchronize: process.env.OVERRIDE_DATABASE == 'true',
entities: [ChatFlow, ChatMessage, Tool],
migrations: []
})
break
default:
homePath = process.env.DATABASE_PATH ?? path.join(getUserHome(), '.flowise')
appDataSource = new DataSource({
type: 'sqlite',
database: path.resolve(homePath, 'database.sqlite'),
synchronize: process.env.OVERRIDE_DATABASE == 'true',
entities: [ChatFlow, ChatMessage, Tool],
migrations: []
})
break
}
}

export function getDataSource(): DataSource {
Expand Down
2 changes: 1 addition & 1 deletion packages/server/src/entity/ChatFlow.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ export class ChatFlow implements IChatFlow {
@Column()
name: string

@Column()
@Column({ type: "text" })
flowData: string

@Column({ nullable: true })
Expand Down
2 changes: 1 addition & 1 deletion packages/server/src/entity/ChatMessage.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ export class ChatMessage implements IChatMessage {
@Column()
chatflowid: string

@Column()
@Column({ type: "text" })
content: string

@Column({ nullable: true })
Expand Down
2 changes: 1 addition & 1 deletion packages/server/src/entity/Tool.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ export class Tool implements ITool {
@Column()
name: string

@Column()
@Column({ type: "text" })
description: string

@Column()
Expand Down

0 comments on commit 0de108c

Please sign in to comment.