Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat/enhance the multi-modal support #8818

Merged
merged 37 commits into from
Oct 21, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
0eb53a2
feat(api): Enhance multi modal support.
laipz8200 Sep 1, 2024
b39ede9
feat(podcast_generator): add new podcast generation tools
laipz8200 Oct 12, 2024
bfdbbe3
fix(workflow): handle special values for process data consistently
laipz8200 Oct 14, 2024
1bad1da
refactor(prompt): improve handling of variable templates in advanced …
laipz8200 Oct 14, 2024
9305cb9
refactor(core): simplify role handling and improve usability
laipz8200 Oct 15, 2024
a6513c6
fix(memory): filter non-image file types in prompt message content
laipz8200 Oct 15, 2024
fb64397
refactor(core): improve type annotations and file handling consistency
laipz8200 Oct 15, 2024
5da907a
refactor(http_executor): improve HTTP request logging format
laipz8200 Oct 16, 2024
c9f4622
refactor(workflow): improve handling of outputs and inputs
laipz8200 Oct 16, 2024
bca8163
refactor(workflow): rename and restructure list filter components
laipz8200 Oct 16, 2024
849602f
refactor(workflow/nodes): unify filtering and ordering under a single…
laipz8200 Oct 16, 2024
7fdb1d7
refactor(api): enhance file streaming response handling
laipz8200 Oct 16, 2024
d6ca3b6
fix(file-serving): adjust content disposition header for images
laipz8200 Oct 16, 2024
587751b
refactor(tool_node): simplify tool_file_id assignment
laipz8200 Oct 16, 2024
6dbbad7
feat(podcast_audio_generator): improve audio generation workflow
laipz8200 Oct 17, 2024
7f156d9
fix(podcast_audio_generator): remove explicit format specification fo…
laipz8200 Oct 17, 2024
6854491
fix(podcast_audio_generator): specify audio format for file loading
laipz8200 Oct 17, 2024
740d2c8
fix(audio-generation): simplify audio response handling
laipz8200 Oct 17, 2024
849a2ac
feat(file-download): add inline or attachment option for file previews
laipz8200 Oct 17, 2024
900f5f4
fix(tool_files): correct typo in default argument for as_attachment
laipz8200 Oct 17, 2024
e04fcee
fix(tool_files): adjust argument order for as_attachment in parser
laipz8200 Oct 17, 2024
0c2c4c3
refactor(api): update file preview handling and support for audio files
laipz8200 Oct 18, 2024
df1d607
chore(dependencies): update openai package to version 1.52.0
laipz8200 Oct 18, 2024
05bcb6c
chore(dependencies): update openai version constraint
laipz8200 Oct 18, 2024
6ff72e6
chore(dependencies): update pydub version constraint and regenerate l…
laipz8200 Oct 18, 2024
1c1ff05
refactor(tests): improve unit test setup for prompt and document extr…
laipz8200 Oct 20, 2024
33d7eb9
refactor(constants): adjust document extension handling
laipz8200 Oct 20, 2024
69c3267
refactor(document_extractor): update module structure
laipz8200 Oct 20, 2024
8e49ad5
refactor(http_request): streamline HTTP request handling
laipz8200 Oct 20, 2024
2c799a2
refactor(http_request_node): reorganize imports for better readability
laipz8200 Oct 20, 2024
f1d2215
refactor(workflow): reorganize BaseNode import paths
laipz8200 Oct 20, 2024
40b97b0
refactor(workflow): consolidate BaseNodeData and related entities
laipz8200 Oct 20, 2024
d707c5e
delete(api): remove unused BaseIterationState entity
laipz8200 Oct 20, 2024
5cbc375
refactor(workflow): consolidate LLM-related imports and update events
laipz8200 Oct 20, 2024
650581e
refactor(nodes): restructure imports and organization
laipz8200 Oct 20, 2024
1233a5e
refactor(workflow): simplify node type mapping and initialization
laipz8200 Oct 20, 2024
fe47ecd
refactor(models): relocate and update enums usage
laipz8200 Oct 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
refactor(models): relocate and update enums usage
- Moved enums to a dedicated 'models.enums' module for better organization.
- Updated all import statements across the project to use the new location.
- Removed unnecessary docstring placeholders from enum classes.
  • Loading branch information
laipz8200 committed Oct 21, 2024
commit fe47ecdd373c29e4f5ea7af4ca1ddde4914faa58
2 changes: 1 addition & 1 deletion api/controllers/console/app/workflow_statistic.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@
from controllers.console.app.wraps import get_app_model
from controllers.console.setup import setup_required
from controllers.console.wraps import account_initialization_required
from enums import WorkflowRunTriggeredFrom
from extensions.ext_database import db
from libs.helper import DatetimeString
from libs.login import login_required
from models.enums import WorkflowRunTriggeredFrom
from models.model import AppMode


Expand Down
2 changes: 1 addition & 1 deletion api/core/app/apps/advanced_chat/app_generator.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,10 @@
from core.app.entities.task_entities import ChatbotAppBlockingResponse, ChatbotAppStreamResponse
from core.model_runtime.errors.invoke import InvokeAuthorizationError, InvokeError
from core.ops.ops_trace_manager import TraceQueueManager
from enums import CreatedByRole
from extensions.ext_database import db
from factories import file_factory
from models.account import Account
from models.enums import CreatedByRole
from models.model import App, Conversation, EndUser, Message
from models.workflow import Workflow

Expand Down
2 changes: 1 addition & 1 deletion api/core/app/apps/advanced_chat/app_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@
from core.workflow.entities.variable_pool import VariablePool
from core.workflow.enums import SystemVariableKey
from core.workflow.workflow_entry import WorkflowEntry
from enums import UserFrom
from extensions.ext_database import db
from models.enums import UserFrom
from models.model import App, Conversation, EndUser, Message
from models.workflow import ConversationVariable, WorkflowType

Expand Down
2 changes: 1 addition & 1 deletion api/core/app/apps/advanced_chat/generate_task_pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,11 @@
from core.workflow.enums import SystemVariableKey
from core.workflow.graph_engine.entities.graph_runtime_state import GraphRuntimeState
from core.workflow.nodes import NodeType
from enums import CreatedByRole
from events.message_event import message_was_created
from extensions.ext_database import db
from models import Conversation, EndUser, Message, MessageFile
from models.account import Account
from models.enums import CreatedByRole
from models.workflow import (
Workflow,
WorkflowNodeExecution,
Expand Down
2 changes: 1 addition & 1 deletion api/core/app/apps/agent_chat/app_generator.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,10 @@
from core.app.entities.app_invoke_entities import AgentChatAppGenerateEntity, InvokeFrom
from core.model_runtime.errors.invoke import InvokeAuthorizationError, InvokeError
from core.ops.ops_trace_manager import TraceQueueManager
from enums import CreatedByRole
from extensions.ext_database import db
from factories import file_factory
from models import Account, App, EndUser
from models.enums import CreatedByRole

logger = logging.getLogger(__name__)

Expand Down
2 changes: 1 addition & 1 deletion api/core/app/apps/base_app_generator.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

if TYPE_CHECKING:
from core.app.app_config.entities import AppConfig, VariableEntity
from enums import CreatedByRole
from models.enums import CreatedByRole


class BaseAppGenerator:
Expand Down
2 changes: 1 addition & 1 deletion api/core/app/apps/chat/app_generator.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,10 @@
from core.app.entities.app_invoke_entities import ChatAppGenerateEntity, InvokeFrom
from core.model_runtime.errors.invoke import InvokeAuthorizationError, InvokeError
from core.ops.ops_trace_manager import TraceQueueManager
from enums import CreatedByRole
from extensions.ext_database import db
from factories import file_factory
from models.account import Account
from models.enums import CreatedByRole
from models.model import App, EndUser

logger = logging.getLogger(__name__)
Expand Down
2 changes: 1 addition & 1 deletion api/core/app/apps/completion/app_generator.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@
from core.app.entities.app_invoke_entities import CompletionAppGenerateEntity, InvokeFrom
from core.model_runtime.errors.invoke import InvokeAuthorizationError, InvokeError
from core.ops.ops_trace_manager import TraceQueueManager
from enums import CreatedByRole
from extensions.ext_database import db
from factories import file_factory
from models import Account, App, EndUser, Message
from models.enums import CreatedByRole
from services.errors.app import MoreLikeThisDisabledError
from services.errors.message import MessageNotExistsError

Expand Down
2 changes: 1 addition & 1 deletion api/core/app/apps/workflow/app_generator.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@
from core.app.entities.task_entities import WorkflowAppBlockingResponse, WorkflowAppStreamResponse
from core.model_runtime.errors.invoke import InvokeAuthorizationError, InvokeError
from core.ops.ops_trace_manager import TraceQueueManager
from enums import CreatedByRole
from extensions.ext_database import db
from factories import file_factory
from models import Account, App, EndUser, Workflow
from models.enums import CreatedByRole

logger = logging.getLogger(__name__)

Expand Down
2 changes: 1 addition & 1 deletion api/core/app/apps/workflow/app_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@
from core.workflow.entities.variable_pool import VariablePool
from core.workflow.enums import SystemVariableKey
from core.workflow.workflow_entry import WorkflowEntry
from enums import UserFrom
from extensions.ext_database import db
from models.enums import UserFrom
from models.model import App, EndUser
from models.workflow import WorkflowType

Expand Down
2 changes: 1 addition & 1 deletion api/core/app/task_pipeline/workflow_cycle_manage.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,9 @@
from core.workflow.nodes import NodeType
from core.workflow.nodes.tool.entities import ToolNodeData
from core.workflow.workflow_entry import WorkflowEntry
from enums import CreatedByRole, WorkflowRunTriggeredFrom
from extensions.ext_database import db
from models.account import Account
from models.enums import CreatedByRole, WorkflowRunTriggeredFrom
from models.model import EndUser
from models.workflow import (
Workflow,
Expand Down
2 changes: 1 addition & 1 deletion api/core/tools/tool_engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@
from core.tools.tool.tool import Tool
from core.tools.tool.workflow_tool import WorkflowTool
from core.tools.utils.message_transformer import ToolFileMessageTransformer
from enums import CreatedByRole
from extensions.ext_database import db
from models.enums import CreatedByRole
from models.model import Message, MessageFile


Expand Down
2 changes: 1 addition & 1 deletion api/core/workflow/entities/workflow_entities.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

from core.app.entities.app_invoke_entities import InvokeFrom
from core.workflow.nodes.base import BaseIterationState, BaseNode
from enums import UserFrom
from models.enums import UserFrom
from models.workflow import Workflow, WorkflowType

from .node_entities import NodeRunResult
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from pydantic import BaseModel, Field

from core.app.entities.app_invoke_entities import InvokeFrom
from enums import UserFrom
from models.enums import UserFrom
from models.workflow import WorkflowType


Expand Down
2 changes: 1 addition & 1 deletion api/core/workflow/graph_engine/graph_engine.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@
from core.workflow.nodes.end.end_stream_processor import EndStreamProcessor
from core.workflow.nodes.event import RunCompletedEvent, RunRetrieverResourceEvent, RunStreamChunkEvent
from core.workflow.nodes.node_mapping import node_type_classes_mapping
from enums import UserFrom
from extensions.ext_database import db
from models.enums import UserFrom
from models.workflow import WorkflowNodeExecutionStatus, WorkflowType

logger = logging.getLogger(__name__)
Expand Down
2 changes: 1 addition & 1 deletion api/core/workflow/workflow_entry.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
from core.workflow.nodes.event import NodeEvent
from core.workflow.nodes.llm import LLMNodeData
from core.workflow.nodes.node_mapping import node_type_classes_mapping
from enums import UserFrom
from models.enums import UserFrom
from models.workflow import (
Workflow,
WorkflowType,
Expand Down
2 changes: 1 addition & 1 deletion api/factories/file_factory.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@
from constants import AUDIO_EXTENSIONS, DOCUMENT_EXTENSIONS, IMAGE_EXTENSIONS, VIDEO_EXTENSIONS
from core.file import File, FileBelongsTo, FileExtraConfig, FileTransferMethod, FileType
from core.helper import ssrf_proxy
from enums import CreatedByRole
from extensions.ext_database import db
from models import MessageFile, ToolFile, UploadFile
from models.enums import CreatedByRole


def build_from_message_files(
Expand Down
15 changes: 0 additions & 15 deletions api/enums/__init__.py → api/models/enums.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,30 +2,15 @@


class CreatedByRole(str, Enum):
"""
TODO: Need to write docstring
"""

ACCOUNT = "account"
END_USER = "end_user"


class UserFrom(str, Enum):
"""
TODO: Need to write docstring
"""

ACCOUNT = "account"
END_USER = "end-user"


class WorkflowRunTriggeredFrom(str, Enum):
DEBUGGING = "debugging"
APP_RUN = "app-run"


__all__ = [
"CreatedByRole",
"UserFrom",
"WorkflowRunTriggeredFrom",
]
2 changes: 1 addition & 1 deletion api/models/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@
from core.file import FILE_MODEL_IDENTITY, File, FileExtraConfig, FileTransferMethod, FileType
from core.file import helpers as file_helpers
from core.file.tool_file_parser import ToolFileParser
from enums import CreatedByRole
from extensions.ext_database import db
from libs.helper import generate_string
from models.enums import CreatedByRole

from .account import Account, Tenant
from .types import StringUUID
Expand Down
2 changes: 1 addition & 1 deletion api/models/workflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,10 @@
from constants import HIDDEN_VALUE
from core.helper import encrypter
from core.variables import SecretVariable, Variable
from enums import CreatedByRole
from extensions.ext_database import db
from factories import variable_factory
from libs import helper
from models.enums import CreatedByRole

from .account import Account
from .types import StringUUID
Expand Down
2 changes: 1 addition & 1 deletion api/services/workflow_app_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
from flask_sqlalchemy.pagination import Pagination
from sqlalchemy import and_, or_

from enums import CreatedByRole
from extensions.ext_database import db
from models import App, EndUser, WorkflowAppLog, WorkflowRun
from models.enums import CreatedByRole
from models.workflow import WorkflowRunStatus


Expand Down
2 changes: 1 addition & 1 deletion api/services/workflow_run_service.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from enums import WorkflowRunTriggeredFrom
from extensions.ext_database import db
from libs.infinite_scroll_pagination import InfiniteScrollPagination
from models.enums import WorkflowRunTriggeredFrom
from models.model import App
from models.workflow import (
WorkflowNodeExecution,
Expand Down
2 changes: 1 addition & 1 deletion api/services/workflow_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@
from core.workflow.nodes.event import RunCompletedEvent
from core.workflow.nodes.node_mapping import node_type_classes_mapping
from core.workflow.workflow_entry import WorkflowEntry
from enums import CreatedByRole
from events.app_event import app_draft_workflow_was_synced, app_published_workflow_was_updated
from extensions.ext_database import db
from models.account import Account
from models.enums import CreatedByRole
from models.model import App, AppMode
from models.workflow import (
Workflow,
Expand Down
2 changes: 1 addition & 1 deletion api/tests/integration_tests/workflow/nodes/test_code.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
from core.workflow.graph_engine.entities.graph_runtime_state import GraphRuntimeState
from core.workflow.nodes.code.code_node import CodeNode
from core.workflow.nodes.code.entities import CodeNodeData
from enums import UserFrom
from models.enums import UserFrom
from models.workflow import WorkflowNodeExecutionStatus, WorkflowType
from tests.integration_tests.workflow.nodes.__mock.code_executor import setup_code_executor_mock

Expand Down
2 changes: 1 addition & 1 deletion api/tests/integration_tests/workflow/nodes/test_http.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
from core.workflow.graph_engine.entities.graph_init_params import GraphInitParams
from core.workflow.graph_engine.entities.graph_runtime_state import GraphRuntimeState
from core.workflow.nodes.http_request.node import HttpRequestNode
from enums import UserFrom
from models.enums import UserFrom
from models.workflow import WorkflowType
from tests.integration_tests.workflow.nodes.__mock.http import setup_http_mock

Expand Down
2 changes: 1 addition & 1 deletion api/tests/integration_tests/workflow/nodes/test_llm.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@
from core.workflow.graph_engine.entities.graph_runtime_state import GraphRuntimeState
from core.workflow.nodes.event import RunCompletedEvent
from core.workflow.nodes.llm.node import LLMNode
from enums import UserFrom
from extensions.ext_database import db
from models.enums import UserFrom
from models.provider import ProviderType
from models.workflow import WorkflowNodeExecutionStatus, WorkflowType

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@
from core.workflow.graph_engine.entities.graph_init_params import GraphInitParams
from core.workflow.graph_engine.entities.graph_runtime_state import GraphRuntimeState
from core.workflow.nodes.parameter_extractor.parameter_extractor_node import ParameterExtractorNode
from enums import UserFrom
from extensions.ext_database import db
from models.enums import UserFrom
from models.provider import ProviderType

"""FOR MOCK FIXTURES, DO NOT REMOVE"""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
from core.workflow.graph_engine.entities.graph_init_params import GraphInitParams
from core.workflow.graph_engine.entities.graph_runtime_state import GraphRuntimeState
from core.workflow.nodes.template_transform.template_transform_node import TemplateTransformNode
from enums import UserFrom
from models.enums import UserFrom
from models.workflow import WorkflowNodeExecutionStatus, WorkflowType
from tests.integration_tests.workflow.nodes.__mock.code_executor import setup_code_executor_mock

Expand Down
2 changes: 1 addition & 1 deletion api/tests/integration_tests/workflow/nodes/test_tool.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
from core.workflow.graph_engine.entities.graph_init_params import GraphInitParams
from core.workflow.graph_engine.entities.graph_runtime_state import GraphRuntimeState
from core.workflow.nodes.tool.tool_node import ToolNode
from enums import UserFrom
from models.enums import UserFrom
from models.workflow import WorkflowNodeExecutionStatus, WorkflowType


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
from core.workflow.graph_engine.graph_engine import GraphEngine
from core.workflow.nodes.event import RunCompletedEvent, RunStreamChunkEvent
from core.workflow.nodes.llm.node import LLMNode
from enums import UserFrom
from models.enums import UserFrom
from models.workflow import WorkflowNodeExecutionStatus, WorkflowType


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@
from core.workflow.graph_engine.entities.graph_init_params import GraphInitParams
from core.workflow.graph_engine.entities.graph_runtime_state import GraphRuntimeState
from core.workflow.nodes.answer.answer_node import AnswerNode
from enums import UserFrom
from extensions.ext_database import db
from models.enums import UserFrom
from models.workflow import WorkflowNodeExecutionStatus, WorkflowType


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
from core.workflow.nodes.event import RunCompletedEvent
from core.workflow.nodes.iteration.iteration_node import IterationNode
from core.workflow.nodes.template_transform.template_transform_node import TemplateTransformNode
from enums import UserFrom
from models.enums import UserFrom
from models.workflow import WorkflowNodeExecutionStatus, WorkflowType


Expand Down
2 changes: 1 addition & 1 deletion api/tests/unit_tests/core/workflow/nodes/test_answer.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@
from core.workflow.graph_engine.entities.graph_init_params import GraphInitParams
from core.workflow.graph_engine.entities.graph_runtime_state import GraphRuntimeState
from core.workflow.nodes.answer.answer_node import AnswerNode
from enums import UserFrom
from extensions.ext_database import db
from models.enums import UserFrom
from models.workflow import WorkflowNodeExecutionStatus, WorkflowType


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
HttpRequestNodeData,
)
from core.workflow.nodes.http_request.executor import _plain_text_to_dict
from enums import UserFrom
from models.enums import UserFrom
from models.workflow import WorkflowNodeExecutionStatus, WorkflowType


Expand Down
2 changes: 1 addition & 1 deletion api/tests/unit_tests/core/workflow/nodes/test_if_else.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@
from core.workflow.nodes.if_else.entities import IfElseNodeData
from core.workflow.nodes.if_else.if_else_node import IfElseNode
from core.workflow.utils.condition.entities import Condition, SubCondition, SubVariableCondition
from enums import UserFrom
from extensions.ext_database import db
from models.enums import UserFrom
from models.workflow import WorkflowNodeExecutionStatus, WorkflowType


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
from core.workflow.graph_engine.entities.graph_init_params import GraphInitParams
from core.workflow.graph_engine.entities.graph_runtime_state import GraphRuntimeState
from core.workflow.nodes.variable_assigner import VariableAssignerNode, WriteMode
from enums import UserFrom
from models.enums import UserFrom
from models.workflow import WorkflowType

DEFAULT_NODE_ID = "node_id"
Expand Down
Loading