Dynamic Workflow Orchestration in LLMAgent

[madawei2699]

Dynamic Workflow Orchestration in LLMAgent

Source Code: Investment Portfolio Example

Introduction

LLMAgent's core innovation is its ability to create emergent, adaptive workflows that respond to user inputs, context changes, and intermediate results—rather than following predefined, static execution paths. This document explores how the Investment Portfolio example demonstrates this capability through signal-based architecture, contextual decision-making, and state-driven tool orchestration.

Signal-Based Workflow Architecture

The sequence diagram reveals LLMAgent's signal-based architecture in action:

1. Each user message initiates a signal (user_message) that flows through a system of specialized handlers
2. Handlers transform signals into new signal types (thinking, tool_call, tool_result, response) based on context
3. The flow dynamically adjusts based on each signal's type and content

Unlike traditional workflows with fixed execution paths, LLMAgent creates an emergent workflow where each step determines the next appropriate action.

Contextual Decision Making

The most powerful aspect of LLMAgent's dynamic workflow capability is demonstrated in the logs, where:

23:25:57.118 [info] Processing user message: "I'd like to create a retirement portfolio. I prefer moderate risk investments."
23:25:57.120 [debug] MessageHandler - Calling LLM with provider: MockInvestmentProvider
23:25:57.121 [info] Building conversation context based on: "I'd like to create a retirement portfolio. I prefer moderate risk investments."
23:25:57.125 [info] Current workflow state: Phase 0: Initial Analysis

Here the MockInvestmentProvider (LLM) analyzes the conversation context and dynamically decides:

• Which tools to use (if any)
• What parameters to pass to tools
• Whether to think further before acting
• When to respond directly versus execute tools

This contextual analysis considers:

• User's investment goals and risk preferences
• Current state of portfolio construction
• Previous tool results
• Conversation history

State-Driven Tool Orchestration

The investment_portfolio.exs example demonstrates sophisticated state management:

cond do
  # INITIAL PORTFOLIO ANALYSIS PATH
  (context.is_new_portfolio_request or String.length(last_user_message) > 0) and
      not context.has_etf_data ->
    Logger.info("Starting new portfolio analysis workflow")
    # Calls etf_screener tool

  # PORTFOLIO CONSTRUCTION PATH
  context.has_etf_data and not context.has_portfolio ->
    Logger.info("Proceeding to portfolio construction phase")
    # Calls portfolio_constructor tool
    
  # ERROR RECOVERY PATH
  context.has_error ->
    Logger.info("Recovering from previous error: #{context.error_message}")
    
    case context.failed_tool do
      "portfolio_constructor" -> {...}
      _ -> {...}
    end
end

This conditional logic enables:

1. Branching Paths: Different execution flows based on current state
2. Tool Chaining: Results from one tool feed into subsequent tool calls
3. Error Recovery: Detection of failures with appropriate fallback strategies

Sequence Diagram

The following sequence diagram strictly reflects what's shown in the provided logs and code logic:

sequenceDiagram
    autonumber
    Actor Client as Client
    participant Flow as Conversation Flow
    participant MH as MessageHandler
    participant TOH as ToolHandler
    participant TRH as ToolResultHandler
    participant Store as Store
    participant LLM as MockInvestmentProvider
    participant Tools as InvestmentTools
    
    Client->>+Flow: "I'd like to create a retirement portfolio. I prefer moderate risk investments."
    Note over Flow: 23:25:57.105 [debug]<br/>Processing signal: :user_message
    
    Flow->>+MH: handle_user_message
    Note over MH: 23:25:57.118 [info]<br/>Processing user message
    Note over MH: 23:25:57.120 [debug]<br/>Calling LLM with provider
    
    MH->>+LLM: generate_response(messages, opts)
    Note over LLM: 23:25:57.125 [info]<br/>Starting new portfolio analysis workflow
    
    LLM-->>-MH: Response with etf_screener tool_call<br/>23:25:57.128 [debug] {...tool_calls => [%{..."etf_screener"...}]}
    
    MH->>Flow: {:emit, tool_call_signal, new_state}
    Flow->>+TOH: handle_tool_call
    
    TOH->>+Tools: Execute etf_screener({category: "Broad Market", risk_level: "Moderate"})
    Tools-->>-TOH: ETF screening results
    
    TOH->>Store: Store tool result
    TOH->>Flow: {:emit, tool_result_signal, new_state}
    
    Flow->>+TRH: handle_tool_result
    TRH->>Store: Add function result to history
    TRH->>+LLM: generate_response(with_updated_history)
    
    LLM-->>-TRH: Response with content
    TRH->>Flow: {:emit, response_signal, new_state}
    
    Flow-->>-Client: "I'll analyze available investment options for your Moderate risk profile."
    
    Client->>+Flow: "Can you make it a bit more conservative? I'm worried about market volatility."
    Flow->>+MH: handle_user_message
    Note over MH: 23:25:57.230 [info]<br/>Processing user message
    
    MH->>+LLM: generate_response(messages, opts)
    Note over LLM: 23:25:57.231 [info]<br/>Starting new portfolio analysis workflow
    
    LLM-->>-MH: Response with etf_screener tool_call<br/>{...tool_calls => [%{..."Low" risk_level...}]}
    
    MH->>Flow: {:emit, tool_call_signal, new_state}
    Flow->>+TOH: handle_tool_call
    
    TOH->>+Tools: Execute etf_screener({category: "Broad Market", risk_level: "Low"})
    Tools-->>-TOH: Conservative ETF screening results
    
    TOH->>Store: Store tool result
    TOH->>Flow: {:emit, tool_result_signal, new_state}
    
    Flow->>+TRH: handle_tool_result
    TRH->>+LLM: generate_response(with_updated_history)
    LLM-->>-TRH: Response with content
    
    TRH->>Flow: {:emit, response_signal, new_state}
    Flow-->>-Client: "I'll analyze available investment options for your Conservative risk profile."
    
    Client->>+Flow: "Show me how this portfolio has performed historically."
    Flow->>+MH: handle_user_message
    Note over MH: 23:25:57.332 [info]<br/>Processing user message
    
    MH->>+LLM: generate_response(messages, opts)
    Note over LLM: 23:25:57.333 [info]<br/>Starting new portfolio analysis workflow
    
    LLM-->>-MH: Response with etf_screener tool_call<br/>{...tool_calls => [%{..."etf_screener"...}]}
    
    MH->>Flow: {:emit, tool_call_signal, new_state}
    Flow->>+TOH: handle_tool_call
    
    TOH->>+Tools: Execute etf_screener({category: "Broad Market", risk_level: "Moderate"})
    Tools-->>-TOH: ETF screening results
    
    TOH->>Store: Store tool result
    TOH->>Flow: {:emit, tool_result_signal, new_state}
    
    Flow->>+TRH: handle_tool_result
    TRH->>+LLM: generate_response(with_updated_history)
    
    Note over LLM: In a full implementation, this would likely call portfolio_constructor<br/>and portfolio_backtester tools in sequence
    
    LLM-->>-TRH: Response with content
    TRH->>Flow: {:emit, response_signal, new_state}
    
    Flow-->>-Client: "I'll analyze available investment options for your Moderate risk profile."

Loading

Multi-Phase Workflow Execution

The sequence diagram above shows how LLMAgent can orchestrate multi-phase workflows:

1. Analysis Phase: Using etf_screener to gather investment options
2. Construction Phase: Using portfolio_constructor to build the portfolio
3. Evaluation Phase: Using portfolio_backtester to evaluate performance
4. Optimization Phase: Using portfolio_optimizer to refine based on feedback
5. Simulation Phase: Using market_simulator to test market scenarios

Each phase transitions naturally to the next based on state changes and LLM decisions, without requiring predefined DAG structures. This is clearly demonstrated in the logs:

23:25:57.125 [info] Current workflow state: Phase 0: Initial Analysis
23:25:57.125 [info] Starting new portfolio analysis workflow
...
23:25:57.231 [info] Starting new portfolio analysis workflow

Benefits Over Static Workflows

The investment_portfolio example reveals several advantages over traditional, static workflow approaches:

1. Adaptability: Workflows adjust to unexpected user inputs or system states
2. Contextual Intelligence: Decision-making incorporates full conversation context
3. Emergent Complexity: Simple components combine to create sophisticated behaviors
4. Error Resilience: The system can recover from failures with alternative approaches
5. Conversational Coherence: The workflow maintains conversational context through state

Implementation Architecture

Key components enabling dynamic workflows include:

1. Store: Maintains conversation state, tool results, and decision context
2. Signals: Represent different stages of the workflow (messages, thinking, tool calls)
3. Handlers: Process signals and determine next steps based on signal type
4. Flow: Orchestrates the overall signal processing pipeline
5. Tools: Provide domain-specific capabilities that can be dynamically invoked

Together, these components create a system where the workflow emerges from interactions rather than being explicitly defined, allowing for sophisticated, adaptive agent behaviors that more closely mimic human-like problem-solving approaches.

Real-World Applications

This dynamic workflow orchestration approach enables practical applications including:

1. Financial Advisory: Portfolio construction and optimization based on client needs
2. Medical Diagnosis: Adaptive symptom analysis and testing recommendations
3. Customer Support: Dynamic troubleshooting paths based on issue complexity
4. Educational Tutoring: Personalized learning paths responsive to student progress

The ability to adapt workflows in real-time based on conversation context and analysis results makes LLMAgent suitable for complex domains where static workflows would be too limiting.