Skip to content

Commit 5429b12

Browse files
[pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
1 parent f5d435b commit 5429b12

File tree

5 files changed

+36
-30
lines changed

5 files changed

+36
-30
lines changed

FinanceAgent/README.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Finance Agent Example
1+
# Finance Agent Example
22

33
## Table of Contents
44

@@ -10,20 +10,20 @@
1010
- [Deployment Options](#deployment-options)
1111
- [Contribution](#contribution)
1212

13-
14-
1513
## Overview
1614

1715
The Finance Agent example showcases a hierarchical multi-agent system designed to assist users with financial document processing and analysis. It provides three main functionalities: summarizing lengthy financial documents, answering queries related to financial documents, and conducting research to generate investment reports on public companies.
1816

1917
Users interact with the system via a graphical user interface (UI), where requests are managed by a supervisor agent that delegates tasks to worker agents or the summarization microservice. The system supports document uploads through the UI for processing.
2018

21-
2219
## Problem Motivation
20+
2321
Navigating and analyzing extensive financial documents can be challenging and time-consuming. Users often require concise summaries, answers to specific queries, or comprehensive investment reports. The Finance Agent addresses these needs by automating document summarization, query answering, and research tasks, thereby enhancing productivity and decision-making efficiency.
2422

2523
## Architecture
24+
2625
### High-Level Diagram
26+
2727
The Finance Agent system is structured as a hierarchical multi-agent architecture. User interactions are managed by a supervisor agent, which coordinates tasks among worker agents and the summarization microservice. The system supports document uploads and processing through the UI.
2828

2929
The architecture of this Finance Agent example is shown in the figure below. The agent is a hierarchical multi-agent system and has 3 main functions:
@@ -36,7 +36,8 @@ The user interacts with the supervisor agent through the graphical UI. The super
3636

3737
![Finance Agent Architecture](assets/finance_agent_arch.png)
3838

39-
### OPEA Microservices Diagram for Data Handling
39+
### OPEA Microservices Diagram for Data Handling
40+
4041
The architectural diagram of the `dataprep` microservice is shown below. We use [docling](https://github.com/docling-project/docling) to extract text from PDFs and URLs into markdown format. Both the full document content and tables are extracted. We then use an LLM to extract metadata from the document, including the company name, year, quarter, document type, and document title. The full document markdown then gets chunked, and LLM is used to summarize each chunk, and the summaries are embedded and saved to a vector database. Each table is also summarized by LLM and the summaries are embedded and saved to the vector database. The chunks and tables are also saved into a KV store. The pipeline is designed as such to improve retrieval accuracy of the `search_knowledge_base` tool used by the Question Answering worker agent.
4142

4243
![dataprep architecture](assets/fin_agent_dataprep.png)
@@ -55,8 +56,8 @@ The Question Answering worker agent uses `search_knowledge_base` tool to get rel
5556

5657
![finqa search tool arch](assets/finqa_tool.png)
5758

58-
5959
## Deployment Options
60+
6061
This CodeGen example can be deployed manually on various hardware platforms using Docker Compose or Kubernetes. Select the appropriate guide based on your target environment:
6162

6263
| Hardware | Deployment Mode | Guide Link |
@@ -65,7 +66,6 @@ This CodeGen example can be deployed manually on various hardware platforms usin
6566

6667
_Note: Building custom microservice images can be done using the resources in [GenAIComps](https://github.com/opea-project/GenAIComps)._
6768

68-
6969
## Contribution
70-
We welcome contributions to the OPEA project. Please refer to the contribution guidelines for more information.
7170

71+
We welcome contributions to the OPEA project. Please refer to the contribution guidelines for more information.

FinanceAgent/docker_compose/intel/hpu/gaudi/README.md

Lines changed: 24 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
# Deploy Finance Agent on Intel Gaudi HPU with Docker Compose
2+
23
This README provides instructions for deploying the Finance Agent application using Docker Compose on systems equipped with Intel Gaudi HPUs.
34

45
## Table of Contents
@@ -14,6 +15,7 @@ This README provides instructions for deploying the Finance Agent application us
1415
This guide focuses on running the pre-configured Finance Agent service using Docker Compose on Intel Gaudi HPUs. It leverages containers optimized for Gaudi for the LLM serving component, along with CPU-based containers for other microservices like embedding, retrieval, data preparation and the UI.
1516

1617
## Prerequisites
18+
1719
- Docker and Docker Compose installed.
1820
- Intel Gaudi HPU(s) with the necessary drivers and software stack installed on the host system. (Refer to Intel Gaudi Documentation).
1921
- Git installed (for cloning repository).
@@ -31,8 +33,11 @@ Clone the GenAIExamples repository:
3133
```
3234

3335
## Start Deployment
36+
3437
This uses the default vLLM-based deployment profile (vllm-gaudi-server).
38+
3539
### Configure Environment
40+
3641
Set required environment variables in your shell:
3742

3843
```shell
@@ -41,12 +46,12 @@ Set required environment variables in your shell:
4146
# Path to your model cache
4247
export HF_CACHE_DIR="./data"
4348
# Go to https://finnhub.io/ to get your free api key
44-
export FINNHUB_API_KEY=<your-finnhub-api-key>
49+
export FINNHUB_API_KEY=<your-finnhub-api-key>
4550
# Go to https://docs.financialdatasets.ai/ to get your free api key
46-
export FINANCIAL_DATASETS_API_KEY=<your-api-key>
51+
export FINANCIAL_DATASETS_API_KEY=<your-api-key>
4752

4853
# Optional: Configure HOST_IP if needed
49-
# Replace with your host's external IP address (do not use localhost or 127.0.0.1).
54+
# Replace with your host's external IP address (do not use localhost or 127.0.0.1).
5055
# export HOST_IP=$(hostname -I | awk '{print $1}')
5156
# Optional: Configure proxy if needed
5257
# export http_proxy="your_http_proxy"
@@ -59,19 +64,21 @@ Set required environment variables in your shell:
5964
Note: The compose file might read additional variables from set_env.sh. Ensure all required variables like ports (LLM_SERVICE_PORT, TEI_EMBEDDER_PORT, etc.) are set if not using defaults from the compose file. For instance, edit the set_env.sh to change the LLM model:
6065

6166
### Start Services
67+
6268
#### Deploy with Docker Compose
69+
6370
Below is the command to launch services
64-
- vllm-gaudi-server
65-
- tei-embedding-serving
66-
- redis-vector-db
67-
- redis-kv-store
68-
- dataprep-redis-server-finance
69-
- finqa-agent-endpoint
70-
- research-agent-endpoint
71-
- docsum-vllm-gaudi
72-
- supervisor-agent-endpoint
73-
- agent-ui
7471

72+
- vllm-gaudi-server
73+
- tei-embedding-serving
74+
- redis-vector-db
75+
- redis-kv-store
76+
- dataprep-redis-server-finance
77+
- finqa-agent-endpoint
78+
- research-agent-endpoint
79+
- docsum-vllm-gaudi
80+
- supervisor-agent-endpoint
81+
- agent-ui
7582

7683
```shell
7784
docker compose up -d
@@ -102,14 +109,16 @@ If deploy on Gaudi, also need to build vllm image.
102109
docker build --no-cache -f Dockerfile.hpu -t opea/vllm-gaudi:latest --shm-size=128g . --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy
103110
```
104111

105-
106112
## Validate Services
113+
107114
Wait several minutes for models to download and services to initialize (Gaudi initialization can take time). Check container logs (docker compose logs -f <service_name>, especially vllm-gaudi-server).
115+
108116
```bash
109117
docker logs --tail 2000 -f vllm-gaudi-server
110-
```
118+
```
111119

112120
### Validate Data Services
121+
113122
Ingest data and retrieval from database
114123

115124
```bash

FinanceAgent/docker_compose/intel/hpu/gaudi/compose.yaml

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,7 @@ services:
118118
REDIS_URL_KV: ${REDIS_URL_KV}
119119
TEI_EMBEDDING_ENDPOINT: ${TEI_EMBEDDING_ENDPOINT}
120120
LLM_ENDPOINT: ${LLM_ENDPOINT}
121-
LLM_MODEL: ${LLM_MODEL_ID}
121+
LLM_MODEL: ${LLM_MODEL_ID}
122122
HUGGINGFACEHUB_API_TOKEN: ${HF_TOKEN}
123123
HF_TOKEN: ${HF_TOKEN}
124124
LOGFLAG: true
@@ -215,5 +215,3 @@ services:
215215
ports:
216216
- "5175:8080"
217217
ipc: host
218-
219-

FinanceAgent/docker_compose/intel/set_env.sh

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ export HTTP_PROXY="${http_proxy}"
3232
export HTTPS_PROXY="${https_proxy}"
3333

3434
# VLLM configuration
35-
export VLLM_PORT="${VLLM_PORT:-8086}"
35+
export VLLM_PORT="${VLLM_PORT:-8086}"
3636
export VLLM_VOLUME="${VLLM_VOLUME:-/data2/huggingface}"
3737
export VLLM_IMAGE="${VLLM_IMAGE:-opea/vllm-gaudi:latest}"
3838
export LLM_MODEL_ID="${LLM_MODEL_ID:-meta-llama/Llama-3.3-70B-Instruct}"
@@ -79,7 +79,7 @@ export FINANCIAL_DATASETS_API_KEY="${FINANCIAL_DATASETS_API_KEY}"
7979

8080

8181
# Toolset and prompt paths
82-
if check_var "WORKDIR"; then
82+
if check_var "WORKDIR"; then
8383
export TOOLSET_PATH=$WORKDIR/GenAIExamples/FinanceAgent/tools/
8484
export PROMPT_PATH=$WORKDIR/GenAIExamples/FinanceAgent/prompts/
8585

@@ -96,4 +96,3 @@ if check_var "WORKDIR"; then
9696
fi
9797
done
9898
fi
99-

FinanceAgent/tests/test_compose_on_gaudi.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ export HTTPS_PROXY="${https_proxy}"
1717

1818
# VLLM configuration
1919
MODEL=meta-llama/Llama-3.3-70B-Instruct
20-
export VLLM_PORT="${VLLM_PORT:-8086}"
20+
export VLLM_PORT="${VLLM_PORT:-8086}"
2121

2222
# export HF_CACHE_DIR="${HF_CACHE_DIR:-"./data"}"
2323
export HF_CACHE_DIR=${model_cache:-"./data2/huggingface"}

0 commit comments

Comments
 (0)