Skip to content

Commit b2bb049

Browse files
luoluoyuyuluoluoyuyu
committed
update README.md
1 parent 729e9b3 commit b2bb049

File tree

1 file changed

+64
-101
lines changed

1 file changed

+64
-101
lines changed

‎README.md‎

Lines changed: 64 additions & 101 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,6 @@
22

33
**Function Stream** is a high-performance, event-driven stream processing framework built in Rust. It provides a modular runtime to orchestrate serverless-style processing functions compiled to **WebAssembly (WASM)**, supporting functions written in **Go, Python, and Rust**.
44

5-
6-
75
## Key Features
86

97
* **Event-Driven WASM Runtime**: Executes polyglot functions (Go, Python, Rust) with near-native performance and sandboxed isolation.
@@ -12,147 +10,112 @@
1210

1311
## Repository Layout
1412

15-
```
13+
```text
1614
function-stream/
17-
├── src/ # Core runtime, coordinator, server, config, SQL parser
18-
├── protocol/ # Protocol Buffers definitions and generated gRPC code
19-
├── cli/cli/ # SQL REPL client
15+
├── src/ # Core runtime, coordinator, server, config
16+
├── protocol/ # Protocol Buffers definitions
17+
├── cli/ # SQL REPL client
2018
├── conf/ # Default runtime configuration
21-
├── examples/ # Sample processors and integration examples
22-
├── python/ # Python API, client, and runtime WASM generator
23-
├── Makefile # Build and packaging automation
19+
├── examples/ # Sample processors
20+
├── python/ # Python API, Client, and Runtime (WASM)
21+
├── scripts/ # Build and environment automation scripts
22+
├── Makefile # Unified build system
2423
└── Cargo.toml # Workspace manifest
2524
```
2625

2726
## Prerequisites
2827

29-
- Rust toolchain (recommend `rustup` with stable >= 1.77)
30-
- `protoc` (Protocol Buffers compiler) for generating gRPC bindings
31-
- Build tooling for `rdkafka`: `cmake`, `pkg-config`, and OpenSSL headers
32-
- Optional (for Python processors and full package):
33-
- Python 3.9+ with `venv`
34-
- `componentize-py` (installed via `make install-deps` inside `python/functionstream-runtime`)
35-
36-
## Build From Source
37-
38-
Clone the repository and install dependencies:
28+
* **Rust Toolchain**: Stable >= 1.77 (via rustup).
29+
* **Python 3.9+**: Required for building the Python WASM runtime.
30+
* **Protoc**: Protocol Buffers compiler (for generating gRPC bindings).
31+
* **Build Tools**: cmake, pkg-config, OpenSSL headers (for rdkafka).
3932

40-
```
41-
git clone https://github.com/<your-org>/function-stream.git
42-
cd function-stream
43-
cargo fetch
44-
```
45-
46-
Build targets:
33+
## Quick Start (Local Development)
4734

48-
- Debug build (fast iteration):
49-
```
50-
cargo build
51-
```
52-
- Release build with Python support (default features):
53-
```
54-
cargo build --release
55-
```
56-
- Release build without Python (lite):
57-
```
58-
cargo build --release --no-default-features --features incremental-cache
59-
```
35+
### 1. Initialize Environment
6036

61-
To regenerate Protocol Buffers, rerun the build; `tonic-build` automatically recompiles when `protocol/proto/function_stream.proto` changes.
37+
We provide an automated script to set up the Python virtual environment (`.venv`), install dependencies, and compile necessary sub-modules.
6238

63-
## Run Locally
39+
```bash
40+
make env
41+
```
6442

65-
### Prepare configuration
43+
This runs `scripts/setup.sh`, creating a `.venv` and installing the API, Client, and Runtime builder tools.
6644

67-
1. Review `conf/config.yaml` and adjust service host/port, logging, state storage, and Python runtime settings as needed.
68-
2. Optionally point `FUNCTION_STREAM_CONF` to a custom configuration file or directory:
69-
```
70-
export FUNCTION_STREAM_CONF=/path/to/config.yaml
71-
```
72-
3. Ensure `data/` and `logs/` directories are writable (they are created automatically on startup).
45+
### 2. Build & Run Server
7346

74-
### Start the control plane
47+
Start the control plane in release mode. The build system will automatically compile the Python WASM runtime if it's missing.
7548

76-
```
49+
```bash
7750
cargo run --release --bin function-stream
7851
```
7952

80-
The server logs appear under `logs/app.log` (default JSON format). Stop the service with `Ctrl+C`.
53+
Logs are output to `logs/app.log` (JSON format) and stdout. Default configuration is loaded from `conf/config.yaml`.
8154

82-
### Use the SQL CLI
55+
### 3. Run CLI
8356

84-
Run the interactive CLI from another terminal to issue SQL statements:
57+
In a separate terminal, launch the SQL REPL:
8558

59+
```bash
60+
cargo run -p function-stream-cli -- --host 127.0.0.1 --port 8080
8661
```
87-
cargo run -p function-stream-cli -- --ip 127.0.0.1 --port 8080
88-
```
89-
90-
The CLI connects to the gRPC endpoint exposed by the server.
9162

92-
### Try sample processors
63+
## Building & Packaging
9364

94-
- Python processor example:
95-
```
96-
cd examples/python-processor
97-
python main.py
98-
```
99-
- Kafka integration tests and Go processor examples are under `examples/`.
65+
The project uses a standard Makefile to handle compilation and distribution. Artifacts are generated in the `dist/` directory.
10066

101-
## Packaging
67+
### Build Targets
10268

103-
Packaging is orchestrated by the top-level `Makefile`. Outputs are placed in `dist/`.
69+
| Command | Description |
70+
|----------------|--------------------------------------------------|
71+
| `make` | Alias for `make build`. Compiles Rust binaries and Python WASM runtime. |
72+
| `make build` | Builds the Full version (Rust + Python Support). |
73+
| `make build-lite` | Builds the Lite version (Rust only, no Python dependencies). |
10474

105-
### Full distribution (includes Python runtime)
75+
### Distribution (Packaging)
10676

107-
1. Create the virtual environment once:
108-
```
109-
python3 -m venv .venv
110-
source .venv/bin/activate
111-
make -C python/functionstream-runtime install-deps build
112-
deactivate
113-
```
114-
2. Build and package:
115-
```
116-
make package-full
117-
```
77+
To create production-ready archives (`.tar.gz` and `.zip`), use the dist targets.
11878

119-
Artifacts:
120-
- `dist/function-stream-<version>/` directory containing binaries, config, logs/data skeletons, README, and Python WASM runtime.
121-
- `dist/packages/function-stream-<version>.zip` and `.tar.gz`.
79+
#### 1. Function Stream (Full)
12280

123-
### Lite distribution (Rust-only)
81+
Includes the Rust binary, Python WASM runtime, configuration, and default directory structure.
12482

125-
```
126-
make package-lite
83+
```bash
84+
make dist
12785
```
12886

129-
Artifacts:
130-
- `dist/function-stream-<version>-lite/` directory with binaries and configs.
131-
- `dist/packages/function-stream-<version>-lite.zip` and `.tar.gz`.
87+
**Output:**
88+
* `dist/function-stream-<version>.tar.gz`
89+
* `dist/function-stream-<version>.zip`
13290

133-
### Package all variants
91+
#### 2. Function Stream Lite
13492

135-
```
136-
make package-all
137-
```
93+
A lightweight distribution without the Python WASM runtime or dependencies.
13894

139-
The script cleans previous `dist/` contents, produces both full and lite packages, and lists generated archives.
95+
```bash
96+
make dist-lite
97+
```
14098

141-
## Testing
99+
**Output:**
100+
* `dist/function-stream-<version>-lite.tar.gz`
101+
* `dist/function-stream-<version>-lite.zip`
142102

143-
Run the Rust test suite:
103+
## Maintenance
144104

145-
```
146-
cargo test
147-
```
105+
| Command | Description |
106+
|----------------|------------------------------------------------------------------|
107+
| `make clean` | Deep clean. Removes `target/`, `dist/`, `.venv/`, and all temporary artifacts. |
108+
| `make env-clean` | Removes only the Python virtual environment and Python artifacts. |
109+
| `make test` | Runs the Rust test suite. |
148110

149-
Python components expose their own `Makefile` targets (for example, `make -C python/functionstream-runtime test` if defined).
111+
## Configuration
150112

151-
## Environment Variables
113+
The runtime behavior is controlled by `conf/config.yaml`. You can override the configuration location using environment variables:
152114

153-
- `FUNCTION_STREAM_HOME`: Overrides the project root for resolving data/log directories.
154-
- `FUNCTION_STREAM_CONF`: Points to a configuration file or directory containing `config.yaml`.
115+
```bash
116+
export FUNCTION_STREAM_CONF=/path/to/custom/config.yaml
117+
```
155118

156119
## License
157120

158-
Licensed under the Apache License, Version 2.0. See `LICENSE` for details.
121+
Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) for details.

0 commit comments

Comments
 (0)