Skip to content

Commit

Permalink
Feat: Monolake graphs and minor fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
har23k committed Nov 14, 2024
1 parent d3ac52a commit 8463d2b
Show file tree
Hide file tree
Showing 22 changed files with 156 additions and 51 deletions.
2 changes: 1 addition & 1 deletion content/en/docs/monolake/Architecture/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: "Architecture"
linkTitle: "Architecture"
weight: 3
keywords: ["Proxy", "Rust", "io_uring", "Architecture"]
description: "This doc covers architecture design and features"
description: "Architecture and design deep dive"
---


3 changes: 1 addition & 2 deletions content/en/docs/monolake/Architecture/context.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,9 @@
title: "Context Management"
linkTitle: "Context"
weight: 4
description: "Deep dive into Monolake's io_uring-based runtime and performance characteristics compared to traditional event-based runtimes"
---

# Context Management with `certain_map`
# `certain_map`

In a service-oriented architecture, managing the context data that flows between different services is a critical aspect of the system design. The [`certain_map`](https://docs.rs/certain-map/latest/certain_map/) crate provides a powerful way to define and work with typed context data, ensuring the existence of required information at compile-time.

Expand Down
9 changes: 5 additions & 4 deletions content/en/docs/monolake/Architecture/runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,16 @@ One of the defining characteristics of Monoio is its thread-per-core architectur
When working with asynchronous I/O in Rust, understanding the underlying mechanisms of different runtimes is crucial. Two prominent approaches are the io-uring-based runtimes(Monolake) and the traditional event notification based runtimes (Tokio, async-std) which use mechanisms like kequeue and epoll. The fundamental difference between these two models lies in how they manage resource ownership and I/O operations.

io_uring operates on a submission-based model, where the ownership of resources (such as buffers) is transferred to the kernel upon submission of an I/O request. This model allows for high performance and reduced context switching, as the kernel can process the requests asynchronously. When an I/O operation is completed, the ownership of the buffers is returned to the caller. This ownership transfer leads to several implications:
1. Ownership Semantics: In io-uring, since the kernel takes ownership of the buffers during the operation, it allows for more efficient memory management. The caller does not need to manage the lifecycle of the buffers while the operation is in progress.

2. Concurrency Model: The submission-based model allows for a more straightforward handling of concurrency, as multiple I/O operations can be submitted without waiting for each to complete. This can lead to improved throughput, especially in I/O-bound applications.
1. **Ownership Semantics**: In io-uring, since the kernel takes ownership of the buffers during the operation, it allows for more efficient memory management. The caller does not need to manage the lifecycle of the buffers while the operation is in progress.

2. **Concurrency Model**: The submission-based model allows for a more straightforward handling of concurrency, as multiple I/O operations can be submitted without waiting for each to complete. This can lead to improved throughput, especially in I/O-bound applications.

In contrast, Tokio employs systems like kequeue and epoll. In this model, the application maintains ownership of the buffers throughout the lifetime of the I/O operation. Instead of transferring ownership, Tokio merely borrows the buffers, which has several implications:

1. Buffer Management: Since Tokio borrows buffers, the application is responsible for managing their lifecycle. This can introduce complexity, especially when dealing with concurrent I/O operations, as developers must ensure that buffers are not inadvertently reused while still in use.
1. **Buffer Management**: Since Tokio borrows buffers, the application is responsible for managing their lifecycle. This can introduce complexity, especially when dealing with concurrent I/O operations, as developers must ensure that buffers are not inadvertently reused while still in use.

2. Polling Mechanism: The polling model in Tokio requires the application to actively wait for events, which can result in increased context switches and potentially less efficient use of system resources compared to the submission-based model of io-uring.
2. **Polling Mechanism**: The polling model in Tokio requires the application to actively wait for events, which can result in increased context switches and potentially less efficient use of system resources compared to the submission-based model of io-uring.

## Async IO Trait divergence

Expand Down
8 changes: 7 additions & 1 deletion content/en/docs/monolake/Architecture/service.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,13 @@ description: "Overview of Monolake's service architecture, factory patterns, and

## Services

{{< figure src="/img/docs/monolake_service.jpeg" width="1000" height="600" caption="Service Architecture" >}}
<style>
.figure-caption {
text-align: center;
}
</style>

{{< figure src="/img/docs/monolake_service.jpeg" width="1000" height="600" caption="Service Architecture" class="figure-caption" >}}

The Service pattern is a fundamental abstraction in network programming, popularized by the Tower library in the Rust ecosystem. At its core, a Service represents an asynchronous function that processes requests and returns responses. This pattern is particularly powerful for building networking applications as it enables:

Expand Down
1 change: 0 additions & 1 deletion content/en/docs/monolake/Config Guide/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
title: "Config Guide"
linkTitle: "Config Guide"
weight: 5
date: 2023-07-3
description: "Comphrensive guide to configuring monolake proxy"
---

Expand Down
1 change: 0 additions & 1 deletion content/en/docs/monolake/FAQ/_index.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
---
title: "FAQ"
linkTitle: "FAQ"
date: 2023-07-03
weight: 7
keywords: ["Monolake", "HTTP", "Proxy", "Q&A"]
description: "Monolake Frequently Asked Questions and Answers."
Expand Down
2 changes: 1 addition & 1 deletion content/en/docs/monolake/Getting Started/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ description: "This page provides a quick start guide for setting up and running

## Prerequisites

- **Linux Kernel Support**: IO-uring requires a Linux kernel version that has io_uring support. Generally, kernel versions 5.1 and above provide the necessary support. Ensure that your target system has an appropriate kernel version installed. Monolake will fall back to epoll on Mac OS.
- **Linux Kernel Support**: io_uring requires linux kernel support. Generally, kernel versions 5.1 and above provide the necessary support. Ensure that your target system has an appropriate kernel version installed. Monolake will fall back to epoll on Mac OS.
- **Rust nightly**: See the "Rust installation section" below

## Quick Start
Expand Down
49 changes: 19 additions & 30 deletions content/en/docs/monolake/Overview/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ title: "Overview"
linkTitle: "Overview"
weight: 1
keywords: ["Proxy", "Rust", "io-uring"]
description: "This doc provides an overview of Monolake, a high-performance network service framework."
---

## Monolake
Expand All @@ -27,25 +26,19 @@ By focusing on cutting-edge Rust and io_uring, Monolake aims to provide develope

## Performance

HAR TODO: Move performance section from benchmarks to here

### Test environment
#### Request Per Second (RPS) vs. Body Size
| RPS | TP99 |
| :------------------------------------------------- | :-------------------------------------------------: |
| ![image](/img/docs/https_req_per_sec_vs_body_size.png) | ![image](/img/docs/https_tp99_latency_vs_body_size.png) |
| |
| ![image](/img/docs/http_req_per_sec_vs_body_size.png) | ![image](/img/docs/http_tp99_latency_vs_body_size.png) |

### Concurrency performance

HAR TODO: Update with latest benchmarks

| QPS | TP99 | TP999 |
| :------------------------------------------------- | :-------------------------------------------------: | :--------------------------------------------------: |
| ![image](/img/docs/performance_concurrent_qps.png) | ![image](/img/docs/performance_concurrent_tp99.png) | ![image](/img/docs/performance_concurrent_tp999.png) |

### Throughput performance

Change packet size with a fixed concurrency of 100.

| QPS | TP99 | TP999 |
| :----------------------------------------------- | :-----------------------------------------------: | :------------------------------------------------: |
| ![image](/img/docs/performance_bodysize_qps.png) | ![image](/img/docs/performance_bodysize_tp99.png) | ![image](/img/docs/performance_bodysize_tp999.png) |
| RPS | TP99 |
| :------------------------------------------------- | :-------------------------------------------------: |
| ![image](/img/docs/https_req_per_sec_vs_worker_threads.png) | ![image](/img/docs/https_tp99_latency_vs_worker_threads.png) |
| |
| ![image](/img/docs/http_req_per_sec_vs_worker_threads.png) | ![image](/img/docs/http_tp99_latency_vs_worker_threads.png) |

## Related Projects

Expand All @@ -57,15 +50,11 @@ Change packet size with a fixed concurrency of 100.

| Crate | Description |
|-------|-------------|
| [monoio-transports](https://github.com/monoio-rs/monoio-transports) | A foundational crate that provides high-performance, modular networking capabilities, including connectors and utilities for efficient network communications |
| [service-async](https://github.com/ihciah/service-async) | A foundational crate that introduces a refined Service trait with efficient borrowing and zero-cost abstractions, as well as utilities for service composition and state management |
| [certain-map](https://github.com/ihciah/certain-map) | A foundational crate that provides a typed map data structure, ensuring the existence of specific items at compile-time, useful for managing data dependencies between services |
| [monoio-thrift](https://github.com/monoio-rs/monoio-thrift) | Monoio native, io_uring compatible thrift implementation |
| [monoio-http](https://github.com/monoio-rs/monoio-http) | Monoio native, io_uring compatible HTTP/1.1 and HTTP/2 implementation |
| [monoio-nativetls](https://github.com/monoio-rs/monoio-tls) | The native-tls implementation compatible with monoio |
| [monoio-rustls](https://github.com/monoio-rs/monoio-tls) | The rustls implementation compatible with monoio |

## Blogs

- [Monolake: How ByteDance Developed Its Own Rust Proxy to Save Hundreds of Thousands of CPU Cores](TODO)
- [Monolake open source summit](TODO)
| [monoio-transports](https://crates.io/crates/monoio-transports) | A foundational crate that provides high-performance, modular networking capabilities, including connectors and utilities for efficient network communications |
| [service-async](https://crates.io/crates/service-async) | A foundational crate that introduces a refined Service trait with efficient borrowing and zero-cost abstractions, as well as utilities for service composition and state management |
| [certain-map](https://crates.io/crates/certain-map) | A foundational crate that provides a typed map data structure, ensuring the existence of specific items at compile-time, useful for managing data dependencies between services |
| [monoio-thrift](https://crates.io/crates/monoio-thrift) | Monoio native, io_uring compatible thrift implementation |
| [monoio-http](https://crates.io/crates/monoio-http) | Monoio native, io_uring compatible HTTP/1.1 and HTTP/2 implementation |
| [monoio-nativetls](https://crates.io/crates/monoio-native-tls) | The native-tls implementation compatible with monoio |
| [monoio-rustls](https://crates.io/crates/monoio-rustls) | The rustls implementation compatible with monoio |

3 changes: 1 addition & 2 deletions content/en/docs/monolake/Tutorial/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ title: "Tutorial"
linkTitle: "Tutorial"
weight: 4
keywords: ["Proxy", "Rust", "io_uring", "Architecture"]
description: "This is a developer guide"
---

In this guide, we'll walk through the process of implementing a HTTP routing service using the [monolake-services](TODO) crate. This service will handle incoming HTTP requests and route them to appropriate upstream servers based on the configured routes and their associated endpoints.
In this guide, we'll walk through the process of implementing an HTTP routing service using the `monolake-services` crate. This service will handle incoming HTTP requests and route them to appropriate upstream servers based on the configured routes and their associated endpoints.
1 change: 0 additions & 1 deletion content/en/docs/monolake/Tutorial/code.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
title: "Putting it all together"
linkTitle: "Putting it all together"
weight: 3
description: "This doc covers how to manage configuration and context"
---

## Putting it all together
Expand Down
6 changes: 2 additions & 4 deletions content/en/docs/monolake/Tutorial/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ description: "This doc covers how to manage configuration and context"

When building a service-oriented application using the `monolake-services` crate, you'll need to define the necessary configuration fields in your main `ServerConfig` struct. These fields will be used by the service factories to construct the services that make up your application.

The specific fields you'll need to add will depend on the services you're using. For example, if you're implementing a routing HTTP service, you'll need to add a routes field to hold the routing configuration.
The specific fields you'll need to add will depend on the services you're using. For example, if you're implementing a routing HTTP service, you'll probably want to add a routes field to hold the routing configuration.

To configure the routing service, you'll need to add the routes field to your `ServerConfig` struct. This field will hold the RouteConfig structures that define the routing rules.
```rust
Expand Down Expand Up @@ -51,9 +51,7 @@ certain_map::certain_map! {
}
```

In this example, the Context struct has a single field: peer_addr of type PeerAddr. The #[empty(EmptyContext)] and #[full(FullContext)] attributes define the type aliases for the empty and full versions of the context, respectively.

It's important to note that the fields in the Context struct should correspond to the data that the `RoutingHandler` service expects to be available. In this case, the RoutingHandler requires the peer_addr information to be set in the context.
In this example, the Context struct has a single field: peer_addr of type `PeerAddr`. It's important to note that the fields in the Context struct should correspond to the data that the `RoutingHandler` service expects to be available. In this case, the RoutingHandler requires the peer_addr information to be set in the context.

By defining the Context using the certain_map crate, you can ensure that the necessary data is available at compile-time, preventing runtime errors and simplifying the implementation of your services.

Expand Down
4 changes: 1 addition & 3 deletions content/en/docs/monolake/Tutorial/service.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@ linkTitle: "Creating Service and Factory"
weight: 2
---

## Prerequisites
For a better
## Defining the Service and Factory
The `RoutingHandlerFactory` is responsible for creating and updating the `RoutingHandler` service instances. This factory implements the AsyncMakeService trait, allowing it to be used in a FactoryStack for service composition.

Expand Down Expand Up @@ -57,7 +55,7 @@ In this implementation, the RoutingHandlerFactory takes two parameters:

The AsyncMakeService implementation for the RoutingHandlerFactory defines how to create a new RoutingHandler instance. It first creates a Router from the configured RouteConfig instances, and then creates the RoutingHandler by calling the make_via_ref method in the inner service factory.

Note that in this case, we don't rely on any state from the previous RoutingHandler instance, as the routing configuration is fully defined by the RouteConfig instances. If the inner service factory had some stateful resources (like a connection pool) that needed to be preserved, we could clone those resources when creating the new RoutingHandler.
Note that in this case, we don't rely on any state from the previous RoutingHandler instance, as the routing configuration is fully defined by the RouteConfig instances. If the inner service factory had some stateful resources (like a connection pool) that needed to be preserved, we could clone those resources when creating the new RoutingHandler. For a more detailed example involving resource transfer, see [UpstreamHandler](https://github.com/cloudwego/monolake/blob/fd2cbe1a8708c379d6355b3cc979540ec49fdb4f/monolake-services/src/http/handlers/upstream.rs#L338), which involves transfer of a HTTP connection pool from the previous UpstreamHandler instance.

To integrate the `RoutingHandler` into a service stack, we can use the layer function provided by the `RoutingHandler` type:

Expand Down
1 change: 1 addition & 0 deletions content/en/docs/monolake/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
title: 'Monolake'
linkTitle: 'Monolake'
weight: 7
Description: "Monolake is a framework for developing high-performance networks services like proxies and gateways in Rust."
menu:
main:
weight: 6
Expand Down
117 changes: 117 additions & 0 deletions content/en/docs/monolake/cloudwego/Getting Started/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
---
title: "Getting Started"
linkTitle: "Getting Started"
weight: 2
keywords: ["Monolake", "Rust", "Proxy", "Getting Started"]
description: "This page provides a quick start guide for setting up and running Monolake"
---

## Prerequisites

- **Linux Kernel Support**: IO-uring requires a Linux kernel version that includes IO-uring support. Generally, kernel versions 5.1 and above provide the necessary support. Ensure that your target system has an appropriate kernel version installed. Monolake will fall back to epoll on Mac OS.
- **Rust nightly**: See the "Rust installation section" below

## Quick Start

This chapter will get you started with Monolake using a simple example config

### Rust installation

To download and install Rust, and set `rustup` to nightly, you can follow these instructions:

1. Download and install Rust by running the following command in your terminal:

```markup
$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```

This command will download the Rust installation script and initiate the installation process.

3. Close the terminal and open a new terminal window to ensure the changes take effect.

4. Set `rustup` to nightly by running the following command:

```markup
$ rustup default nightly
```

This command sets the default Rust toolchain to the nightly version.

5. Verify the installation and check the Rust version by running:

```markup
$ rustc --version
```

This command will display the installed Rust version, which should now be set to nightly.

### Setup an HTTP upstream server

1. **Install Nginx**: Install Nginx using the package manager of your Linux distribution.

2. **Start Nginx**: Start Nginx using the following command:

```shell
sudo service nginx start
```

3. **Configure Port**: Open the Nginx configuration file using a text editor:

```shell
sudo nano /etc/nginx/nginx.conf
```

Inside the file, locate the `http` block and add or modify the `listen` directive to use port 9080:

```nginx
http {
...
server {
listen 9080;
...
}
...
}
```

Save the changes and exit the text editor.

4. **Restart Nginx**: Restart Nginx for the changes to take effect:

```shell
sudo service nginx restart
```

### Build Monolake

1. Clone the monolake repository `git clone https://github.com/cloudwego/monolake.git`.
2. Build the binary:

```markup
$ cargo build --release
```
3. Generate the certificates
```markup
$ sh examples/gen_cert.sh
```
### Run the example
1. Make sure your endpoints are up and certificates are generated
2. Start the proxy

```markup
$ ./target/release/monolake -c examples/config.toml
```
3. Send a request to the socket based listener on server_basic.

```markup
$ curl -vvv http://127.0.0.1:9081/
```
4. Send a request to the server_tls.

```markup
$ curl --resolve gateway.monolake.rs:8081:127.0.0.1 --cacert examples/certs/rootCA.crt -vvv https://gateway.monolake.rs:8081/
```

### Detail configuration

Please check [Configuration](_config.md) for more details.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 8463d2b

Please sign in to comment.