Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A38: Admin Interface API #218

Merged
merged 7 commits into from
Mar 5, 2021
Merged
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 30 additions & 43 deletions A38-admin-interface-api.md
Original file line number Diff line number Diff line change
@@ -1,57 +1,36 @@
Admin Interface API
----
* Author(s): lidiz
* Approver: markroth
* Status: Draft
* Approver: markdroth
* Status: In-Review
* Implemented in: C++, Golang, Java
* Last updated: 2021-01-27
* Discussion at: TBD
* Last updated: 2021-02-05
* Discussion at: https://groups.google.com/g/grpc-io/c/0B2f9ha5HoM

## Abstract

This proposal describes a convenience API in each gRPC language to improve the usability of creating a gRPC server with admin services.
This proposal describes a convenience API in each gRPC language to improve the usability of creating a gRPC server with admin services to expose states in the gRPC library.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: s/convenience/convenient/ ?



## Background

gRPC Admin Interface is part of the Proxyless Service Mesh Observability project which includes new gRPC admin services and a CLI tool (yet to release). The admin services are responsible for exposing states of the gRPC library via gRPC protocol, and the CLI tool will help users to better interpret the fetched data from the gRPC application. Currently, the existing admin server should at least expose two admin services:
### Services Exposing gRPC Internal States
lidizheng marked this conversation as resolved.
Show resolved Hide resolved

* Channelz: the channel tracing library that collects stats about a channel’s traffic, underlying sockets, and credentials ([proto definition](https://github.com/grpc/grpc/blob/master/src/proto/grpc/channelz/channelz.proto));
* [Client Status Discovery Service](https://github.com/envoyproxy/envoy/blob/main/api/envoy/service/status/v3/csds.proto): the xDS config dump library that dumps the accepted xDS configuration with config status;

However, the number of admin services might increase in the future. Solutions with no code changes, like env, or config files, are not desired (see rationale section below). On the other hand, as a library, gRPC would like to promote the usage of admin services. So, there is a need for an ideally one LOC to create and start an admin server.

In addition, by providing new APIs, we abstract away the individual services with the concept admin services. We can also achieve **dynamic admin services selection**, for example, we **only add CSDS service if the xDS package is pulled in**. In this way, we won’t force users to take unnecessary dependencies.


### Types of Services
gRPC provides a set of official gRPC services that serve gRPC internal states. This proposal will focus on the ones that expose the state of the entire gRPC library, which is agnostic to which gRPC server they were bound to:
markdroth marked this conversation as resolved.
Show resolved Hide resolved

To help communicate the scenarios, we can define all services into several types, from the view of administrative functions:

* **Primary service**: the user-provided services that handle the application logic;
* **Binary-global service**: the services that expose the state of the entire gRPC library, and they serve the same information regardless of which server they have been registered (e.g., Channelz, CSDS);
* **Per-server service**: the services that distribute the state of other registered services on the same server (e.g., Health, Reflection).

The binary-global services contain sensitive information about traffic, configuration, even security, hence we hope users can set up the services securely in our recommended usage or the upcoming new API.

The recommended way of securing the admin service port is binding it to either `localhost` or using a `UDS` port.
* Channelz: the channel tracing library that collects stats about a channel’s traffic, underlying sockets, and credentials ([gRFC A14](https://github.com/grpc/proposal/blob/master/A14-channelz.md));
* [Client Status Discovery Service](https://github.com/envoyproxy/envoy/blob/main/api/envoy/service/status/v3/csds.proto): the xDS config dump library that dumps the accepted xDS configuration with config status;


## Proposal

There are several different options for the convenience API design to help create a more usable admin interface API. Each option is a tradeoff between simplicity to use and flexibility. The simplicity to use is defined by lines of code to create a serving gRPC server with admin services, and the additional cognitive cost to learn the new API or manage dependencies.
The admin interface API is a set of API that registers available admin services to a given gRPC server. There will be more gRPC services boosting debuggability of gRPC down the road. With this API, applications will automatically get the update as they pulling in the latest gRPC library.

The flexibility is defined by how can users provide additional server creation options. To date, users can specify one or more **listening addresses, credentials, TCP/HTTP config, codec, interceptors and service implementations** during the server creation process. Not to mention the potential impact on the threading model during the start of the server. If we want to bundle as many options as possible into one line of code, we will inevitably sacrifice some flexibility.
There are several different options for the convenience API design. Each option is a tradeoff between simplicity to use and flexibility. The simplicity to use is defined by lines of code to create a serving gRPC server with admin services, and the additional cognitive cost to learn the new API or manage dependencies. The flexibility is defined by how can users provide additional server creation options. To date, users can specify one or more **listening addresses, credentials, TCP/HTTP config, codec, interceptors and service implementations** during the server creation process. Not to mention the potential impact on the threading model during the start of the server. If we want to bundle as many options as possible into one line of code, we will inevitably sacrifice some flexibility. For other alternative designs, see section below.
markdroth marked this conversation as resolved.
Show resolved Hide resolved

Options to create an admin server are ordered from the most flexible to the least:
This proposal recommend the admin interface API design to be **a function that combines admin services from different packages**.

1. The control group: using existing API to create the server;
2. A function that combines admin services from different packages; **<-RECOMMENDED BY THIS PROPOSAL**
3. A method of the gRPC server class that adds admin services;
4. A function that creates the gRPC server object bound with admin services;
5. A function that creates and starts the gRPC server with admin services.

This proposal will recommend option 2 as the admin interface API (for other alternatives, see rationale section below). The usage snippet of admin interface API in each language:
Here are the usage snippets of admin interface API in each language:


### Java
markdroth marked this conversation as resolved.
Show resolved Hide resolved
Expand Down Expand Up @@ -124,7 +103,7 @@ if err := grpcServer.Serve(lis); err != nil {
```cpp
grpc::ServerBuilder builder;
builder.RegisterService(grpc::ChannelzService());
builder.RegisterService(grpc::experimental::CsdsService());
builder.RegisterService(grpc::CsdsService());
builder.AddListeningPort(":50051", grpc::ServerCredentials(...));
std::unique_ptr<grpc::Server> server(builder.BuildAndStart());
```
Expand All @@ -136,11 +115,15 @@ The admin services will use the gRPC C++ Sync API to offload the responsibility

```cpp
grpc::ServerBuilder builder;
grpc::experimental::AddAdminServices(&builder);
grpc::AddAdminServices(&builder);
builder.AddListeningPort(":50051", grpc::ServerCredentials(...));
std::unique_ptr<grpc::Server> server(builder.BuildAndStart());
```

### Dependency-based Admin Services List

In addition, by providing new APIs, we abstract away the individual services with the concept admin services. We can also achieve **dynamic admin services selection**, for example, we **only add CSDS service if the xDS package is pulled in as a dependency of the application**. In this way, the application can avoid adding unnecessary dependencies.
lidizheng marked this conversation as resolved.
Show resolved Hide resolved


## Rationale

Expand All @@ -154,9 +137,9 @@ In other softwares, like Envoy, it’s common to see a pattern that specifying a

### Alternative Admin Interface API Designs

Here are the usage snippets for other API options for Java:
Here are the usage snippets for other API Designs in Java:

Option 3: Server class method
#### A method of the gRPC server class that adds admin services
```java
server = ServerBuilder.forPort(50051)
.useTransportSecurity(certChainFile, privateKeyFile)
Expand All @@ -166,7 +149,10 @@ server = ServerBuilder.forPort(50051)
server.awaitTermination();
```

Option 4: Create admin server
The drawback for this option is dependency management. To date, the gRPC official admin services Channelz/Reflection/Health are distributed as separated package to keep the main package focus as a network library. If we apply this option, then we will void the separate effort.
markdroth marked this conversation as resolved.
Show resolved Hide resolved


#### A function that creates the gRPC server object bound with admin services;
```java
server = AdminServerBuilder.forPort(50051)
.useTransportSecurity(certChainFile, privateKeyFile)
Expand All @@ -175,15 +161,16 @@ server = AdminServerBuilder.forPort(50051)
server.awaitTermination();
```

Option 5: Create and start serving
For this option, the `AdminServerBuilder` allows some flexibility, but it conflicts with `XdsServerBuilder` if users want to enable xDS, and it would be repetitive if we offers `AdminXdsServerBuilder` on the side.


#### A function that creates and starts the gRPC server with admin services.
```java
server = admin.StartAdminServer(50051, certChainFile, privateKeyFile)
server.awaitTermination();
```

For option 4/5, they well achieved the ultimate goal of reducing the line-of-code changes, but they also terrible at satisfying any need for extension. For option 5, advanced users might want to play with the admin service like adding interceptors or change credential types, but even tiny amount of change means fallback to option 1 (existing API). For option 4, the `AdminServerBuilder` allows some flexibility, but it conflicts with `XdsServerBuilder` if users want to enable xDS, and it would be repetitive if we offers `AdminXdsServerBuilder` on the side.

The drawback for option 3 is dependency management. To date, the gRPC official admin services Channelz/Reflection/Health are distributed as separated package to keep the main package focus as a network library. If we apply option 3, then we will void the separate effort.
For this option, advanced users might want to play with the admin service like adding interceptors or change credential types, but even tiny amount of change means fallback to use existing API.


## Future Work
Expand Down