Skip to content

Added Service discovery docs #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 14, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions docs/structure.md → docs/contributing/structure.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,3 @@ This view is usually used by Kubernetes, Eureka and other systems to check if ou

### pyms/logger
Print logger in JSON format to send to server like Elasticsearch. Inject span traces in logger.

### pyms/tracer
Create an injector `flask_opentracing.FlaskTracer` to use in our projects.
Binary file added docs/imgs/consul_example.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
12 changes: 6 additions & 6 deletions docs/overrides/home.html
Original file line number Diff line number Diff line change
Expand Up @@ -11,21 +11,21 @@
<div class="tx-hero__content">
<br><br>
<h1>Python Microservices for the real world </h1>
<p>When we started to create a microservice with no previous knowledge, we started looking for tutorials, guides, best practices, but we found
nothing to create professional projects. Most articles say: </p>
<p>Without previous knowledge, we started to create a microservice. We started looking for tutorials,
guides and best practices, but we found nothing to create professional projects. Most articles say: </p>
<ul>
<li>"Install flask"</li>
<li>"Create routes"</li>
<li>(Sometimes) "Create a swagger specs"</li>
<li>"TA-DA! you have a microservice"</li>
</ul>
<p>But... what happens when we want our configuration out of the code, such as a Kubernetes configmap? what happens with transactionality?
<p>But... what happens when we want to use our configuration out of the code, such as a Kubernetes configmap? what happens with transactionality?
If we have many microservices, what happens with traces?.</p>

<p>There are many problems around Python and microservices and we couldn't find anyone to give us a solution.</p>
<p>There are many problems around Python and microservices and we couldn't find anyone to give us any solutions.</p>

<p>We start creating these projects to try to solve all the problems we have found in our professional lives about
microservices architecture.</p>
<p>We start creating these projects to try to solve all the problems we have found in our careers about
microservices architecture.</p>

<p>Nowadays, is not perfect and we have a looong roadmap, but we hope this library could help other felas and friends ;)</p>
<br>
Expand Down
File renamed without changes.
37 changes: 32 additions & 5 deletions docs/services.md → docs/services/services.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Services

Services are libraries, resources and extensions added to the Microservice in the configuration file.
These services are created as an attribute of the [Microservice class](ms_class.md) to use in the code.
These services are created as an attribute of the [Microservice class](../ms_class.md) to use in the code.

To add a service check the [configuration section](configuration.md).
To add a service check the [configuration section](../configuration.md).

You can declare a service but **activate/deactivate** with the keyword `enabled`, like so:

Expand Down Expand Up @@ -84,6 +84,8 @@ pyms:
Add traces to all executions with [opentracing](https://github.com/opentracing-contrib/python-flask). This service
solves the problem of [distributed tracing](https://microservices.io/patterns/observability/distributed-tracing.html)

See [Propagate traces tutorial](/tutorials/tutorial_propagate_traces) in this documentation for more information.

### Installation

You must install `pyms` with `pip install pyms[all]` or `pip install pyms[trace]`
Expand Down Expand Up @@ -111,7 +113,7 @@ pyms:
Adds [Prometheus](https://prometheus.io/) metrics using the [Prometheus Client
Library](https://github.com/prometheus/client_python).

The folowing metrics are currently available:
The following metrics are currently available:

- Incoming requests latency as a histogram
- Incoming requests number as a counter, divided by HTTP method, endpoint and
Expand All @@ -128,12 +130,37 @@ You must install `pyms` with `pip install py-ms[all]` or `pip install py-ms[metr
```yaml
pyms:
services:
metrics: true
metrics: true
```

This will add the endpoint `/metrics` to your microservice, which will expose
the metrics.

## Service discovery
Adds [Consul](https://www.consul.io/) as service discovery using the [Consul Client
Library](https://github.com/python-microservices/consulate) or create your own service discovery

See [Service Ddiscovery section](services_discovery.md) in this documentation for more information.

### Installation

You must install `pyms` with `pip install py-ms[all]` or `pip install py-ms[consul]`

### Example

```yaml
pyms:
services:
service_discovery:
service: consul
host: localhost
port: 8500
autoregister: true
```

This will add the endpoint `/metrics` to your microservice, which will expose
the metrics.

## How to contrib:

[See tutorial](tutorial_create_services.md)
[See tutorial](/tutorials/tutorial_create_services)
91 changes: 91 additions & 0 deletions docs/services/services_discovery.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Service Discovery

PyMS support at this moment this service discovery

- Consul
- Eureka (TODO)
- Kubernetes (TODO)


## Consul
This service use [Consul Client
Library](https://github.com/python-microservices/consulate) to connect to [Consul](https://www.consul.io/)

```yaml
pyms:
services:
service_discovery:
service: "consul"
host: "localhost"
autoregister: true
config:
DEBUG: true
TESTING: false
APP_NAME: "Python Microservice"
APPLICATION_ROOT: ""
```

Check your local configuration with Docker. Run:

```shell
docker run -d --net=host -e 'CONSUL_LOCAL_CONFIG={"leave_on_terminate": true, "enable_script_checks": true}' consul
```

Open `http://localhost:8500/` to check if Consul is up. If it's ok, when you run your microservice with
`autoregister: true` parameter, you will see in Consul your microservice like this image:

![Consul](/imgs/consul_example.png)

## Consulate client

You can use all [consulate](https://github.com/python-microservices/consulate) options with
`ms.service_discovery.client` or `current_app.ms.service_discovery.client`. In example:

```python
ms.service_discovery.client.agent.checks()
```

See [official docs](https://consulate.readthedocs.io/) for more information.

## Create your own Service Discovery

Instead of define a service keyword, you can point to a class in your proyect that inherit from ServiceDiscoveryBase
(see example above)

```yaml
pyms:
services:
service_discovery:
service: "myproject.servicesdiscovery_file.MySvcDiscovery"
host: "localhost"
autoregister: true
config:
DEBUG: true
TESTING: false
APP_NAME: "Python Microservice"
APPLICATION_ROOT: ""
```


Under `myproject/servicesdiscovery_file.py`:

```python
from pyms.flask.services.service_discovery import ServiceDiscoveryBase


class ServiceDiscoveryConsulBasic(ServiceDiscoveryBase):
def register_service(self, id_app, host, port, healtcheck_url, app_name):
headers = {"Content-Type": "application/json; charset=utf-8"}
data = {
"id": app_name + "-" + id_app,
"name": app_name,
"check": {"name": "ping check", "http": healtcheck_url, "interval": "30s", "status": "passing"},
}
response = requests.put(
"http://{host}:{port}/v1/agent/service/register".format(host=host, port=port),
data=json.dumps(data),
headers=headers,
)
if response.status_code != 200:
raise Exception(response.content)
```
6 changes: 3 additions & 3 deletions docs/tutorials/tutorial_propagate_traces.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Each service handles a request by performing one or more operations, e.g. databa

PyMS injects a unique request ID with [opentracing](https://github.com/opentracing-contrib/python-flask) and
passes the external request id to all services that are involved in handling the current request with the
[service request](services.md)
[service request](/services/services)

## 1. Simple Trace

Expand Down Expand Up @@ -133,7 +133,7 @@ curl 'http://localhost:5001/'
}
```

![Terminal](imgs/multiple-ms.png)
![Terminal](/imgs/multiple-ms.png)

The second MS will print these logs:

Expand Down Expand Up @@ -184,7 +184,7 @@ As you can see, both microservices have `"trace": "bb785b88d0456d69"`.

You can see the flow of these requests in this diagram:

![Distributed tracing](imgs/PyMSdistributedtracing.png)
![Distributed tracing](/imgs/PyMSdistributedtracing.png)

## Code

Expand Down
12 changes: 7 additions & 5 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,12 @@ nav:
- Installation: installation.md
- Quickstart: quickstart.md
- Configuration: configuration.md
- Services: services.md
- Microservice class: ms_class.md
- Routing: routing.md
- Services:
- List of services: services/services.md
- Routing: services/routing.md
- Service Discovery: services/services_discovery.md
- Encrypt/Decrypt Configuration: encrypt_decryt_configuration.md
- PyMS structure: structure.md
- Command line: command_line.md
- Introduction to microservices:
- Introduction to microservices (ENG): microservices.md
Expand All @@ -35,11 +36,12 @@ nav:
- Run in Kubernetes: scaffold/runinkubernetes.md
- Template:
- Index: template/index.md
- Contributing:
- Index: contributing/index.md
- PyMS structure: contributing/structure.md
- Articles and events:
- External Links and Articles: links_articles.md
- HacktoberfestES: hacktoberfest_es_2020.md
- Contributing:
- Index: contributing/index.md

markdown_extensions:
- admonition: null
Expand Down