Skip to content

Commit

Permalink
Merge pull request #73 from orkes-io/staging
Browse files Browse the repository at this point in the history 
fix blog links
  • Loading branch information
@dougsillars
dougsillars authored Mar 24, 2022
2 parents c0d4f1e + 538b768 commit 270f7f8
Show file tree
Hide file tree
Showing 11 changed files with 344 additions and 10 deletions.
2 changes: 1 addition & 1 deletion blog/2022-03-24-Why-is-microservice-orchestration-important-now.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,6 @@ More concretely, an orchestration workflow might be ideal for you if one or more

In this article, you learned about the importance of keeping your microservices autonomous and flexible. You also learned about the use of microservice orchestration to effectively communicate, visualize, identify, and resolve the challenges of managing microservices.

The downside of the process of building an orchestration system that implements all of the features your business requires is that it’s rather complex and time consuming. A purpose-built framework offering scalable and low-overhead orchestration—such as [Azure’s Durable Functions](https://docs.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-overview?tabs=csharp), [Uber’s Cadence](https://github.com/uber/cadence), [Netflix’s Conductor](https://netflix.github.io/conductor/), or [Camunda’s Zeebe](https://docs.camunda.io/docs/components/zeebe/zeebe-overview/)—can be a solution. Frameworks like these, however, still require significant work to configure and deploy for production use.
The downside of the process of building an orchestration system that implements all of the features your business requires is that it’s rather complex and time consuming. A purpose-built framework offering scalable and low-overhead orchestration— like [Netflix’s Conductor](https://netflix.github.io/conductor/), is an open source tool that fits this purpose.

[Orkes](https://orkes.io/) is a platform that offers a fully managed, cloud-hosted version of Conductor with tiered support. Orkes builds on top of Netflix Conductor to abstract out installation, tuning, patching, and managing high-performing Conductor clusters. Learn more about Orkes and get started for free within minutes [here](https://orkes.io/cloud/).
144 changes: 135 additions & 9 deletions docs/codelab/orderfulfillment.md
View file
Original file line number Diff line number Diff line change
@@ -1,20 +1,146 @@
# Order Fulfillment Codelab
# Order Fulfillment Codelab part 1

IN this codelab, we'll follow Bob (owner of the growing company Bob's Widgets), as he build s a workflow to ship his widgets to customers around the USA.
Bob's Widgets has just moved out of Bob's garage, and he's hired you to overhaul the shipping and fulfillment process. Everything today is manual and you think that building an automated workflow to efficiently get the orders out to the growing customer base is the best way to scale the system (and save your team from going insane).

Bob's Widgets has just moved out of Bob's garage, and he's excited to unveil online ordering and building an automated workflow to efficiently get the orders out to his growing customer base.
You've heard of Netflix Conductor, and have read about the flexibility of a microservice-based architecture. You are pretty sure that building small modular applications and wiring them into a Conductor workflow is the way to go. You know that the initial workflow will be really simple (these widgets don't ship themselves, you know!), but will quickly grow in complexity as the company grows.

Bob has heard of Netflix Conductor, and has read bout the flexibility of a microservice-based architecture, and hes keen to give that a shot - since he knows that the initial workflow will be really simple (these widgets don't ship themselves, you know!), but will quickly grow in complexity as the company grows.
In this codelab, we'll follow your work as you build out a Conductor workflow to automate fulfillment for Bob's Widgets.

In this codelab, we'll follow Bob as be builds out a Conductor workflow to handle his shipping.
## What you need to complete this codelab

You'll need an SDK to edit and run your local workers. To run conductor, we'll use the [Orkes Playground](https://play.orkes.io), which requires a free account to be created.

## A simple order Fulfillment workflow

Bob's in a bind. The business is taking off, and he needs to get orders shipped, and that leaves him just a little bit of time to build the workflow, but he knows he can take on small pieces at a time.
You're in a world of hurt. Bob's Widgets is taking off, and you are underwater to just get orders shipped. You don't have a lot of time to build any automation... but at the same time you know that without it, things are only going to get worse.

## Conductor basics

* **Workflow** A workflow defines the set of tasks - and the order that the tasks are run.
* **Tasks** Tasks are the job queues that are created by the workflow. Each task has a queue of jobs that can be run - and they are assigned to the workers.
* **Workers** Workers are your microservices. They may be Java, Go, Python - it doesn't matter. Each worker will poll the task queue for jobs, complete the job and return the results to the task.

You have one application (worker) at your disposal. Bob wrote a Java app that takes in an address, buys a shipping label, and sends the label to the printer in the shipping bay to be affixed to a widget box. With just a little bit of work, we can make this application a part of a Conductor workflow.

## Creating the workflow

In our initial workflow, we'll wire up Bob's Java app as a worker, and have a simple workflow with just one task and one worker. As we find time, we can add additional workers to the workflow, and improve the automation of the order fulfillment process.

In the [Orkes Playground](https://play.orkes.io), we'll want to create a workflow that incorporates the one application that is already written. Click "Workflow Definitions" on the left navigation. This page will list all of your workflows (and it may be empty right now). Click the "Define Workflow" button to begin creating your first workflow.

Workflows (in the playground) are JSON based. Here's an outline of the workflow you should create:

* Name: A name that describes your workflow
* Description: not required, but a nice way to help you recall what the workflow does.
* Tasks: This List isn array of all the tasks that will be inside your workflow. In this case, there is just one - ```widget_shipping_<uniqueId>```.

> Note: All tasks in the Playground must have a unique name, so pick an ID to replace ```<uniqueId>``` and reuse it across this codelab.
* outputParameters - These are the parameters that the workflow will return upon completion.

```json
{
"name": "Bobs_widget_fulfillment",
"description": "Shipping widgets right from Bob",
"version": 1,
"tasks": [
{
"name": "widget_shipping_<uniqueID>",
"taskReferenceName": "widget_shipping",
"inputParameters": {
"name": "${workflow.input.name}",
"street": "${workflow.input.street}",
"city": "${workflow.input.city}",
"state": "${workflow.input.state}",
"zip": "${workflow.input.zip}"
},
"type": "SIMPLE",
"decisionCases": {},
"defaultCase": [],
"forkTasks": [],
"startDelay": 0,
"joinOn": [],
"optional": false,
"defaultExclusiveJoinTask": [],
"asyncComplete": false,
"loopOver": []
}
],
"inputParameters": [],
"outputParameters": {
"address": "${widget_shipping.output.fullAddress}",
"tracking": "${widget_shipping.output.trackingNumber}"
},
"schemaVersion": 2,
"restartable": true,
"workflowStatusListenerEnabled": false,
"ownerEmail": "bob@bobswidgets.com",
"timeoutPolicy": "ALERT_ONLY",
"timeoutSeconds": 0,
"variables": {},
"inputTemplate": {}
}
```

Paste this JSON into the workflow editor, press "save" and "confirm save.' You've created a workflow! IN the diagram box, you should see a simple diagram of your workflow:

<p align="center"><img src="/content/img/codelab/of1_diagram.png" alt="version 1 diagram" width="400" style={{paddingBottom: 40, paddingTop: 40}} /></p>


## The task (in the workflow)

Inside the workflow, we've defined a task. Let's walk through the parameters of the task:
* inputParameters. These are the values that the task requires to complete the job. In this case, we are inputting the name, street, city, state and Zip - pretty standard shipping parameters (for the USA).
* These parameters are all variables: ```${workflow.input.zip} ``` means that the workflow input data will include an attribute with the key "zip" that corresponds to a zipcode.
* ```type```: This Task is a ```SIMPLE``` type. This means that there is an external worker that will process this task for us. It also means that we must create a task definition for the task.

There are a bunch of additional attributes for the task, but we can just ignore them at this point.

## Defining the task

To define the task, we need to create a definition. Click ```Task Definitions``` in the left navigation. This page lists all of your tasks. Click the ```Define Task``` button to create the task.

The Task definition is JSON formatted as well, and paste this to create your task (don't forget to update the uniqueId in the name!):

```JSON
{
"createdBy": "",
"name": "widget_shipping_<uniqueId>",
"retryCount": 3,
"timeoutSeconds": 30,
"inputKeys": [
"name",
"address",
"city",
"state",
"zip"
],
"outputKeys": [
"fullAddress",
"trackingNumber"
],
"timeoutPolicy": "TIME_OUT_WF",
"retryLogic": "FIXED",
"retryDelaySeconds": 30,
"responseTimeoutSeconds": 30,
"concurrentExecLimit": 100,
"inputTemplate": {},
"rateLimitPerFrequency": 50,
"rateLimitFrequencyInSeconds": 30,
"ownerEmail": "bob@bobswidgets.com",
"pollTimeoutSeconds": 30,
"backoffScaleFactor": 1
}
```

* Name: the unique name of your task
* inputKeys: Values that the task expects to have in order to run.
* outputKeys: Values that are returned to the workflow from the task

The remainder of the parameters define retries and delays. The values are standard values, and can be left as is.

He has a warehouse full of widgets, and he's built a Java shipping application that prints a label for one widget box. Let's look at this application, and then figure out a workflow for Bob's order fulfillment.
Press ```Save``` and ```confirm save``` to create this task. You've now created a task (and the requisite task queues) that your worker needs to run.

## The Shipping workflow
In the next step of the code lab, we'll create a worker, and connect it up to our workflow. Then, we'll begin working to better automate our order fulfillment system.

Bob's application is completely integrated with his shipping partner

90 changes: 90 additions & 0 deletions docs/codelab/orderfulfillment2.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Order Fulfillment Codelab part 2

You're running order fulfillment at Bob's Widgets, and it's totally manual. You're working with Conductor to create workflows and work to better automate the system.

In part 1 of this codelab, we created our first workflow and task - all built to run the ```widget_shipping``` worker that Bob gave you on your first day.

In part 2, we'll get the worker (the microservice) up and running, and connect the application with our remote Conductor server. Are you ready? Let's get started!

## Worker Code

The worker can be found in the [Orkesworkers GitHub repository](https://github.com/orkes-io/orkesworkers)). This repo has a number of workers for different demos. IN this case, we;re interested in widgetShipping.java (and also the OrkesWorkersApplication.java).

## Our First worker

Something is better than nothing, so let's take a look at the code, modified to make it a Conductor Worker. (you can find this worker in the [Orkesworkers GitHub repository](https://github.com/orkes-io/orkesworkers))

A Conductor worker has 2 parts:

* ```getTaskDefName``` that returns the name of the worker to Conductor (in this case "widget_shipping").
* ```TaskResult``` This is the Java that is actually executing the task for us. This is a demo, so there is some *magic* here. The tracking number is a 16 digit random number, and of course, the label never actually prints anywhere.



```java
@Component
public class widgetShipping implements Worker {
@Override
public String getTaskDefName() {
return "widget_shipping";
}

@Override
public TaskResult execute(Task task) {

TaskResult result = new TaskResult(task);
String name = (String) task.getInputData().get("name");
String street = (String) task.getInputData().get("street");
String city = (String) task.getInputData().get("city");
String state = (String) task.getInputData().get("state");
String zip = (String) task.getInputData().get("zip");
String fullAddress = name + ","+ street + ","+ city + ", "+ state + " " + zip;
try {
//generate 16 number shipping label
int eightdigit1 = (int)(Math.floor(Math.random()*100000000));
int eightdigit2 = (int)(Math.floor(Math.random()*100000000));

String tracking = Integer.toString(eightdigit1) + " " +Integer.toString(eightdigit2);

//magic that creates the label and prints it in the shipping bay

result.setStatus(TaskResult.Status.COMPLETED);
result.addOutputData("fullAddress", fullAddress);
result.addOutputData("trackingNumber", tracking);
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return result;
}
}

```

It isn't a huge application, but with Conductor, each app can be small and easy to work with. Let's get this application wired into a Conductor Workflow!

## Security

With Orkes Playground, you don't want anyone to be able to call your ```widget_shipping``` worker. Imagine coming in one morning with 1,000 fake labels printed. You don't want to deal with that.

There's application security built around your workflows in the Orkes Playground. For your workflow and task to run your worker, you'll need a key & secret from the playground embedded in your worker. This ensures that only the applications provisioned to create shipping labels will actually create shipping labels.

### Key and secret

To create your key & secret, click ```Applications``` in the left nav. Click ```Create Application``` to create the application. We'll name our new application "Bobs_orders." There are two tables (both empty). At the of the pag click the ```Create Access Key``` button. This will generate an Id and Secret. Record both of these values (if you forget, you can create additional Id/Secrets). We'll use these in the worker application to allow for the connection.

These are added to the ```application_properties``` file in the orkesworkers application. The OrkesWorkersApplication uses these values to create a JWT authentication token.

### Workflow and Task Permissions

In the 2nd table, click the + to begin adding workflows and tasks to the application. You'll want to add:

* Workflow: ```Bobs_widget_fulfillment``` permissions: Execute
* Task: ```widget_shipping_<uniqueId>``` permissionsL Execute


Now we've given the security connections between the workflow and task in Conductor to talk with the worker!

You can run the OrkesWOrkerApplication,java in your IDE. You'll see a lot of errors in the terminal - as only the widgetSSgipping.java has permission to connect to the Playground. You can remove the other workers from the folder to remove these error messages.

Now we're ready to run our first workflow!
48 changes: 48 additions & 0 deletions docs/codelab/orderfulfillment3.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Order Fulfillment Codelab part 3

You're running order fulfillment at Bob's Widgets, and it's totally manual. You're working with Conductor to create workflows and work to better automate the system.

In part 1 of this codelab, we created our first workflow and task - all built to run the ```widget_shipping``` worker that Bob gave you on your first day.

In part 2, we got the worker (the microservice) up and running, and connected the application with our remote Conductor server.

In part 3, we'll actually run the worker, and see Conductor in action! We'll also see some of the limitations of the current workflow, and create V2 of the workflow.

Let's get started by running the workflow!

## Running the workflow

To run our workflow, click the ```Run Workflow``` box in the left nav. This will open a panel looking for parameters. Choose the workflow named ```Bobs_widget_fulfillment``` and as input, add an address. Here's an example address:

```json
{
"name": "bob",
"street":"123 Main St.",
"city": "Anytown",
"state":"ME",
"zip":"00234"
}
```

<p align="center"><img src="/content/img/codelab/of3_runworkflow.png" alt="version 1 run workflow" width="800" style={{paddingBottom: 40, paddingTop: 40}} /></p>

Press ```Run Workflow``` to get started!

## Seeing the workflow in action

Once you press Run workflow, a workflowId will appear below the form. CLick this to see your workflow run through the tasks.

Click the workflowId, and byu the time the new page has loaded, your workflow will likely be completed:

<p align="center"><img src="/content/img/codelab/of3_workflowresults.png" alt="version 1 results" width="800" style={{paddingBottom: 40, paddingTop: 40}} /></p>

There is a lot of information about your workflow on this page, and in this short video, we walk through the various screens:

<p align="center"><iframe width="560" height="315" src="https://www.youtube.com/embed/agL-WHXbfX4" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe></p>

## Seeing the Limitations

Once ou run the workflow, you realize that this works for one shipping label. But most of your customers buy in quantities of 5. When you ask Bob, he says "Just re-run the app 'x' times for each order." Knowing this is suboptimal, you begin planning your first update to the Conductor workflow - generating one shipping label *per item ordered*.

Read on to the next section where we learn about creating parallel workflows - one for each order - using a FORK (and more accurately a DYNAMIC_FORK) to create the correct number of shipping labels.

Loading

0 comments on commit 270f7f8

Please sign in to comment.