This repository will provision an environment that may be used as a Lab to build an end to end scenario that does the following:
- Place an online order on an eCommerce web site
- Place an order with a chatbot running on an Azure Bot Service web site
- Query and create orders on our legacy on-prem Order system
- Detect the language that the customer engages with Bot Service on
- Mail the user with order confirmations and follow ups in their preferred language
This solution brings together Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Software as a Service (SaaS) and Serverless components on Microsoft Azure to build a realistic end to end scenario to nurture customers. Furthermore, the democratization of AI is tied in by incorporating Cognitive Services to perform language detection and dynamic translation.
This solution will allow a customer to place an order for a coffee in a multi-channel online store, via website or a chat bot. The language that the user engages with the chat bot on will be determined and stored as personal preferences against the user's record in a locked down legacy database. Future correspondence and order confirmations will be in the customer's preferred communication language.
- How legacy lift and shift applications on IaaS can be incorporated into modern solutions to quickly derive value from higher value services in the cloud.
- How non-API enabled legacy workloads can be modernised.
- The ease with which On-premise, public and private components may be brought together to build workloads that bring business value
- The meshing of IaaS, PaaS, SaaS, Serverless and AI with tools that are accessible to non-developers
- OSS workloads running on Azure
- How Logic Apps can parse and apply JSON schemas to mixed datasources on the fly
The following technology components are used in this solution:
- eCommerce online store built with Java running on Azure App Services (PaaS)
- Ubuntu running a legacy mysql database (IaaS)
- Azure networking to isolate legacy workloads (IaaS)
- Azure virtual network Gateway to provide point to site connectivity
- Serverless components to serve as a proxy between the IaaS legacy database and the PaaS layer (Serverless)
- Azure logic apps to provide serverless integration that is accessible to non-developers (Serveless)
- Microsoft Cognitive Services to detect language and translation in real time
- Azure Resource Manager templates to automate the provisioning and inflation of a full environment
This solution will install and configure all of the components required to build the end to end Personalisation scenario. The Lab attendees just need to apply a few configuration changes and wire everything together in a Logic App.
For this Lab you will require:
- A cognitive services trial account key, get it here - https://www.microsoft.com/cognitive-services/en-us/sign-up
- A Gmail account for sending emails, get it here - https://accounts.google.com/SignUp?service=mail&continue=http%3A%2F%2Fmail.google.com%2Fmail%2Fe-11-14a952576b5f2ffe90ec0dd9823744e0-46a962fbb7947b68059374ddde7f29b5490a6b4d
- Install Postman for REST API troubleshooting, get it here - https://www.getpostman.com
- If using Windows 10 get Bash for Windows - https://msdn.microsoft.com/en-us/commandline/wsl/install_guide or putty if on an older version - http://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html
1. Provisioning the components: Select Deploy to Azure to deploy to your Azure instance that you are currently logged in to.
Select or create a Resource Group to deploy to and the only parameter you need to change is the Deployment Name - give it any name of 12 characters or less as it will be used to generate a hash to ensure your site names are unique. Make a note of the Deployment Name and Resource Group you have deployed to as we will need them again.
Note, you can always get the parameters for a deployment by clicking on Resource Groups --> [Resource Group] --> Deployments --> [Deployment] - here you can see status and parameters.
This will take roughly 30 minutes as this will provision:
- Two VNETs
- A virtual network Gateway
- An Ubuntu VM and place in inside the VNET isolated with NSGs
- A Bot Service chat bot running on App Services
- An App Service Web app (eCommerce site) and deploy a Java shopping experience on it
- An App Service serverless function as part of a App Service plan so that it can be connected to the virtual network Gateway
- Storage accounts to house the VM VHD, the Function logging and the App Service logging
Make sure you have the same Deployment Name as for the first deploy and ensure you deploy to the same resource group, see image below:
See the image below:
This will take roughly 5 minutes as this will:
- Deploy a number node.js functions to connect directly to a remote mysql instance
- A node.js chat bot shopping experience
We could deploy this script as a custom script extension on the VM but that will complicate troubleshooting in a lab scenario so we will manually connect to the machine and run the build script, it is a single install script that will set up everything required.
Navigate to your VM, the default name will be comvmmmm[hash] and navigate to the Overview blade and copy the value in the field Public IP Address/DNS label, see below:
We will now ssh onto the machine using Bash for Windows on Windows 10, or putty or just plain old terminal on a mac or Linux.
- Type ssh MiniCADAdmin@[pasted ip address - without value '/none' on the end) e.g. ssh MiniCADAdmin@12.34.56.78 and press enter - see below:
- Select yes to the message "Are you sure you want to continue connecting"
- Type in password MiniCADAdmin123 - note this is hardcoded in the deploy
- Paste the following in the command line:
git clone https://github.com/shanepeckham/CADScenario_Personalisation.git - Now type
cd CADScenario_Personalisation - Now type
bash installVM.sh - Upon completion you will see a screen similar to that below, with the final status 'Active Internet Connections'
Here you will now see that our legacy database is running IP 10.1.0.4 port 3306 within our VNET.
Navigate to the Function component provisioned within Azure, its name will be generated by default with the following format commSQL[hash].
If all has provisioned correctly you should see that 4 functions have been created, namely:
- AddNewCustomerOrder - this will upsert a customer and create an order
- GetCustomer - this will get a customer record and requires the customerId as input
- GetOrder - this will get an order record and requires the orderId as input
- GetOrderByCustomerId - this will get a customer's order by the customerId as input
Note what these functions do as they will be required in the hands-on Lab component.
Click on 'Function app settings' on the bottom left of the screen, see below:
This will navigate you to settings page, now select 'Go To App Service Settings' see below:
Now select 'Networking', see below:
Now select 'Setup' in the VNET Integration section, see below:
Now select the VNET connection that we provisioned, it will have the default name legacycommvnet[hash]. If you have many connections just look at the top navigation history and you will see the correct hash prefix, see below:
This will take around a minute and once complete you should see the legacycommvnet[hash] virtual network as a connection, see below:
Now we can go and check whether our Serverless Function proxy can connect to the legacy database running on 10.1.0.4:3306. Click on 'Console', see below:
In the console type tcpping 10.1.0.4:3306 You should get a successful connection which means our Serverless proxy can now retrieve and update data from the legacy database running on the isolated VNET on behalf of our chat bot and eCommerce web site, see below:
Now we can go test a method to check whether all is working as planned. Navigate back to the code view of the Function and select method 'GetCustomer'. Expand the menu on the right of the page to gain access to the test pane, see below:
Select the Test item and enter the following in the 'Request Body' section and select run, see below:
{ "customerId": "1" }
You should see the result "bobby@turtlenecksweater.com" in the logs below if successful, you might get an error as illustrated below upon first invocation but if you run it again the error should not appear again.
2017-03-29T12:45:49.621 Retrieving a single customer
2017-03-29T12:45:54.595 SELECT * FROM customers where customerId = "1"
2017-03-29T12:45:54.627 Function completed (Failure, Id=88f13e38-3867-4f0c-a6aa-d29e1330b309)
2017-03-29T12:45:54.646 Script for function 'GetCustomer' changed. Reloading.
2017-03-29T12:45:54.783 RowDataPacket {
customerId: 1,
emailAddress: 'bobby@turtlenecksweater.com',
preferredLanguage: 'English' }
2017-03-29T12:45:54.955 Exception while executing function: Functions.GetCustomer. mscorlib: ReferenceError: res is not defined
at module.exports (D:\home\site\wwwroot\GetCustomer\index.js:28:24)
at D:\Program Files (x86)\SiteExtensions\Functions\1.0.10841\bin\azurefunctions\functions.js:93:24.
When you run it again you should see the following, if successful:
Navigate back to your Resource Group and select your provisioned eCommerce website, it will have a default name of commcoffee[hash]. Now select Browse, see below:
This will open a new page and start up your Java eCommerce site. Have a look around, you can order 1 of 4 coffees but we have not yet wired up the web site to the ordering process, we will do this in the hands-on Lab component. See below:
Navigate back to your Resource Group and select your provisioned Bot Service App Service App, it will have a default name of commcombot3[hash]. Now select it and you will be navigated to the setup screen where you will register the Bot App Id so that it can be added to other channels if required, see below:
Click 'Create Microsoft App Id and Password'. This will open up a new window where you will register your Bot App Id. Select 'Generate an app password to continue', see below:
This will open up a popup with your password in, copy this value for immediate reuse. Note, it only appears this once. Click 'Finish and go back to Bot Framework'. Paste your password in the entry box.
Now select 'NodeJS' in the Choose a Language section and select 'Basic' in the Choose a Template section (at the time of writing this a necessary step even though our code will overwrite these settings) and click 'Create Bot', see below:
This will take roughly 2-3 minutes, once complete you will be navigated to the code view of the Bot. If all has provisioned correctly you will see the status 'Edit continuous integration'. Click on settings to check the status of the code deploy, see below:
Now click on the 'Edit' dropdown on the right of the screen in the Continuous Integration section, see below:
This will open the Continuous Integration menu. Here you can see the status of the code deploy, see below. Select the Configure Continuous Integration button.
This will open a blade on the right which will display the status of the deployment source and latest commit, see below:
We want to get the customer's details, find their last associated case and then check the feedback against it.
See the diagram below for the simplistic data model to help you query the right data.
Create a HTTP Request Step, click save - you will receive an endpoint upon save.
Select the Request step, see below:
Once you save the Logic app you will get and endpoint, you can now invoke your logic app with Postman - add the URL and select POST. Ensure you have set the Header "Content-Type" with value "application/json". Select body, select "raw" and enter the follow value for your body content:
{
"APIMKey": "[Your APIM Key goes here]",
"id": 1
}
Now add a step to include an API Management API - select your API "Contact List API". You will want to select the method GET for contacts/{id}
You will need to navigate to the code view to be able to select the json fields that will be posted as part of the body. Note, you can select the values you are posting in the body using javascript object notation.
Your code view should look like this:
"QueryContactsById": {
"inputs": {
"api": {
"id": "/subscriptions/1b987fd6-b38e-40a1-bca8-4f67e6272c12/resourceGroups/[ResourceGroup]/providers/Microsoft.ApiManagement/service/cadapimxdb3o43h6p7bq/apis/58cd4c45dc78ac0f84da1287"
},
"method": "get",
"pathTemplate": {
"parameters": {
"id": "@{encodeURIComponent(triggerBody()['id'])}"
},
"template": "/Contacts/contacts/{id}"
},
"subscriptionKey": "@{triggerBody()['APIMKey']}"
},
"runAfter": {},
"type": "ApiManagement"
}
Navigating back to the designer should show your values resolved like below:
Now you need to query the Legacy Ticket API (Contacts Case List API) which is inside the isolated network to get the last case for that customer and retrieve the customer's feedback on the case. From the demo model hint above you want to use the caseNum field from the QueryContactsById to query the Legacy Ticket API field primary key field CaseId.
Add an API Management API step, select the Contact Case List API and once again query by Id, which in this CaseId which maps to the caseNum output from the previous step and add the API Management subscription key.
Ensure you select the correct outputs from the previous step(s) as inputs to this step, see below:
Now if you applied the logic from the previous step to select the correct outut field values (which would be a sensible approach) you will probably get the following error:
This is due to us outputting an array but not specifying which record/index we want to use. Change the following line from
"id": "@{encodeURIComponent(body('QueryContactsById')?['caseNum'])}"
to use the first value of the array, namely the 0 index:
"id": "@{encodeURIComponent(body('QueryContactsById')[0]['caseNum'])}"
Here is what your code view should look like for this step:
"QueryCasesById": {
"inputs": {
"api": {
"id": "/subscriptions/1b987fd6-b38e-40a1-bca8-4f67e6272c12/resourceGroups/[ResourceGroup]/providers/Microsoft.ApiManagement/service/cadapimxdb3o43h6p7bq/apis/58cd5516dc78ac0f84da1289"
},
"method": "get",
"pathTemplate": {
"parameters": {
"id": "@{encodeURIComponent(body('QueryContactsById')[0]['caseNum'])}"
},
"template": "/LegacyAPI/contacts/{id}"
},
"subscriptionKey": "@{triggerBody()['APIMKey']}"
},
"runAfter": {
"QueryContactsById": [
"Succeeded"
]
},
"type": "ApiManagement"
},
Now we want to add our Cognitive Services 'Detect Sentiment' (note you will need to have your key ready that you got when you signed up for the Text Analytics preview as part of the pre-requisites) step so that we can analyse the sentiment of the Ticket Feedback:
We now need to make sure we send the correct output from the QueryCasesById step to the Detect Sentiment step, use the javascript dot notation approach again with an index value. Your steps should look like this:
Your code view should look like this:
"Detect_Sentiment": {
"inputs": {
"body": {
"text": "@{body('QueryCasesById')?['last_feedback']}"
},
"host": {
"api": {
"runtimeUrl": "https://logic-apis-northeurope.azure-apim.net/apim/cognitiveservicestextanalytics"
},
"connection": {
"name": "@parameters('$connections')['cognitiveservicestextanalytics']['connectionId']"
}
},
"method": "post",
"path": "/sentiment"
},
"runAfter": {
"QueryCasesById": [
"Succeeded"
]
},
"type": "ApiConnection"
},
Now we want to add a condition to check the sentiment, if the probability outcome is less than 0.5, then it negative sentiment and therefore qualifies for our discount coupon.
Your condition should look like this:
With the following in code view:
"expression": "@less(body('Detect_Sentiment')?['score'], 0.5)",
"runAfter": {
"Detect_Sentiment": [
"Succeeded"
]
},
"type": "If"
Now you can call the GenerateCoupon function if the condition is met, pass in the name of the user that you want to generate the digital coupon for:
With the following in the code view:
"GenerateCoupon": {
"inputs": {
"body": {
"name": "@body('QueryContactsById')[0]['name']"
},
"function": {
"id": "/subscriptions/1b987fd6-b38e-40a1-bca8-4f67e6272c12/resourceGroups/NewLoyaltyPlan/providers/Microsoft.Web/sites/CADFuncxdb3o43h6p7bq/functions/GenerateCoupon"
}
},
"runAfter": {},
"type": "Function"
}
},
Now we want to send an email to every receipient to inform them that they can download a digital coupon which we have generated for them.
Your email step should look like this:
With the following code view:
"Send_email": {
"inputs": {
"body": {
"Body": "Please accept this coupon: @{body('GenerateCoupon')}",
"Subject": "We heard that you were not happy",
"To": "@{body('QueryContactsById')[0]['email']}"
},
"host": {
"api": {
"runtimeUrl": "https://logic-apis-northeurope.azure-apim.net/apim/gmail"
},
"connection": {
"name": "@parameters('$connections')['gmail']['connectionId']"
}
},
"method": "post",
"path": "/Mail"
},
"runAfter": {
"GenerateCoupon": [
"Succeeded"
]
},
"type": "ApiConnection"
}
The full code solution view looks like this:
{
"$connections": {
"value": {
"cognitiveservicestextanalytics": {
"connectionId": "/subscriptions/1b987fd6-b38e-40a1-bca8-4f67e6272c12/resourceGroups/NewLoyaltyPlan/providers/Microsoft.Web/connections/cognitiveservicestextanalytics",
"connectionName": "cognitiveservicestextanalytics",
"id": "/subscriptions/1b987fd6-b38e-40a1-bca8-4f67e6272c12/providers/Microsoft.Web/locations/northeurope/managedApis/cognitiveservicestextanalytics"
},
"gmail": {
"connectionId": "/subscriptions/1b987fd6-b38e-40a1-bca8-4f67e6272c12/resourceGroups/NewLoyaltyPlan/providers/Microsoft.Web/connections/gmail",
"connectionName": "gmail",
"id": "/subscriptions/1b987fd6-b38e-40a1-bca8-4f67e6272c12/providers/Microsoft.Web/locations/northeurope/managedApis/gmail"
}
}
},
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Condition": {
"actions": {
"GenerateCoupon": {
"inputs": {
"body": {
"name": "@body('QueryContactsById')[0]['name']"
},
"function": {
"id": "/subscriptions/1b987fd6-b38e-40a1-bca8-4f67e6272c12/resourceGroups/NewLoyaltyPlan/providers/Microsoft.Web/sites/CADFuncxdb3o43h6p7bq/functions/GenerateCoupon"
}
},
"runAfter": {},
"type": "Function"
},
"Send_email": {
"inputs": {
"body": {
"Body": "Please accept this coupon: @{body('GenerateCoupon')}",
"Subject": "We heard that you were not happy",
"To": "@{body('QueryContactsById')[0]['email']}"
},
"host": {
"api": {
"runtimeUrl": "https://logic-apis-northeurope.azure-apim.net/apim/gmail"
},
"connection": {
"name": "@parameters('$connections')['gmail']['connectionId']"
}
},
"method": "post",
"path": "/Mail"
},
"runAfter": {
"GenerateCoupon": [
"Succeeded"
]
},
"type": "ApiConnection"
}
},
"expression": "@less(body('Detect_Sentiment')?['score'], 0.5)",
"runAfter": {
"Detect_Sentiment": [
"Succeeded"
]
},
"type": "If"
},
"Detect_Sentiment": {
"inputs": {
"body": {
"text": "@{body('QueryCasesById')?['last_feedback']}"
},
"host": {
"api": {
"runtimeUrl": "https://logic-apis-northeurope.azure-apim.net/apim/cognitiveservicestextanalytics"
},
"connection": {
"name": "@parameters('$connections')['cognitiveservicestextanalytics']['connectionId']"
}
},
"method": "post",
"path": "/sentiment"
},
"runAfter": {
"QueryCasesById": [
"Succeeded"
]
},
"type": "ApiConnection"
},
"QueryCasesById": {
"inputs": {
"api": {
"id": "/subscriptions/1b987fd6-b38e-40a1-bca8-4f67e6272c12/resourceGroups/NewLoyaltyPlan/providers/Microsoft.ApiManagement/service/cadapimxdb3o43h6p7bq/apis/58cd5516dc78ac0f84da1289"
},
"method": "get",
"pathTemplate": {
"parameters": {
"id": "@{encodeURIComponent(body('QueryContactsById')[0]['caseNum'])}"
},
"template": "/LegacyAPI/contacts/{id}"
},
"subscriptionKey": "@{triggerBody()['APIMKey']}"
},
"runAfter": {
"QueryContactsById": [
"Succeeded"
]
},
"type": "ApiManagement"
},
"QueryContactsById": {
"inputs": {
"api": {
"id": "/subscriptions/1b987fd6-b38e-40a1-bca8-4f67e6272c12/resourceGroups/NewLoyaltyPlan/providers/Microsoft.ApiManagement/service/cadapimxdb3o43h6p7bq/apis/58cd4c45dc78ac0f84da1287"
},
"method": "get",
"pathTemplate": {
"parameters": {
"id": "@{encodeURIComponent(triggerBody()['id'])}"
},
"template": "/Contacts/contacts/{id}"
},
"subscriptionKey": "@{triggerBody()['APIMKey']}"
},
"runAfter": {},
"type": "ApiManagement"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
"manual": {
"inputs": {
"schema": {}
},
"kind": "Http",
"type": "Request"
}
}
}
}
If you get and 'Internal Server Error' 500 on the Contact Case List (Legacy Ticket API) this could be because the node API has stopped. ssh into the VM and navigate to the /LegacyAPI/CADContacts folder and run 'node server.js'. e.g.
cd CADContacts
node server.js