Test Plan

Test Plan for Gradle Functions Plugin

Prerequisites

Tool	Required Version
JDK	1.8
Gradle	4.10 and above

Gradle Versions

We support 4.x, 5.x, 6.x versions of gradle, so the best way of switching gradle version is to use gradle wrapper, you can use this tutorial to create a simple demo gradle project, then you can change the gradle version by modifying file gradle/wrapper/gradle-wrapper.properties, now you can use .\gradlew(./gradlew for linux) for executing gradle commands. Use this page to find some popular gradle versions, we should test the following versions:

• Gradle 6.0.1 [P0]
• Gradle 5.6.2 [P1]
• Gradle Gradle 4.10.2 [P2]

Java versions

Azure functions only support Java 8, you need to test whether there is a warning "Azure Functions only support JDK 8, which is lower than local JDK version " on JDK 13 for local run and deploy. [P1]

Azure Functions Core Tools

There are several ways of installing Azure Functions Core Tools, for example, use npm, chocolate, brew and apt-get. All types need to be tested, and we need to test some cases when multiple Azure Functions Core Tools installations are at the same machine. As a special case, you need to test the case when the tool is not installed yet for a good error message. The following ways should be tested:

• No installations [P0]
• npm [P0]
• apt-get [P1]
• brew [P1]
• choco [P2]
• npm + choco (windows) [P2]
• apt-get + npm [P2]
• brew + npm [P2]

Authentication

The gradle plugin supports all kinds of authentication ways except maven settings. (you should not use serverId). The following cases should be tested:

• Not login and no auth configurations (Default: auto) -> device login or OAuth due to whether browser is available [P0]
• Service Principles in plugin configuration (key | cert | cert with password) -> Can deploy with right SP configuration [P2]
• Bad Service Principles configuration (missing key | missing cert | cert with wrong password) -> Cannot pass auth with good error message [P1]
• mvn azure:login -> Can deploy with right subscription id (need to test four cases with subscription id is set and not set in build.gradle or use 'mvn azure:select-subscripion') [P2]
• Azure Cli -> Can deploy with right subscription id (need to test four cases with subscription id is set and not set in build.gradle or use 'az account set --subscripion=xxx') [P0]
• Cloud shell -> Can deploy with right subscription id (need to two four cases with subscription id is set and not set in build.gradle) [P1]

For the following properties can be set in build.gradle for Service Principles in plugin configuration

Property	Required	Description
client	true	Specifies the Client ID of your service principal.
tenant	true	Specifies the Tenant ID of your service principal.
key	false	Specifies the password if your service principal uses password authentication.
certificate	false	Specifies the absolute path of your certificate if your service principal uses certificate authentication. Note: Only PKCS12 certificates are supported.
certificatePassword	false	Specifies the password for your certificate, if there is any.
environment	false	Specifies the target Azure cloud environment; the default value is AZURE. The possible values are: - AZURE - AZURE_CHINA - AZURE_GERMANY - AZURE_US_GOVERNMENT

Here is a sample configuration for authentication:

azurefunctions {
   ...
   auth {        
        client = 'df4d03fa-135b-4b7b-932d-2f2ba6449792'
        tenant = '72f988bf-86f1-41af-91ab-2d7cd011db47'
        key = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
        environment = 'AZURE'
    }
}

Telemetry

First time notice

When running first time on the machine (marked by file <user_home>/.azure/gradleplugins.properties), there will be a notice like the following message, verify it is shown first time and disappeared next time.

Data/Telemetry
---------
This project collects usage data and sends it to Microsoft to help improve our products and services.
Read Microsoft's privacy statement to learn more: https://privacy.microsoft.com/en-us/privacystatement.

You can change your telemetry configuration through 'allowTelemetry' property.
For more information, please go to https://aka.ms/azure-gradle-config.

• First Time notice [P2]

Collect BI data

Patch the Application-Insight jar as described in vsc-java-test, check the normal/error trace by at ${user.home}/applicationinsight-output.txt like the following:

"name":"LocalRunTask_Decorated.start",
"properties": { "pluginVersion":"1.0.0-SNAPSHOT","pluginName":"azurefunctions","installationId":"a5d32983f792076e1f6c8b7c9b4... }

• Telemetry data [P0]

Package

The gradle plugin supports to package an azure functions project into a staging folder, you can use .\gradlew azureFunctionsPackage to build the staging folder. You should verify several azure functions sample projects with the same staging folder files with the staging folder generated by mvn package. The basic test is for the generated project fabrikam-functions by following tutorial

• original project: fabrikam-functions (generated by mvn archetype) -> add build.gradle -> Can generate same staging folder by .\gradlew azureFunctionsPackage as mvn package [P0]
• fabrikam-functions -> add some triggers using mvn azure-functions:add -> add build.gradle -> Can generate same staging folder by .\gradlew azureFunctionsPackage as mvn package [P0]
• fabrikam-functions -> add some triggers using mvn azure-functions:add -> make some valid code changes in Function.java(eg: change the text in response) -> Can generate same staging folder by .\gradlew azureFunctionsPackage as mvn package [P2]
• fabrikam-functions -> add some triggers using mvn azure-functions:add -> make some bad code changes that will cause compile error -> Cannot pass .\gradlew azureFunctionsPackage with compile errors. [P2]
• fabrikam-functions -> add some triggers using mvn azure-functions:add -> add a third-party libraries in compile scope with extra dependencies -> Can generate same staging folder by .\gradlew azureFunctionsPackage as mvn package(need to add the libraries in pom.xml) [P1]
• fabrikam-functions -> add some triggers using mvn azure-functions:add -> add a third-party libraries in test scope with extra dependencies -> Can generate same staging folder by .\gradlew azureFunctionsPackage as mvn package(need to add the libraries in pom.xml) [P2]

You should check carefully of the lib folder for added third-party libraries and their dependencies(in compile scope), for example:

org.zeroturnaround:zt-zip:1.6 depends on commons-io:commons-io and org.slf4j:slf4j-api 
org.zeroturnaround:zt-zip:1.6 test depends on junit:junit and log4j:log4j and org.slf4j:slf4j-log4j12

add org.zeroturnaround:zt-zip in build.gradle:

compile group: 'org.zeroturnaround', name: 'zt-zip', version: '1.6'

will add three jars : org.zeroturnaround:zt-zip and commons-io:commons-io and org.slf4j:slf4j-api in lib folder, but will not add junit:junit and log4j:log4j and org.slf4j:slf4j-log4j12 in lib folder.

There is another case of checking dependencies should be updated with build.gradle, eg: remove the org.zeroturnaround:zt-zip from build.gradle, the commons-io:commons-io and org.slf4j:slf4j-api in lib folder will be removed.

compile group: 'org.zeroturnaround', name: 'zt-zip', version: '1.6' // compile scope -> will copy to the *lib* folder
implementation 'org.hibernate:hibernate-core:3.6.7.Final' // runtime scope -> will copy to the *lib* folder
testImplementation 'junit:junit:4.12' // test scope -> will not copy to the *lib* folder

see this doc for how to config scope for dependencies, only test scope dependencies will not be copied to lib folder(gradle projects have no provided scope for java libraries, see here).

Here are all the test cases for package:

• check referenced libraries and their non-test dependencies are copied to the lib folder [P1]
• check referenced libraries in test scope and their dependencies are NOT copied to the lib folder [P2]
• check when removing libraries from build.gradle, the jars and their all dependencies are removed from the lib folder when .\gradlew azureFunctionsPackage finishes [P2]
• check the classes in src/main folder are included in the only jar on the root of staging folder [P2]
• check the classes in src/test folder are NOT included in the only jar on the root of staging folder [P2]
• check host.json and local.settings.json and /function.json are copied to the staging folder, when changing the content of host.json and local.settings.json, the files in staging folders are changed too, changing the function name in Function.java will remove the old folder with old function name [P2]
• check the staging folder can be run successfully by executing func host start on staging folder [P2]
• check the azureFunctionsPackage task will fail if the Function.java and FunctionTest.java are deleted. (maven case will be executed successfuly, will discuss this case with PM) [P2]
• azureFunctionsPackage can be run successfully on multiple functions(add some functions using mvn azure-functions:add) with more directories with the function names at staging folder [P1]
• azureFunctionsRun can be run successfully when there is no host.json or local.settings.json [P2]

Package Zip

The gradle plugin supports to package staging folder as a zip file, you can use .\gradlew azureFunctionsPackageZip to build the zip file. The test cases for package zip is the same as package by check the content of the zip file against with staging folder(but without the file local.settings.json), so every time you finished package task, check the zip file.

Locale run

The gradle plugin supports to local run the staging folder using .\gradlew azureFunctionsRun, you should verify:

• azureFunctionsRun can be run without azureFunctionsPackage and azureFunctionsPackageZip, the http-trigger can be accessed in browser [P2]
• azureFunctionsRun should be able to launch local functions with debug parameter(localDebug = "transport=dt_socket,server=y,suspend=n,address=5005" in build.gradle), which can be attached in VSCode with java debugger plugin(5005). [P0]
• azureFunctionsRun can report port issue when 7071 is occupied by another command func host start
• Use ctrl+c to stop azureFunctionsRun will cause the http://localhost:7071/api/HttpTrigger-Java un-available. [P0]
• azureFunctionsRun fails when the files: Function.java and FunctionTest.java are deleted.(maven case will be executed successfully, will discuss this case with PM) [P2]
• azureFunctionsRun can be run successfully on multiple functions(add some functions using mvn azure-functions:add) [P2]
• azureFunctionsRun can be run successfully when there is no host.json or local.settings.json [P2]

Deploy

The gradle plugin supports to deploy azure cloud according the settings at azurefunctions section of build.gradle using .\gradlew azureFunctionsDeploy, you should verify:

• azureFunctionsDeploy can be run successfully for fabrikam-functions creating new azure windows functions by using non-exist appName, verify subscription/resourceGroup/pricingTier/region/os and app settings are the same with configuration. [P0]
• azureFunctionsDeploy can be run successfully for fabrikam-functions updating an existing windows functions by using existing appName, verify app settings are updated [P1]
• azureFunctionsDeploy can be run successfully for linux os [P2]
• azureFunctionsDeploy can be run successfully for docker os, with public and non-public images. [P2]
• azureFunctionsDeploy can be run successfully for different deployment types, eg: deployment { type = 'run_from_blob' } [P2]
• azureFunctionsDeploy can be run successfully on multiple functions(add some functions using mvn azure-functions:add) [P2]

The sample configuration for private docker image are:

runtime {
  os = 'docker'
  image = 'redis'
  registryUrl = 'http://192.168.56.101:5000/v2'
  username = 'a'
  password = 'b'  
}