Skip to content

TypeSpec Java QuickStart

Jump to bottom
Weidong Xu edited this page Nov 13, 2024 · 11 revisions

Before you Start

Java Azure SDK Design Guidelines is the overall design guideline of the data-plane SDK. Make sure you are familiar with concepts such as "Service Client" and "Packaging".

Make sure you are familiar with Git and Maven, especially the Build Lifecycle.

Please also take a look at Protocol Methods on the overview of the typical protocol methods from minimal data-plane client SDK.

⚠️ If you uses OpenAPI 2.0, please read Protocol Methods Quickstart with AutoRest.

Generate SDK

Module folder in SDK repository

For service, it is usually your service name in REST API specifications. For instance, if the TypeSpec is under specification/storage, the service would normally be storage.

For module, please refer to Namespace in Java Guideline. For example, if the namespace is com.azure.storage.blob, the module should be azure-storage-blob.

☑️ Namespace is decided by SDK arch board.

After service and module, the output-dir would be <sdk-repository-dir>/sdk/<service>/<module>. This is where the SDK code should be generated when you call npx tsp compile.

Use SDK Automation from REST API specifications

☑️ It is recommended to configure TypeSpec package on REST API specifications. Please refer to these guidelines.

Particularly, here is a sample for tspconfig.yaml. Please make sure service-dir, package-dir, namespace (for typespec-java) is correctly configured.

  • "parameters.service-dir.default" would be sdk/<service>
  • "options.@azure-tools/typespec-java.package-dir" would be <module>

After configuration is completed, making a draft pull request on azure-rest-api-specs repository would automatically trigger SDK generation for all configured SDKs. The automation will create new pull request on azure-sdk-for-java repository. One can fork it for further development.

Use TypeSpec defined in REST API specifications

☑️ The package structure in the azure-rest-api-specs repository should follow these guidelines.

SDK will be generated under the SDK project folder of azure-sdk-for-java.

Configuration

You can update tsp-location.yaml under SDK project folder to track the TypeSpec project.

You can refer to the tsp-location.yaml which describes the supported properties in the file.

Generate Code

With tsp-location.yaml ready, run the following tsp-client command from project directory (i.e. your current directory is <sdk-repository-dir>/sdk/<service>/<module>) to generate the code:

tsp-client update

☑️ tsp-client can be installed globally via npm install -g @azure-tools/typespec-client-generator-cli.

The version of TypeSpec-Java is configured in emitter-package.json. Change it in local, if you would like to use a different version of typespec-java.

☑️ If you would like to test with locally modified TypeSpec, invoke tsp-client update with --save-inputs option. This tells the command not to delete the "TempTypeSpecFiles" folder, therefore you can modify the TypeSpec and then run tsp-client generate --save-inputs. Remember that all changes in "TempTypeSpecFiles" folder would be temporary.

Follow-up code generation

If you modify the TypeSpec source, you may need to re-generate the SDK. This could happen frequently when you do SDK arch board review, and reviewer might ask you to rename some TypeSpec models.

If you have not customized anything in the SDK, it is advised to delete <output-dir>/src/main/java folder, before re-generate with the same tsp-client update.

The reason is to avoid previous model/class get remained in Java source files, after TypeSpec source already renamed them to new name.

Build

See Building and Adding a Module.

See the contributing guide.

Here is the list of files that you are required to modify/confirm for your new module.

  • eng/versioning/version_client.txt for versioning
  • sdk/<service>/pom.xml for project
  • sdk/<service>/ci.yml for CI

Install for Local Test

See Building. Install SDK to local maven repository: mvn install -f sdk/<service>/<module>/pom.xml -Dgpg.skip -Drevapi.skip -DskipTests

Tips: Javadoc might fail if other than Java 11. If you see error message like "Error while generating Javadoc", you can add option -Dmaven.javadoc.skip to skip Javadoc.

See Maven Getting Start Guide, if you would like to create a client that uses your SDK. See sdk/<service>/<module>/README.md for adding the SDK to dependency.

Improve SDK Documentation

There is many information about the SDK that TypeSpec-Java will never know. These can only be provided by a contributor.

README

A minimal README.md is generated by TypeSpec-Java. Your first task will be improving it.

☑️ You would want to modify [product_documentation] to point to your service documentation.

If SDK uses an authentication other than Azure Active Directory, please include it in "Getting started" section, and provide a sample.

☑️ All URL link in README should be absolute. It is fine to comment out some link, if it will not appear in main branch until the merge of the PR (e.g. link to the source code of a sample).

It is recommended to run a spell checker, e.g. cspell, to make sure the README is in good quality.

Samples

A blank Java sample file is generated as ReadmeSamples.java, which is synchronized with your README (see Codesnippet Maven Plugin).

Add your sample code between // BEGIN: ... and // END: ... in ReadmeSamples.java, and build the project, you will find your Examples section in README updated with the code.

A collection of generated Java sample files (based on examples-directory option) is generated under <module>.generated package. They might not be directly usable as runnable sample code, however you should get the basic idea on how code works, and may even modify some to be runnable.

⚠️ If there is Partial Update or Customization, the generated samples may not compile pass. This is expected. as we'd like dev only use them as reference. You are free to delete the /generated folder, if it causes trouble in the CI.

generate-samples=false in emitter options turns off the generated Java sample files.

An example of adding samples is demonstrated in this commit.

See Guidelines for how to write a good sample.

Tests

A <client>TestBase file and a collection of generated Java test files (based on examples-directory option) is generated under <module>.generated package.

generate-tests=false in emitter options turns off the generated Java sample files.

See Adding Tests.

Feature Checklist

Authentication and credential

TypeSpec-Java supports OAuth2 and ApiKey authentication.

However, if service requires advanced solution (for example, JWT claim or HMAC-SHA256 authentication scheme), you are responsible to provide the implementation (usually as a subclass of HttpPipelinePolicy), and customize the ClientBuilder.

Long-running operation

TypeSpec-Java supports a default chained polling strategy for long-running operation.

It supports status monitor LRO and simple Location-header-based LRO. It does not support the outdated resource based LRO.

If service uses non-supported or non-standard LRO, you can contact Java SDK for guidance on the implementation of a specialized polling strategy, and customization.

File upload

TypeSpec-Java supports application/octet-stream, but it does not support multipart/form-data. An issue in azure-core tracks the status of the feature.

Review API Documentation

Generate Javadoc: mvn javadoc:javadoc -f sdk/<service>/<module>/pom.xml

Review the generated API documentation sdk/<service>/<module>/target/site/apidocs/index.html

Improve SDK Code

Certain customization (particularly, renaming) can be done via client.tsp, see Typespec documentations.

ClientBuilder, Client, AsyncClient is the user-facing API surface that you might be interested to improve. See Partial Update and Customization.

Code Review

Code review happens on 2 places.

One is the usual GitHub pull request, for developers. Please start with a "draft pull request", and make a round of self-review according to Review Checklist.

After completed the checklist, make it "ready to review". Comment @Azure/dpg-devs for awareness in PR to loop in SDK developers for review.

Another is the apiview, for architects. GitHub pull request (PR) should automatically trigger the apiview. If this does not happen, upload one manually:

  1. Build the SDK.
  2. Login apiview with GitHub account.
  3. "Create review" at top-left corner.
  4. "Browse" and upload sdk/<service>/<module>/target/<module>-<version>-sources.jar, and comment the result link in PR. If you did not find the source jar in step 4, run mvn source:jar.

Package Release

See Release SDK.

Prepare Release Script

In particular, you need to follow "Review release documentation" and "mark your package as in release".

Before running the "Prepare-Release.ps1" script, make sure your CHANGELOG.md is correct. Except for first release, it should contain following text on dependency updates.

### Other Changes

#### Dependency Updates

- Upgraded `azure-core` from `1.32.0` to version `1.33.0`.
- Upgraded `azure-core-http-netty` from `1.12.5` to version `1.12.6`.

After running the script, check the "release date" and make sure it is your scheduled release date.

For first release, the CHANGELOG.md may look like this.

# Release History

## 1.0.0-beta.1 ({release date})

- {describe the client library}

### Features Added

- {describe the features}

Merge Increment Versions PR

After package release, one will see an "Increment versions for releases" PR. Please ping this PR to SDK developers to approve and merge.

Appendix

Java Development Environment

Windows

Download Java, install.

Download Maven, unpack, set to PATH.

Download Node.js, install.

Optional for Git:

  • Download Git, install.

Linux with APT

sudo apt -y install openjdk-17-jdk

sudo apt -y install maven

curl -sL https://deb.nodesource.com/setup_16.x | sudo bash -
sudo apt -y install nodejs

Adding Tests

Adding runnable tests requires both a good understanding of the service, and the knowledge of the client and test framework. Feel free to contact SDK developers, if you encountered issues on client or test framework.

Test environment

Before testing the client, it is recommended that you prepare a Bicep or ARM template (or code, or CLI script, equivalent to the ARM template) that creates all required Azure resources and assigns necessary RBAC for your client. This ARM template will be very helpful for you to re-run the tests after version update, or for other developers to create their test environment.

⚠️ One may see exception java.lang.IllegalAccessError: class com.azure.core.http.netty.NettyAsyncHttpClient (in unnamed module) cannot access class ... when running tests in IDE. In this case, one can temporary delete the src/main/java/module-info.java file and try again.

Client authentication

If your service does not support connection string or API key authentication, it would likely use AzureAD OAuth2 token for authentication. A common solution is to provide an application and its service principal and to provide RBAC to the service principal for the access to the Azure resource of your service.

Client requires following 3 environment variables for the service principal using client ID/secret for authentication:

AZURE_TENANT_ID
AZURE_CLIENT_ID
AZURE_CLIENT_SECRET

On Linux, you can call export:

export AZURE_TENANT_ID=<tenant_id>
export AZURE_CLIENT_ID=<client_id>
export AZURE_CLIENT_SECRET=<client_secret>

On Windows, it is set.

Live tests

Live tests require one more environment variable:

export AZURE_TEST_MODE=LIVE

TypeSpec-Java already provided you the generated Java test files at sdk/<service>/<module>/src/test/java/<namespace>/generated folder, and a <client>TestBase class. You should double check the beforeTest() method in <client>TestBase class, and modify it if necessary.

A sample <client>TestBase class can be found here. The "Client" is configured different for different test mode. For "PLAYBACK", it uses a mock TokenCredential and a playback HttpClient; for "RECORD", it uses a record HttpPolicy; for "LIVE", it uses what customer would use.

It is recommended to copy the test files you would like to modify and use to sdk/<service>/<module>/src/test/java/<namespace> folder.

It is common for you to put configuration from your test environment to the code, as long as they are not a secret. Alternatively, you can keep rely on Configuration.getGlobalConfiguration().get("<env_variable>") and define a few additional environment variables.

After you settle down the builder, you can decide which REST APIs you want to test, and write the code and verification in the testClient() method. The generated samples under <module>.generated package can be used as reference for your test code.

After you finish the test code, call mvn test under sdk/<service>/<module> to execute the tests. If everything works fine, you will see following log.

[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.776 s - in com.azure.messaging.webpubsub.ClientTests
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

An example of adding live tests is demonstrated in this commit.

Record/playback tests

The live tests only run when environment variable AZURE_TEST_MODE is set to LIVE, and hence it is usually run manually.

Next step, you will convert the live tests to record/playback tests.

⚠️ Record/playback tests MUST use the Test Proxy tool. If your previous tests is recorded in repository, please follow Test Proxy Migration.

⚠️ Use TestBase.setPlaybackSyncPollerPollInterval on the SyncPoller instance, if your test involves long-running operation API.

Record

Firstly, the HTTP requests in the tests need to be recorded.

Remove @DoNotRecord(skipInPlayback = true) annotation in ClientTests.java. And set environment variable AZURE_TEST_MODE to RECORD:

export AZURE_TEST_MODE=RECORD

Then call mvn test as if this is live tests.

Please follow this guide on updating the test recordings.

☑️ Permission is required for updating the asserts repository. If you failed in test-proxy push, please join a partner write team.

Playback

These recorded tests can now be run in playback mode automatically (it can be run locally via the same mvn test call, by setting environment variable AZURE_TEST_MODE to PLAYBACK).

Be aware that in playback mode during continuous integration, there is no test environment nor environment variables available. Hence if you rely on Configuration.getGlobalConfiguration().get("<env_variable>"), make sure you provide a second parameter as the default value, for the case when <env_variable> is not available.

Enable test coverage

Modify pom.xml to enable test coverage.

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
-    <jacoco.skip>true</jacoco.skip>
  </properties>

At beta, one can use a lower minimum coverage.

Partial Update

You can in-place modify the generated code, and TypeSpec-Java will keep the hand-written code intact when it re-generates the SDK.

The user-facing API is of pattern:

FooClient client = new FooClientBuilder()
    .endpoint(...)
    .credential(new DefaultAzureCredentialBuilder().build())
    .buildClient();

FooAsyncClient asyncClient = new FooClientBuilder()
    .endpoint(...)
    .credential(new DefaultAzureCredentialBuilder().build())
    .buildAsyncClient();

And the classes involved is FooClient, FooAsyncClient, and FooClientBuilder.

All of these classes, as well as classes under models package can be customized in-place, to enhance the usability of the SDK, or to add new feature.

Javadoc in package-info.java can be customized in-place.

You can add new class variables and new class methods to these classes. With partial-update option in emitter options, new code generated via TypeSpec-Java will not override your change.

  "@azure-tools/typespec-java":
    package-dir: "azure-contoso-widgetmanager"
    namespace: com.azure.contoso.widgetmanager
+    partial-update: true

For modifying the signature or the implementation of a class method, please remove the @Generated annotation, so TypeSpec-Java can recognize that you have modified the method.

If you would like to remove a class method, simply make the method package private, so it is not exposed to customer. Also you need to remove the @Generated annotation.

It is advised that you keep the modifications simple. If certain feature requires complicated code, it is better to put the implementation code in some other classes within implementation package, and only add the public class method and invocation in these Client or ClientBuilder classes.

Here are a few examples.

Formatter

TypeSpec-Java will format the Java file.

<!-- @formatter:off --> and <!-- @formatter:on --> can be added to a block of Javadoc to disable the formatter.

// @formatter:off and // @formatter:on can be added to a block of code to disable the formatter.

Customization

The customization is an advanced feature, that requires developer to code the customization that may modify the abstract syntax tree directly.

It is usually used on customization on class annotation or class signature, where Partial Update @Generated annotation is not able to exactly specify.

You can add the customization-class option in emitter options. TypeSpec-Java will load and compile the class, and execute it to modify the generated code.

  "@azure-tools/typespec-java":
    package-dir: "azure-contoso-widgetmanager"
    namespace: com.azure.contoso.widgetmanager
+    customization-class: customization/src/main/java/MyCustomization.java

Here is an example of the customization code that modifies the class signature.

⚠️ At present, customization may fail due to certain caching problem. Deleting the "autorest-java-language-server" folder under system temporary folder (typically \Users\<user>\AppData\Local\Temp on Windows) may solve the problem.

Troubleshoot CI Failure

Check "Build" and "Analyze" first, the "Test"s.

Make sure the branch of the PR is updated from latest main branch.

It helps to run mvn clean verify locally, before checking CI failure.

Verify versions in POM files

See the contributing guide on versioning.

TLDR: run python eng/versioning/update_versions.py --ut library --bt client --sr

Check spelling

Fix spelling in code or in markdown.

Suppression at .vscode/cspell.json.

Verify Readmes / Verify sample metadata / Verify ChangeLogEntry

Fix corresponding files.

For sample metadata error "invalid slug", see Taxonomies for Learn. Update Test-SampleMetadata.ps1 script after Taxonomies added.

Run SpotBugs, Checkstyle, RevApi, and Javadoc

Fix code for SpotBugs, Checkstyle, and Javadoc.

If you run it locally, make sure your build tool is re-installed.

Avoid breaking changes for RevApi.

Suppression at eng/code-quality-reports/src/main/resources.

Verify TypeSpec Code Generation

The generated code maybe outdated. Generate it again with command line.

Run tests for From Source run

Besides test failure, it could also be caused by insufficient test coverage.

[WARNING] Rule violated for bundle azure-ai-anomalydetector: lines covered ratio is 0.21, but expected minimum is 0.40
[WARNING] Rule violated for bundle azure-ai-anomalydetector: branches covered ratio is 0.27, but expected minimum is 0.30

For beta SDK, you are allowed to lower the minimum coverage, by adding following Jacoco settings in pom.xml.

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+    <jacoco.min.linecoverage>0.2</jacoco.min.linecoverage>
+    <jacoco.min.branchcoverage>0.2</jacoco.min.branchcoverage> 
  </properties>

For GA SDK, it is expected to follow the default coverage, which is 0.40 for line coverage, 0.30 for branch coverage.

Review Checklist

For first-time onboard:

  • Namespace is approved by SDK arch board review. It helps to comment the link to SDK arch board review PR.
  • Related file for Build is added or updated.
  • [If no CI run] Comment /azp run prepare-pipelines in PR, to create the DevOps CI pipeline (contact SDK developers, if you cannot run it).
  • [If no CI run] Comment /azp run java - <service> - ci in PR, to start the newly created CI pipeline.

Common for any PR:

  • CI pass. See Troubleshoot CI Failure.
  • README.md has correct description of the client library and service, including "Getting started" section, "Key Concept" section, and "Example" section. All link should be provided, and no locale (e.g. "en-us") is allowed in any link.
  • Sample code in README.md must be in a Java file under samples folder, and synchronized via Codesnippet Maven Plugin.
  • Samples and tests are included. See Guidelines for how to write a good sample.
  • Tests must use TestBase.setPlaybackSyncPollerPollInterval on long-running operation API.
  • Test recordings does not contain "secret" ("Build Compliance" CI likely able to flag it).

For PR with GA (e.g. 1.0.0) release:

  • README.md and samples must be of high quality.
  • Good test coverage. It is recommended that every operation is covered in tests.

Client Library

User Guides

Extension Points

Contributor Guides

Management Library

User Guides

Contributor Guides

Spring Cloud Azure

Clone this wiki locally