From d8761bcff42f8292e1b99e23690d800a5edf0a21 Mon Sep 17 00:00:00 2001 From: Blake Li Date: Wed, 1 Feb 2023 09:33:44 -0500 Subject: [PATCH] doc: Update DEVELOPMENT.md for local development. (#1237) --- DEVELOPMENT.md | 142 ++++++++++++++++++++++++++++++++++--------------- 1 file changed, 100 insertions(+), 42 deletions(-) diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index e963700c41..f813d7de4a 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -4,13 +4,23 @@ 1. Clone this repo. -2. Copy the Git pre-commit hooks. This will automatically check the build, run - tests, and perform linting before each commit. (Symlinks don't seem to work, - but if you find a way, please add it here!) +2. (OPTIONAL) Copy the Git pre-commit hooks. This will automatically check the build, run + tests, and perform linting before each commit. (Symlinks don't seem to work, + but if you find a way, please add it here!) - ```sh - cp .githooks/pre-commit .git/hooks/pre-commit - ``` + ```sh + cp .githooks/pre-commit .git/hooks/pre-commit + ``` + + ### Note: You may see this error with the pre-commits due to the monorepo migration: + ``` +[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.10.1:compile (default-compile) on project gapic-generator-java: Compilation failure: Compilation failure: +[ERROR] gapic-generator-java/gapic-generator-java/src/main/java/com/google/api/generator/gapic/composer/rest/ServiceClientTestClassComposer.java:[19,43] package com.google.api.gax.httpjson.testing does not exist +[ERROR] gapic-generator-java/gapic-generator-java/src/main/java/com/google/api/generator/gapic/composer/rest/ServiceClientTestClassComposer.java:[24,38] package com.google.api.gax.rpc.testing does not exist +[ERROR] gapic-generator-java/gapic-generator-java/src/main/java/com/google/api/generator/gapic/composer/grpc/ServiceClientTestClassComposer.java:[18,39] package com.google.api.gax.grpc.testing does not exist + ``` + Remove the pre-commit hooks. Tracking the issue in https://github.com/googleapis/gapic-generator-java/issues/1253 + 3. Install [`bazelisk`](https://github.com/bazelbuild/bazelisk) in your `PATH`. ## Code Formatting @@ -32,17 +42,17 @@ - Run all unit and integration tests. ```sh - mvn test # unit tests + mvn install # unit tests, maven test wouldn't work in root folder because gapic-generator-java is dependant on test jars of gax-java bazel test //... # integration tests ``` - Run all unit tests. ```sh - mvn test + mvn install ``` - -- Run a single or multiple unit tests: +- For running unit tests in `gapic-generator-java` submodule, first build all modules with `mvn install -DskipTests`, then `cd` into `gapic-generator-java` submodule for the following commands: + - Run a single or multiple unit tests: ```sh mvn test -Dtest=JavaCodeGeneratorTest @@ -50,32 +60,32 @@ mvn test "-Dtest=Basic*, !%regex[.*.Unstable.*], !%regex[.*.MyTest.class#one.*|two.*], %regex[#fast.*|slow.*]" ``` -- Update all unit test golden files: + - Update all unit test golden files: - ```sh - mvn test -DupdateUnitGoldens - ``` + ```sh + mvn test -DupdateUnitGoldens + ``` -- Update a single unit test golden file, for example `JavaCodeGeneratorTest.java`: + - Update a single unit test golden file, for example `JavaCodeGeneratorTest.java`: - ```sh - mvn test -DupdateUnitGoldens -Dtest=JavaCodeGeneratorTest - ``` + ```sh + mvn test -DupdateUnitGoldens -Dtest=JavaCodeGeneratorTest + ``` - Run a single integration test for API like `Redis`, it generates Java source code using the Java microgenerator and compares them with the goldens files in `test/integration/goldens/redis`. - ```sh - bazel test //test/integration:redis - ``` + ```sh + bazel test //test/integration:redis + ``` - Update integration test golden files, for example `Redis`. This clobbers all the files in `test/integration/goldens/redis`. - ```sh - bazel run //test/integration:update_redis - ``` + ```sh + bazel run //test/integration:update_redis + ``` ## Showcase Integration Testing @@ -161,40 +171,67 @@ $ cd showcase $ mvn compile -P update ``` -## Running the Plugin +## Running the Plugin under googleapis with local gapic-generator-java -See also above section "Showcase Integration Testing". +For running the Plugin with showcase protos and local gapic-generator-java, see above section "Showcase Integration Testing". To generate a production GAPIC API: 1. Clone [googleapis](https://github.com/googleapis/googleapis). -2. Point to local gapic-generator-java +2. Modify `googleapis/WORKSPACE` to point to local gapic-generator-java - Normally, googleapis's build pulls in googleapis/gapic-generator-java from the - Internet: + Normally, googleapis's build pulls in gapic-generator-java from Maven Central. + For a local run, we first need to build a local SNAPSHOT jar of the generator. Then we point googleapis to + both the local SNAPSHOT jar and the local copy of the generator. + Replace the following section in googleapis ``` - # Java microgenerator. - … - _gapic_generator_java_version = "2.1.0" - - http_archive( - name = "gapic_generator_java", - … - urls = ["https://github.com/googleapis/gapic-generator-java/archive/v%s.zip" % _gapic_generator_java_version], - ) + _gapic_generator_java_version = "2.13.0" + + maven_install( + artifacts = [ + "com.google.api:gapic-generator-java:" + _gapic_generator_java_version, + ], + #Update this False for local development + fail_on_missing_checksum = True, + repositories = [ + "m2Local", + "https://repo.maven.apache.org/maven2/", + ] + ) + + http_archive( + name = "gapic_generator_java", + strip_prefix = "gapic-generator-java-%s" % _gapic_generator_java_version, + urls = ["https://github.com/googleapis/gapic-generator-java/archive/v%s.zip" % _gapic_generator_java_version], + ) ``` - By replacing this portion using the built-in local_repository rule, you can make - it refer to your local development repo: + to ``` + _gapic_generator_java_version = "2.13.1-SNAPSHOT" + + maven_install( + artifacts = [ + "com.google.api:gapic-generator-java:" + _gapic_generator_java_version, + ], + #Update this False for local development + fail_on_missing_checksum = False, + repositories = [ + "m2Local", + "https://repo.maven.apache.org/maven2/", + ] + ) + local_repository( - name = "gapic_generator_java", - path = "/home//gapic-generator-java", + name = "gapic_generator_java", + path = "/absolute/path/to/your/local/gapic-generator-java", ) ``` + + Note: At the time of writing, the gapic-generator version was `2.13.0`. Update the version to the latest version in the pom.xml 3. Build the new target. @@ -208,5 +245,26 @@ To generate a production GAPIC API: bazel build //google/cloud/speech/v2:google-cloud-speech-v2-java ``` + Note: If you are running into bazel build issues, you can try to remove gapic-generator-java cached in your local m2 + Try running this command: + ``` + rm -rf ~/.m2/repository/com/google/api/ + ``` + and then rebuild gapic-generator-java (`mvn clean install`). + +## FAQ + +### Error in workspace: workspace() got unexpected keyword argument 'managed_directories' +Full Error: + +``` +ERROR: Traceback (most recent call last): + File "/home/alicejli/googleapis/WORKSPACE", line 1, column 10, in + workspace( +Error in workspace: workspace() got unexpected keyword argument 'managed_directories' +ERROR: Error computing the main repository mapping: Encountered error while reading extension file 'tools/build_defs/repo/http.bzl': no such package '@bazel_tools//tools/build_defs/repo': error loading package 'external': Could not load //external package +``` +You may be using the latest version of bazel which this project does not support yet. Try installing bazelisk to force +bazel to use the version specified in `.bazeliskrc` \ No newline at end of file