Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: generate a GAPIC library from api definition #3208

Merged
merged 23 commits into from
Sep 20, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
change readme
  • Loading branch information
JoeWang1127 committed Sep 17, 2024
commit 02e6ae20931fe5f45519f70ee2e56178dfa54f89
2 changes: 1 addition & 1 deletion library_generation/DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ shell session.
## Running the script
The entrypoint script (`library_generation/cli/entry_point.py`) allows you to
update the target repository with the latest changes starting from the
googleapis committish declared in `generation_config.yaml`.
googleapis commitish declared in `generation_config.yaml`.

### Download the repo
For example, google-cloud-java
Expand Down
33 changes: 22 additions & 11 deletions library_generation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,12 @@ right version for each library.
Please refer [here](go/java-client-releasing#versionstxt-manifest) for more info
of versions.txt.

### Api definition path (`api_definition_path`), optional

The path to where the api definition (proto, service yaml) resides.

The default value is the current working directory when running the script.

## Output of `entry_point.py`

### GAPIC libraries
Expand All @@ -74,11 +80,13 @@ will be created/modified:
| pom.xml (repo root dir) | Always generated from inputs |
| versions.txt | New entries will be added if they don’t exist |

### googleapis commit history
### Change history

If both `baseline_generation_config` and `current_generation_config` are
specified, and they contain different googleapis commit, the commit history will
be generated into `pr_description.txt` in the `repository_path`.
specified and the contents are different, the changed contents will be generated
into `pr_description.txt` in the `repository_path`.
In addition, if the `googleapis_commitish` is different, the googleapis commit
history will be generated.

## Configuration to generate a repository

Expand All @@ -96,7 +104,7 @@ They are shared by library level parameters.
| gapic_generator_version | No | set through env variable if not specified |
| protoc_version | No | inferred from the generator if not specified |
| grpc_version | No | inferred from the generator if not specified |
| googleapis-commitish | Yes | |
| googleapis_commitish | Yes | |
| libraries_bom_version | No | empty string if not specified |

### Library level parameters
Expand Down Expand Up @@ -183,22 +191,25 @@ The virtual environment can be installed to any folder, usually it is recommende
2. Assuming the virtual environment is installed under `sdk-platform-java`.
Run the following command under the root folder of `sdk-platform-java` to install the dependencies of `library_generation`

```bash
python -m pip install -r library_generation/requirements.txt
```
```bash
python -m pip install -r library_generation/requirements.txt
```

3. Run the following command to install `library_generation` as a module, which allows the `library_generation` module to be imported from anywhere
```bash
python -m pip install library_generation/
```
```bash
python -m pip install library_generation/
```

4. Download api definition to a local directory

## An example to generate a repository using `entry_point.py`

```bash
python library_generation/entry_point.py generate \
--baseline-generation-config-path=/path/to/baseline_config_file \
--current-generation-config-path=/path/to/current_config_file \
--repository-path=path/to/repository
--repository-path=path/to/repository \
--api-definition-path=path/to/api_definition
```
If you run `entry_point.py` with the example [configuration](#an-example-of-generation-configuration)
shown above, the repository structure is:
Expand Down
8 changes: 4 additions & 4 deletions library_generation/test/integration_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,13 +41,13 @@
repo_prefix = "https://github.com/googleapis"
output_dir = shell_call("get_output_folder")
# this map tells which branch of each repo should we use for our diff tests
committish_map = {
commitish_map = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fun fact: for a long time I thought it only was committish (two ts).
I happens to be an alternative convenience term coming from commit-ish.

The git glossary offers either commit-ish or committish. I always thought this would be too much to start a campaign to update all config yamls and its definition.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Diego! That's good to know! Yeah I agree we should follow the naming conventions. Maybe an enhancement in the backlog.

"google-cloud-java": "chore/test-hermetic-build",
"java-bigtable": "chore/test-hermetic-build",
}
baseline_config_name = "baseline_generation_config.yaml"
current_config_name = "current_generation_config.yaml"
googleapis_committish = "113a378d5aad5018876ec0a8cbfd4d6a4f746809"
googleapis_commitish = "113a378d5aad5018876ec0a8cbfd4d6a4f746809"
# This variable is used to override the jar created by building the image
# with our own downloaded jar in order to lock the integration test to use
# a constant version specified in
Expand All @@ -71,15 +71,15 @@ def setUp(cls) -> None:
os.makedirs(f"{golden_dir}", exist_ok=True)

def test_entry_point_running_in_container(self):
api_definition_path = self.__copy_api_definition(googleapis_committish)
api_definition_path = self.__copy_api_definition(googleapis_commitish)
config_files = self.__get_config_files(config_dir)
for repo, config_file in config_files:
config = from_yaml(config_file)
repo_location = f"{output_dir}/{repo}"
config_location = f"{golden_dir}/../{repo}"
# 1. pull repository
repo_dest = self.__pull_repo_to(
Path(repo_location), repo, committish_map[repo]
Path(repo_location), repo, commitish_map[repo]
)
# 2. prepare golden files
library_names = self.__get_library_names_from_config(config)
Expand Down