Skip to content

Commit c89faab

Browse files
docs: Move metadata models documentation to CONTRIBUTING.md
- Moved detailed documentation from README to CONTRIBUTING.md - Simplified README to just show usage and link to CONTRIBUTING.md - Added docstring to __init__.py pointing to CONTRIBUTING.md - Removed py.typed from connector_metadata (already exists at root) Co-Authored-By: AJ Steers <aj@airbyte.io>
1 parent 933d478 commit c89faab

File tree

4 files changed

+37
-83
lines changed

4 files changed

+37
-83
lines changed

airbyte_cdk/test/models/connector_metadata/README.md

Lines changed: 3 additions & 82 deletions
Original file line numberDiff line numberDiff line change
@@ -2,94 +2,15 @@
22

33
This package contains Pydantic models for validating Airbyte connector `metadata.yaml` files.
44

5-
## Overview
6-
7-
The models are automatically generated from JSON Schema YAML files maintained in the [airbytehq/airbyte](https://github.com/airbytehq/airbyte) repository at:
8-
```
9-
airbyte-ci/connectors/metadata_service/lib/metadata_service/models/src/
10-
```
11-
12-
During the CDK build process (`poetry run poe build`), these schemas are downloaded from GitHub and used to generate Pydantic models via `datamodel-code-generator`. All models are generated into a single Python file for simplicity and easier imports.
13-
145
## Usage
156

16-
### Validating a metadata.yaml file
17-
187
```python
19-
from pathlib import Path
20-
import yaml
218
from airbyte_cdk.test.models import ConnectorMetadataDefinitionV0
9+
import yaml
2210

23-
# Load metadata.yaml
24-
metadata_path = Path("path/to/metadata.yaml")
25-
metadata_dict = yaml.safe_load(metadata_path.read_text())
26-
27-
# Validate using Pydantic
28-
try:
29-
metadata = ConnectorMetadataDefinitionV0(**metadata_dict)
30-
print("✓ Metadata is valid!")
31-
except Exception as e:
32-
print(f"✗ Validation failed: {e}")
33-
```
34-
35-
### Accessing metadata fields
36-
37-
```python
38-
from airbyte_cdk.test.models import ConnectorMetadataDefinitionV0
39-
40-
metadata = ConnectorMetadataDefinitionV0(**metadata_dict)
41-
42-
# Access fields with full type safety
43-
print(f"Connector: {metadata.data.name}")
44-
print(f"Docker repository: {metadata.data.dockerRepository}")
45-
print(f"Docker image tag: {metadata.data.dockerImageTag}")
46-
print(f"Support level: {metadata.data.supportLevel}")
47-
```
48-
49-
### Accessing other models
50-
51-
All generated models are available in the `generated.models` module:
52-
53-
```python
54-
from airbyte_cdk.test.models.connector_metadata.generated.models import (
55-
ConnectorBreakingChanges,
56-
ConnectorReleases,
57-
ReleaseStage,
58-
SupportLevel,
59-
)
11+
metadata = ConnectorMetadataDefinitionV0(**yaml.safe_load(metadata_yaml))
6012
```
6113

62-
### Available models
63-
64-
The main model is `ConnectorMetadataDefinitionV0`, which includes nested models for:
65-
66-
- `ConnectorType` - Source or destination
67-
- `ConnectorSubtype` - API, database, file, etc.
68-
- `SupportLevel` - Community, certified, etc.
69-
- `ReleaseStage` - Alpha, beta, generally_available
70-
- `ConnectorBreakingChanges` - Breaking change definitions
71-
- `ConnectorReleases` - Release information
72-
- `AllowedHosts` - Network access configuration
73-
- And many more...
74-
7514
## Regenerating Models
7615

77-
Models are regenerated automatically when you run:
78-
79-
```bash
80-
poetry run poe build
81-
```
82-
83-
This command:
84-
1. Downloads the latest schema YAML files from the airbyte repository
85-
2. Generates all Pydantic models into a single file using `datamodel-code-generator`
86-
3. Generates a consolidated JSON schema file for external validation tools
87-
4. Outputs to `airbyte_cdk/test/models/connector_metadata/generated/`:
88-
- `models.py` - All Pydantic models in a single file
89-
- `metadata_schema.json` - Consolidated JSON schema
90-
91-
## Schema Source
92-
93-
The authoritative schemas are maintained in the [airbyte monorepo](https://github.com/airbytehq/airbyte/tree/master/airbyte-ci/connectors/metadata_service/lib/metadata_service/models/src).
94-
95-
Any changes to metadata validation should be made there, and will be automatically picked up by the CDK build process.
16+
See the [Contributing Guide](../../../docs/CONTRIBUTING.md#regenerating-connector-metadata-models) for information on regenerating these models.

airbyte_cdk/test/models/connector_metadata/__init__.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,10 @@
1+
"""Connector metadata models for validation and testing.
2+
3+
These models are auto-generated from JSON schemas in the airbytehq/airbyte repository.
4+
For information on regenerating these models, see the Contributing Guide:
5+
https://github.com/airbytehq/airbyte-python-cdk/blob/main/docs/CONTRIBUTING.md#regenerating-connector-metadata-models
6+
"""
7+
18
from .generated.models import ConnectorMetadataDefinitionV0, ConnectorTestSuiteOptions
29

310
__all__ = [

airbyte_cdk/test/models/connector_metadata/py.typed

Lines changed: 0 additions & 1 deletion
This file was deleted.

docs/CONTRIBUTING.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,33 @@ poetry run poe build
8585

8686
This will generate the code generator docker image and the component manifest files based on the schemas and templates.
8787

88+
## Regenerating Connector Metadata Models
89+
90+
The CDK includes Pydantic models for validating connector `metadata.yaml` files. These models are automatically generated from JSON Schema YAML files maintained in the [airbytehq/airbyte repository](https://github.com/airbytehq/airbyte/tree/master/airbyte-ci/connectors/metadata_service/lib/metadata_service/models/src).
91+
92+
To regenerate the metadata models, run:
93+
94+
```bash
95+
poetry run poe build
96+
```
97+
98+
This command:
99+
1. Downloads the latest schema YAML files from the airbyte repository
100+
2. Generates all Pydantic models into a single file using `datamodel-code-generator`
101+
3. Generates a consolidated JSON schema file for external validation tools
102+
4. Outputs to `airbyte_cdk/test/models/connector_metadata/generated/`:
103+
- `models.py` - All Pydantic models in a single file
104+
- `metadata_schema.json` - Consolidated JSON schema
105+
106+
The models can be imported and used for validation:
107+
108+
```python
109+
from airbyte_cdk.test.models import ConnectorMetadataDefinitionV0
110+
import yaml
111+
112+
metadata = ConnectorMetadataDefinitionV0(**yaml.safe_load(metadata_yaml))
113+
```
114+
88115
## Generating API Reference Docs
89116

90117
Documentation auto-gen code lives in the `/docs` folder. Based on the doc strings of public methods, we generate API documentation using [pdoc](https://pdoc.dev).

0 commit comments

Comments
 (0)