Skip to content

Commit

Permalink
Remove outdated note about CDK not supporting destinations (airbytehq…
Browse files Browse the repository at this point in the history
  • Loading branch information
sherifnada authored Jul 26, 2021
1 parent 4c8ce60 commit 9d8b173
Show file tree
Hide file tree
Showing 3 changed files with 27 additions and 25 deletions.
45 changes: 26 additions & 19 deletions airbyte-cdk/python/README.md
Original file line number Diff line number Diff line change
@@ -1,78 +1,85 @@
# Connector Development Kit (Python CDK)
# Connector Development Kit \(Python\)

The Airbyte Python CDK is a framework for rapidly developing production-grade Airbyte connectors.
The CDK currently offers helpers specific for creating Airbyte source connectors for:
* HTTP APIs (REST APIs, GraphQL, etc..)
The Airbyte Python CDK is a framework for rapidly developing production-grade Airbyte connectors. The CDK currently offers helpers specific for creating Airbyte source connectors for:

* HTTP APIs \(REST APIs, GraphQL, etc..\)
* Singer Taps
* Generic Python sources (anything not covered by the above)
* Generic Python sources \(anything not covered by the above\)

The CDK provides an improved developer experience by providing basic implementation structure and abstracting away low-level glue boilerplate.

This document is a general introduction to the CDK. Readers should have basic familiarity with the [Airbyte Specification](https://docs.airbyte.io/architecture/airbyte-specification) before proceeding.

## Getting Started

Generate an empty connector using the code generator. First clone the Airbyte repository then from the repository root run
```

```text
cd airbyte-integrations/connector-templates/generator
./generate.sh
```

then follow the interactive prompt. Next, find all `TODO`s in the generated project directory -- they're accompanied by lots of comments explaining what you'll need to do in order to implement your connector. Upon completing all TODOs properly, you should have a functioning connector.

Additionally, you can follow [this tutorial](docs/tutorials/http_api_source.md) for a complete walkthrough of creating an HTTP connector using the Airbyte CDK.
Additionally, you can follow [this tutorial](https://github.com/airbytehq/airbyte/tree/184dab77ebfbc00c69eea9e34b7db29c79a9e6d1/airbyte-cdk/python/docs/tutorials/http_api_source.md) for a complete walkthrough of creating an HTTP connector using the Airbyte CDK.

### Concepts & Documentation
See the [concepts docs](docs/concepts/README.md) for a tour through what the API offers.

See the [concepts docs](concepts/) for a tour through what the API offers.

### Example Connectors

**HTTP Connectors**:

* [Exchangerates API](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-exchange-rates/source_exchange_rates/source.py)
* [Stripe](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-stripe/source_stripe/source.py)
* [Slack](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-slack/source_slack/source.py)

**Singer connectors**:

* [Salesforce](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-salesforce-singer/source_salesforce_singer/source.py)
* [Github](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-github-singer/source_github_singer/source.py)

**Simple Python connectors using the barebones `Source` abstraction**:

* [Google Sheets](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-google-sheets/google_sheets_source/google_sheets_source.py)
* [Mailchimp](https://github.com/airbytehq/airbyte/blob/master/airbyte-integrations/connectors/source-mailchimp/source_mailchimp/source.py)

## Contributing

### First time setup
We assume `python` points to python >=3.7.

We assume `python` points to python >=3.7.

Setup a virtual env:
```

```text
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]" # [dev] installs development-only dependencies
```

#### Iteration

* Iterate on the code locally
* Run tests via `pytest -s unit_tests`
* Perform static type checks using `mypy airbyte_cdk`. `MyPy` configuration is in `.mypy.ini`.
* The `type_check_and_test.sh` script bundles both type checking and testing in one convenient command. Feel free to use it!


#### Testing
All tests are located in the `unit_tests` directory. Run `pytest --cov=airbyte_cdk unit_tests/` to run them.
This also presents a test coverage report.


All tests are located in the `unit_tests` directory. Run `pytest --cov=airbyte_cdk unit_tests/` to run them. This also presents a test coverage report.

#### Publishing a new version to PyPi

1. Bump the package version in `setup.py`
2. Open a PR
3. An Airbyte member must comment `/publish-cdk dry-run=<true or false>`. Dry runs publish to test.pypi.org. Not including the `dry-run` parameter defaults to a value of `true`.
3. An Airbyte member must comment `/publish-cdk --dry-run=<true or false>`. Dry runs publish to test.pypi.org.

## Coming Soon
* Full OAuth 2.0 support (including refresh token issuing flow via UI or CLI)

* Full OAuth 2.0 support \(including refresh token issuing flow via UI or CLI\)
* Airbyte Java HTTP CDK
* CDK for Async HTTP endpoints (request-poll-wait style endpoints)
* CDK for Async HTTP endpoints \(request-poll-wait style endpoints\)
* CDK for other protocols
* General CDK for Destinations
* Don't see a feature you need? [Create an issue and let us know how we can help!](github.com/airbytehq/airbyte/issues/new/choose)
* Don't see a feature you need? [Create an issue and let us know how we can help!](https://github.com/airbytehq/airbyte/tree/184dab77ebfbc00c69eea9e34b7db29c79a9e6d1/airbyte-cdk/python/github.com/airbytehq/airbyte/issues/new/choose/README.md)
2 changes: 1 addition & 1 deletion airbyte-cdk/python/docs/concepts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Having trouble figuring out how to write a `stream_slices` function or aren't su
Airbyte recommends using the CDK template generator to develop with the CDK. The template generates
created all the required scaffolding, with convenient TODOs, allowing developers to truly focus on
implementing the API.

For tips on useful Python knowledge, see the [Python Concepts](./python-concepts.md) page.

You can find a complete tutorial for implementing an HTTP source connector in [this tutorial](../tutorials/http_api_source.md)
Expand Down
5 changes: 0 additions & 5 deletions docs/contributing-to-airbyte/python/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,6 @@ The CDK provides an improved developer experience by providing basic implementat

This document is a general introduction to the CDK. Readers should have basic familiarity with the [Airbyte Specification](https://docs.airbyte.io/architecture/airbyte-specification) before proceeding.

{% hint style="info" %}
The CDK currently does not support creating destinations, but it will very soon.
{% endhint %}

## Getting Started

Generate an empty connector using the code generator. First clone the Airbyte repository then from the repository root run
Expand Down Expand Up @@ -86,6 +82,5 @@ All tests are located in the `unit_tests` directory. Run `pytest --cov=airbyte_c
* Airbyte Java HTTP CDK
* CDK for Async HTTP endpoints \(request-poll-wait style endpoints\)
* CDK for other protocols
* General CDK for Destinations
* Don't see a feature you need? [Create an issue and let us know how we can help!](https://github.com/airbytehq/airbyte/tree/184dab77ebfbc00c69eea9e34b7db29c79a9e6d1/airbyte-cdk/python/github.com/airbytehq/airbyte/issues/new/choose/README.md)

0 comments on commit 9d8b173

Please sign in to comment.