Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🚨🚨 Migrate source-zendesk-talk to low code #35780

Merged
merged 47 commits into from
May 13, 2024
Merged
Show file tree
Hide file tree
Changes from 45 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
b1be44a
Add builder-generated manifest
ambirdsall Mar 1, 2024
b379423
Make schema fields nullable per json schemas
ambirdsall Mar 1, 2024
63e6402
Update object/array schemas, date-time formats to match json
ambirdsall Mar 1, 2024
4fb1c4d
Update ivr_menus and ivr_routes schemas
ambirdsall Mar 1, 2024
51734e5
Remove builder metadata from yaml
ambirdsall Mar 1, 2024
b0765de
Add non-nullable current_timestamp per json schema
ambirdsall Mar 1, 2024
9b55e09
Clean up user inputs/config specification
ambirdsall Mar 1, 2024
5c4a3f7
Fix interpolation values, get test reads working with basic auth
ambirdsall Mar 1, 2024
befaa12
Fix pagination for incremental streams
ambirdsall Mar 2, 2024
bb08a6d
Add spec.advanced_auth to manifest
ambirdsall Mar 2, 2024
4c82504
snake_case stream names
ambirdsall Mar 2, 2024
d294027
set up parallel connector definitions and test commands
ambirdsall Mar 2, 2024
97db14b
Inject current_timestamp field into single record streams
ambirdsall Mar 2, 2024
9b729eb
Fix pagination stop condition
ambirdsall Mar 3, 2024
804c52e
Custom extractors for ivr{route,menu}s; fix broken routes extraction
ambirdsall Mar 3, 2024
3114640
Add custom authenticator
ambirdsall Mar 3, 2024
66c7745
Delete python connector, support files, & scripts
ambirdsall Mar 3, 2024
33e7bd9
Automated code format fixes
ambirdsall Mar 4, 2024
a3fe02a
Require min CDK version supporting pagination fix
ambirdsall Mar 4, 2024
f087020
Add unit tests for custom components, remove source unit tests
ambirdsall Mar 4, 2024
3e54865
List oauth first to make CAT happy
ambirdsall Mar 4, 2024
cb9a551
Change all numeric types in schemas to integer
ambirdsall Mar 4, 2024
30ad95f
Specify min, not approximate, version of airbyte-cdk
ambirdsall Mar 4, 2024
2263a3d
Bump source-zendesk-talk version
ambirdsall Mar 4, 2024
d9f2ba7
Reorder spec, add missing schema fields
ambirdsall Mar 7, 2024
29247d9
Add backoff based on Retry-After header
ambirdsall Mar 7, 2024
7680b52
Fix cursor_field definition for incremental streams
ambirdsall Mar 7, 2024
1af3de5
Add pagination for non-incremental batch streams
ambirdsall Mar 7, 2024
a1d8311
Fix config value fetching for legacy_basic_auth
ambirdsall Mar 7, 2024
5017dff
Tidy up connector version declarations
ambirdsall Mar 7, 2024
a84f074
Fix CAT config for a check with bad credentials
ambirdsall Mar 8, 2024
428bc68
Update cdk tag in metadata.yaml
ambirdsall Mar 8, 2024
94c8764
Add (almost) all missing fields from json schemas
ambirdsall Mar 9, 2024
0003bee
Fix misplaced `ivr.menus[].greeting_id` schema field
ambirdsall Mar 12, 2024
a770db0
Use account_overview for check as in python source
ambirdsall Mar 12, 2024
089a0a2
Document breaking changes and migration steps
ambirdsall Mar 15, 2024
75b5f35
Merge branch 'master' into amb/migrate-source-zendesk-talk-to-low-code
ambirdsall Mar 15, 2024
e6b4a12
Document breaking changes and migration steps
ambirdsall Mar 15, 2024
87704ac
Merge master to branch
lazebnyi Apr 22, 2024
f6d5647
Updated breaking changes version
lazebnyi Apr 22, 2024
3686a25
Change breaking cahnges release date
lazebnyi Apr 22, 2024
43c5a96
Update docs/integrations/sources/zendesk-talk.md
girarda May 6, 2024
be05ce6
Update airbyte-integrations/connectors/source-zendesk-talk/metadata.yaml
girarda May 6, 2024
a9da4c4
bump to 1.0.0 because this is a breaking change
girarda May 6, 2024
897d5d4
Merge branch 'master' into amb/migrate-source-zendesk-talk-to-low-code
girarda May 6, 2024
f47fde5
Update docs/integrations/sources/zendesk-talk-migrations.md
girarda May 6, 2024
0545c49
Merge master to branch
lazebnyi May 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ acceptance_tests:
- config_path: "secrets/config.json"
status: "succeed"
- config_path: "integration_tests/invalid_config.json"
status: "exception"
status: "failed"
- config_path: "secrets/config_old.json"
status: "succeed"
discovery:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ data:
connectorSubtype: api
connectorType: source
definitionId: c8630570-086d-4a40-99ae-ea5b18673071
dockerImageTag: 0.2.0
dockerImageTag: 1.0.0
dockerRepository: airbyte/source-zendesk-talk
documentationUrl: https://docs.airbyte.com/integrations/sources/zendesk-talk
githubIssueLabel: source-zendesk-talk
Expand All @@ -30,7 +30,12 @@ data:
enabled: true
releaseStage: generally_available
supportLevel: certified
ambirdsall marked this conversation as resolved.
Show resolved Hide resolved
releases:
breakingChanges:
1.0.0:
upgradeDeadline: "2024-05-31"
message: "The source Zendesk Talk connector is being migrated from the Python CDK to our declarative low-code CDK. Due to changes to the incremental stream state message format and the removal of a nonexistent field from the ivrs stream schema, this migration constitutes a breaking change. After updating, please reset your source before resuming syncs. For more information, see our migration documentation for source Zendesk Talk."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@katmarkham can you review the breaking change banner when you get a chance? The connector will be good to go once that's done

tags:
- language:python
- cdk:python
- cdk:low-code
metadataSpecVersion: "1.0"
428 changes: 357 additions & 71 deletions airbyte-integrations/connectors/source-zendesk-talk/poetry.lock

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ requires = [ "poetry-core>=1.0.0",]
build-backend = "poetry.core.masonry.api"

[tool.poetry]
version = "0.2.0"
version = "1.0.0"
name = "source-zendesk-talk"
description = "Source implementation for Zendesk Talk."
authors = [ "Airbyte <contact@airbyte.io>",]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Copyright (c) 2023 Airbyte, Inc., all rights reserved.
ambirdsall marked this conversation as resolved.
Show resolved Hide resolved

from dataclasses import dataclass
from typing import Any, List, Mapping

import requests
from airbyte_cdk.sources.declarative.auth.declarative_authenticator import DeclarativeAuthenticator
from airbyte_cdk.sources.declarative.auth.token import BasicHttpAuthenticator, BearerAuthenticator
from airbyte_cdk.sources.declarative.extractors.record_extractor import RecordExtractor
from airbyte_cdk.sources.declarative.types import Record


@dataclass
class IVRMenusRecordExtractor(RecordExtractor):
def extract_records(self, response: requests.Response) -> List[Record]:
ivrs = response.json().get("ivrs", [])
records = []
for ivr in ivrs:
for menu in ivr.get("menus", []):
records.append({"ivr_id": ivr["id"], **menu})
return records


@dataclass
class IVRRoutesRecordExtractor(RecordExtractor):
def extract_records(self, response: requests.Response) -> List[Record]:
ivrs = response.json().get("ivrs", [])
records = []
for ivr in ivrs:
for menu in ivr.get("menus", []):
for route in menu.get("routes", []):
maxi297 marked this conversation as resolved.
Show resolved Hide resolved
records.append({"ivr_id": ivr["id"], "ivr_menu_id": menu["id"], **route})
return records


@dataclass
class ZendeskTalkAuthenticator(DeclarativeAuthenticator):
config: Mapping[str, Any]
legacy_basic_auth: BasicHttpAuthenticator
basic_auth: BasicHttpAuthenticator
oauth: BearerAuthenticator

def __new__(cls, legacy_basic_auth, basic_auth, oauth, config, *args, **kwargs):
credentials = config.get("credentials", {})
if config.get("access_token", {}) and config.get("email", {}):
return legacy_basic_auth
elif credentials["auth_type"] == "api_token":
return basic_auth
elif credentials["auth_type"] == "oauth2.0":
return oauth
else:
raise Exception(f"Missing valid authenticator for auth_type: {credentials['auth_type']}")
Loading
Loading