🎉 Source Snapchat Marketing: support oauth #7811

annalvova05 · 2021-11-10T10:02:00Z

What

Add oauth2.0 flow in Python part

How

Change spec.json

Pre-merge Checklist

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Secrets in the connector's spec are annotated with airbyte_secret
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Code reviews completed
Documentation updated
- Connector's README.md
- Connector's bootstrap.md. See description and examples
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
Credentials added to Github CI. Instructions.
/test connector=connectors/<name> command is passing.
New Connector version released on Dockerhub by running the /publish command described here
After the new connector version is published, connector version bumped in the seed directory as described here
Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

…t-oauth

annalvova05 · 2021-11-10T13:20:59Z

/test connector=connectors/source-snapchat-marketing

🕑 connectors/source-snapchat-marketing https://github.com/airbytehq/airbyte/actions/runs/1444334846
✅ connectors/source-snapchat-marketing https://github.com/airbytehq/airbyte/actions/runs/1444334846
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        75      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       38     27    29%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 62     25    60%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  896    440    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                       Stmts   Miss  Cover
	 --------------------------------------------------------------
	 base_python/__init__.py                       13      0   100%
	 base_python/catalog_helpers.py                10      6    40%
	 base_python/cdk/__init__.py                    0      0   100%
	 base_python/cdk/abstract_source.py            83     59    29%
	 base_python/cdk/streams/__init__.py            0      0   100%
	 base_python/cdk/streams/auth/__init__.py       0      0   100%
	 base_python/cdk/streams/auth/core.py           8      1    88%
	 base_python/cdk/streams/auth/jwt.py            5      5     0%
	 base_python/cdk/streams/auth/oauth.py         37     26    30%
	 base_python/cdk/streams/auth/token.py          9      4    56%
	 base_python/cdk/streams/core.py               63     32    49%
	 base_python/cdk/streams/exceptions.py         10      2    80%
	 base_python/cdk/streams/http.py               67     33    51%
	 base_python/cdk/streams/rate_limiting.py      30     14    53%
	 base_python/cdk/utils/__init__.py              0      0   100%
	 base_python/cdk/utils/casing.py                4      0   100%
	 base_python/client.py                         56     33    41%
	 base_python/entrypoint.py                     70     56    20%
	 base_python/integration.py                    52     25    52%
	 base_python/logger.py                         33     19    42%
	 base_python/schema_helpers.py                 56     41    27%
	 base_python/source.py                         51     34    33%
	 main_dev.py                                    3      3     0%
	 --------------------------------------------------------------
	 TOTAL                                        660    393    40%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                    Stmts   Miss  Cover
	 -----------------------------------------------------------
	 source_snapchat_marketing/__init__.py       2      0   100%
	 source_snapchat_marketing/source.py       137     31    77%
	 -----------------------------------------------------------
	 TOTAL                                     139     31    78%

…t-oauth # Conflicts: # docs/integrations/sources/snapchat-marketing.md

…t-oauth

sherifnada · 2021-11-11T02:54:21Z

airbyte-integrations/connectors/source-snapchat-marketing/source_snapchat_marketing/source.py

-            return {self.cursor_field: self.max_state if self.current_slice == self.last_slice else self.initial_state}
+        if not current_stream_state:
+            current_stream_state = {self.cursor_field: self.start_date}
+        return {self.cursor_field: max(latest_record.get(self.cursor_field, ""), current_stream_state.get(self.cursor_field, ""))}


does max work with empty string?

yes. In case,
current_stream_state: {'updated_at': '2021-06-11T09:44:42.583Z'} and
latest_record.get(self.cursor_field, "") = "" max returns '2021-06-11T09:44:42.583Z'

…t-oauth

annalvova05 · 2021-11-18T12:37:04Z

/test connector=connectors/source-snapchat-marketing

🕑 connectors/source-snapchat-marketing https://github.com/airbytehq/airbyte/actions/runs/1476422615
✅ connectors/source-snapchat-marketing https://github.com/airbytehq/airbyte/actions/runs/1476422615
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        75      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       38     27    29%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 62     25    60%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  896    440    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                    Stmts   Miss  Cover
	 -----------------------------------------------------------
	 source_snapchat_marketing/__init__.py       2      0   100%
	 source_snapchat_marketing/source.py       137     31    77%
	 -----------------------------------------------------------
	 TOTAL                                     139     31    78%

annalvova05 · 2021-11-18T12:38:23Z

Oauth2.0 flow is successful

annalvova05 · 2021-11-19T09:59:31Z

/test connector=connectors/source-snapchat-marketing

🕑 connectors/source-snapchat-marketing https://github.com/airbytehq/airbyte/actions/runs/1480469246
✅ connectors/source-snapchat-marketing https://github.com/airbytehq/airbyte/actions/runs/1480469246
Python tests coverage:

	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                                 Stmts   Miss  Cover
	 ------------------------------------------------------------------------
	 source_acceptance_test/__init__.py                       2      0   100%
	 source_acceptance_test/base.py                          10      4    60%
	 source_acceptance_test/config.py                        75      8    89%
	 source_acceptance_test/conftest.py                     108    108     0%
	 source_acceptance_test/plugin.py                        47     47     0%
	 source_acceptance_test/tests/__init__.py                 4      0   100%
	 source_acceptance_test/tests/test_core.py              200     94    53%
	 source_acceptance_test/tests/test_full_refresh.py       38     27    29%
	 source_acceptance_test/tests/test_incremental.py        69     38    45%
	 source_acceptance_test/utils/__init__.py                 6      0   100%
	 source_acceptance_test/utils/asserts.py                 37      2    95%
	 source_acceptance_test/utils/common.py                  41     24    41%
	 source_acceptance_test/utils/compare.py                 62     25    60%
	 source_acceptance_test/utils/connector_runner.py        82     49    40%
	 source_acceptance_test/utils/json_schema_helper.py     115     14    88%
	 ------------------------------------------------------------------------
	 TOTAL                                                  896    440    51%
	 ---------- coverage: platform linux, python 3.8.10-final-0 -----------
	 Name                                    Stmts   Miss  Cover
	 -----------------------------------------------------------
	 source_snapchat_marketing/__init__.py       2      0   100%
	 source_snapchat_marketing/source.py       137     31    77%
	 -----------------------------------------------------------
	 TOTAL                                     139     31    78%

annalvova05 · 2021-11-19T10:08:00Z

/publish connector=connectors/source-snapchat-marketing

🕑 connectors/source-snapchat-marketing https://github.com/airbytehq/airbyte/actions/runs/1480502062
✅ connectors/source-snapchat-marketing https://github.com/airbytehq/airbyte/actions/runs/1480502062

* add python oauth part * bump version * remove old creds * bump version * change spec * bump version * bump version * bump version

annalvova05 added 2 commits November 10, 2021 11:52

add python oauth part

3094c9e

Merge remote-tracking branch 'origin/master' into alvova/7482-snapcha…

324bc33

…t-oauth

annalvova05 requested review from midavadim and lazebnyi November 10, 2021 10:02

annalvova05 self-assigned this Nov 10, 2021

github-actions bot added the area/connectors Connector related issues label Nov 10, 2021

bump version

095e1a8

github-actions bot added the area/documentation Improvements or additions to documentation label Nov 10, 2021

annalvova05 temporarily deployed to more-secrets November 10, 2021 10:09 Inactive

remove old creds

01e0fbc

annalvova05 temporarily deployed to more-secrets November 10, 2021 13:21 Inactive

jrhizor temporarily deployed to more-secrets November 10, 2021 13:23 Inactive

annalvova05 added 2 commits November 10, 2021 15:26

Merge remote-tracking branch 'origin/master' into alvova/7482-snapcha…

32bc78b

…t-oauth # Conflicts: # docs/integrations/sources/snapchat-marketing.md

bump version

2d95ec7

annalvova05 temporarily deployed to more-secrets November 10, 2021 13:30 Inactive

annalvova05 linked an issue Nov 10, 2021 that may be closed by this pull request

Source Snapchat Marketing: support oauth #7482

Closed

annalvova05 added 2 commits November 11, 2021 17:16

Merge remote-tracking branch 'origin/master' into alvova/7482-snapcha…

187cb36

…t-oauth

Merge remote-tracking branch 'origin/master' into alvova/7482-snapcha…

d2e767f

…t-oauth

midavadim approved these changes Nov 15, 2021

View reviewed changes

sherifnada reviewed Nov 15, 2021

View reviewed changes

annalvova05 added 2 commits November 17, 2021 14:27

change spec

31d89f5

Merge remote-tracking branch 'origin/master' into alvova/7482-snapcha…

042d9fb

…t-oauth

annalvova05 temporarily deployed to more-secrets November 17, 2021 12:28 Inactive

bump version

2dc254f

annalvova05 temporarily deployed to more-secrets November 18, 2021 11:20 Inactive

annalvova05 requested a review from sherifnada November 18, 2021 12:30

jrhizor temporarily deployed to more-secrets November 18, 2021 12:39 Inactive

sherifnada approved these changes Nov 18, 2021

View reviewed changes

bump version

062636e

annalvova05 temporarily deployed to more-secrets November 19, 2021 10:00 Inactive

jrhizor temporarily deployed to more-secrets November 19, 2021 10:01 Inactive

jrhizor temporarily deployed to more-secrets November 19, 2021 10:10 Inactive

bump version

9633d4c

annalvova05 merged commit 3226b22 into master Nov 19, 2021

annalvova05 deleted the alvova/7482-snapchat-oauth branch November 19, 2021 10:20

annalvova05 temporarily deployed to more-secrets November 19, 2021 10:21 Inactive

This was referenced Nov 19, 2021

Bump Airbyte version from 0.32.4-alpha to 0.32.5-alpha #8144

Closed

Bump Airbyte version from 0.32.4-alpha to 0.32.5-alpha #8147

Closed

Bump Airbyte version from 0.32.4-alpha to 0.32.5-alpha #8153

Merged

schlattk pushed a commit to schlattk/airbyte that referenced this pull request Jan 4, 2022

🎉 Source Snapchat Marketing: support oauth (airbytehq#7811)

c642fe3

* add python oauth part * bump version * remove old creds * bump version * change spec * bump version * bump version * bump version

lazebnyi removed their request for review May 30, 2022 10:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🎉 Source Snapchat Marketing: support oauth #7811

🎉 Source Snapchat Marketing: support oauth #7811

annalvova05 commented Nov 10, 2021 •

edited

Loading

annalvova05 commented Nov 10, 2021 •

edited by github-actions bot

Loading

sherifnada Nov 11, 2021

annalvova05 Nov 18, 2021 •

edited

Loading

annalvova05 commented Nov 18, 2021 •

edited by github-actions bot

Loading

annalvova05 commented Nov 18, 2021

annalvova05 commented Nov 19, 2021 •

edited by github-actions bot

Loading

annalvova05 commented Nov 19, 2021 •

edited by github-actions bot

Loading

🎉 Source Snapchat Marketing: support oauth #7811

🎉 Source Snapchat Marketing: support oauth #7811

Conversation

annalvova05 commented Nov 10, 2021 • edited Loading

What

How

Recommended reading order

Pre-merge Checklist

Community member or Airbyter

Airbyter

annalvova05 commented Nov 10, 2021 • edited by github-actions bot Loading

sherifnada Nov 11, 2021

Choose a reason for hiding this comment

annalvova05 Nov 18, 2021 • edited Loading

Choose a reason for hiding this comment

annalvova05 commented Nov 18, 2021 • edited by github-actions bot Loading

annalvova05 commented Nov 18, 2021

annalvova05 commented Nov 19, 2021 • edited by github-actions bot Loading

annalvova05 commented Nov 19, 2021 • edited by github-actions bot Loading

annalvova05 commented Nov 10, 2021 •

edited

Loading

annalvova05 commented Nov 10, 2021 •

edited by github-actions bot

Loading

annalvova05 Nov 18, 2021 •

edited

Loading

annalvova05 commented Nov 18, 2021 •

edited by github-actions bot

Loading

annalvova05 commented Nov 19, 2021 •

edited by github-actions bot

Loading

annalvova05 commented Nov 19, 2021 •

edited by github-actions bot

Loading