Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🎉 octavia-cli: Add ability to import existing resources #14137

Merged
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
aa1ff16
init
danieldiamond May 26, 2022
19a52a2
Update docs and add tests
danieldiamond May 27, 2022
0150ae9
Revert docker run docs
danieldiamond May 27, 2022
7748d63
Add "update" commands
danieldiamond May 29, 2022
58303ea
Add "update sources" and "update destinations" to create yaml files f…
danieldiamond May 29, 2022
d66efbc
Merge branch 'master' into danieldiamond/octavia-cli-generate-existin…
danieldiamond Jun 24, 2022
48676c1
WIP
danieldiamond Jun 24, 2022
89549dc
Revert "WIP"
danieldiamond Jun 24, 2022
87f6b48
Remove update method
danieldiamond Jun 24, 2022
a41fd9b
Merge branch 'master' into danieldiamond/octavia-cli-generate-existin…
danieldiamond Jun 24, 2022
29c3a64
Remove update from tests
danieldiamond Jun 24, 2022
35d9cdd
Remove update related methods in generate_renderers
danieldiamond Jun 24, 2022
a4b03ed
Merge branch 'master' into danieldiamond/octavia-cli-generate-existin…
danieldiamond Jun 26, 2022
528afc1
Reformat get commands
danieldiamond Jun 26, 2022
e376f4e
Remove DuplicateResourceError from apply
danieldiamond Jun 26, 2022
3d96529
Remove ConnectorRenderer from generate
danieldiamond Jun 26, 2022
6f3944d
Update unit tests for commands and resources
danieldiamond Jun 26, 2022
c44903e
Update docs and setup
danieldiamond Jun 26, 2022
c31f2bc
init import commands
danieldiamond Jun 26, 2022
810b089
Add quotes to handle yaml characters
danieldiamond Jun 26, 2022
f147e27
Add source and destination commands
danieldiamond Jun 26, 2022
18f6967
Remove ConnectorRenderer from generate
danieldiamond Jun 26, 2022
3ec8ce3
Merge branch 'master' into danieldiamond/octavia-cli-import
alafanechere Jun 27, 2022
9cc897e
Remove commands for import of multiple resources
danieldiamond Jun 27, 2022
da2e0b5
Improve writes: Avoid intermediate yaml file saved to disk, check if …
danieldiamond Jun 27, 2022
42d37a5
Manage connections and store state
danieldiamond Jun 27, 2022
a55547d
Make resource_requirements an optional field in apply.resources
danieldiamond Jun 28, 2022
654e3a3
Slugify resource_name in get_output_path method in generate.renderers
danieldiamond Jun 28, 2022
108d8a4
Add docstring to import commands
danieldiamond Jun 28, 2022
e1dc6fd
Merge branch 'master' into danieldiamond/octavia-cli-import
danieldiamond Jun 28, 2022
57371e3
Add user warning message to update secrets in source or destination
danieldiamond Jun 28, 2022
8779a00
add slugify dependency
alafanechere Jun 29, 2022
678f8db
write unit tests
alafanechere Jun 29, 2022
cbd183b
wip integration tests
alafanechere Jun 29, 2022
938eef0
fix connection config for integration tests
alafanechere Jun 30, 2022
305bafe
improve connection config cleaning on import
alafanechere Jun 30, 2022
efaa21f
finish integration tests
alafanechere Jun 30, 2022
fa69f09
Merge branch 'master' into danieldiamond/octavia-cli-import
alafanechere Jun 30, 2022
488dc01
update readme
alafanechere Jun 30, 2022
b62796c
update readme
alafanechere Jun 30, 2022
e38c422
update readme
alafanechere Jun 30, 2022
1e26425
update readme
alafanechere Jun 30, 2022
269ffdc
update readme
alafanechere Jun 30, 2022
470824b
Merge branch 'master' into danieldiamond/octavia-cli-import
danieldiamond Jun 30, 2022
3c359fe
Iterate on import_configuration in ConnectionRenderer
danieldiamond Jun 30, 2022
36904d2
fix docstring + clean
alafanechere Jul 1, 2022
2c7dfbb
bump to the right version
alafanechere Jul 1, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
WIP
  • Loading branch information
danieldiamond committed Jun 24, 2022
commit 48676c13b7d670873860edae26bb9fd48ebae1ce
4 changes: 2 additions & 2 deletions octavia-cli/octavia_cli/entrypoint.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,15 @@
from .check_context import check_api_health, check_is_initialized, check_workspace_exists
from .generate import commands as generate_commands
from .get import commands as get_commands
from .import_commands import commands as import_commands
from .init import commands as init_commands
from .list import commands as list_commands
from .telemetry import TelemetryClient, build_user_agent
from .update import commands as update_commands

AVAILABLE_COMMANDS: List[click.Command] = [
list_commands._list,
get_commands.get,
update_commands.update,
import_commands.import_commands,
init_commands.init,
generate_commands.generate,
apply_commands.apply,
Expand Down
2 changes: 1 addition & 1 deletion octavia-cli/octavia_cli/get/commands.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright (c) 2021 Airbyte, Inc., all rights reserved.
# Copyright (c) 2022 Airbyte, Inc., all rights reserved.
#

from typing import List
Expand Down
2 changes: 1 addition & 1 deletion octavia-cli/octavia_cli/get/resources.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright (c) 2021 Airbyte, Inc., all rights reserved.
# Copyright (c) 2022 Airbyte, Inc., all rights reserved.
#

import abc
Expand Down
3 changes: 3 additions & 0 deletions octavia-cli/octavia_cli/import_commands/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#
# Copyright (c) 2022 Airbyte, Inc., all rights reserved.
#
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright (c) 2021 Airbyte, Inc., all rights reserved.
# Copyright (c) 2022 Airbyte, Inc., all rights reserved.
#

from typing import List
Expand All @@ -13,9 +13,9 @@
from octavia_cli.list.listings import Destinations, Sources


@click.group("update", help="Update configurations in a YAML spec for a source, destination or a connection.")
@click.group("import", help="Import configurations in a YAML spec for a source, destination or a connection.")
@click.pass_context
def update(ctx: click.Context):
def import_commands(ctx: click.Context):
pass


Expand All @@ -30,7 +30,7 @@ def get_resource_definition_id(resource_type, api_client, workspace_id, resource
return config[f"{resource_type.name}_definition_id"]


def update_resource(resource_type, api_client, workspace_id, resource_id):
def import_resource(resource_type, api_client, workspace_id, resource_id):
resource = resource_type(api_client, workspace_id, resource_id)
definition_type = resource_type.name
config = resource.get_config()
Expand All @@ -42,18 +42,18 @@ def update_resource(resource_type, api_client, workspace_id, resource_id):
renderer = ConnectorSpecificationRenderer(resource_name, definition)

output_path = renderer.update_configuration(project_path=".", configuration=config["connection_configuration"])
message = f"✅ - Updated the {resource_type.name} template for {resource_name} in {output_path}."
message = f"✅ - Imported the {resource_type.name} template for {resource_name} in {output_path}."
click.echo(click.style(message, fg="green"))


@update.command(cls=OctaviaCommand, name="source", help="Get YAML for a source")
@import_commands.command(cls=OctaviaCommand, name="source", help="Get YAML for a source")
@click.argument("resource_id", type=click.STRING)
@click.pass_context
def source(ctx: click.Context, resource_id: str):
update_resource(Source, ctx.obj["API_CLIENT"], ctx.obj["WORKSPACE_ID"], resource_id)
import_resource(Source, ctx.obj["API_CLIENT"], ctx.obj["WORKSPACE_ID"], resource_id)


@update.command(cls=OctaviaCommand, name="sources", help="Get YAML for a source")
@import_commands.command(cls=OctaviaCommand, name="sources", help="Get YAML for a source")
@click.pass_context
def sources(ctx: click.Context):
api_client = ctx.obj["API_CLIENT"]
Expand All @@ -65,10 +65,10 @@ def sources(ctx: click.Context):

generate_source_or_destination("source", api_client, workspace_id, definition_id, source_name)

update_resource(Source, api_client, workspace_id, source_id)
import_resource(Source, api_client, workspace_id, source_id)


@update.command(cls=OctaviaCommand, name="destinations", help="Get YAML for a destination")
@import_commands.command(cls=OctaviaCommand, name="destinations", help="Get YAML for a destination")
@click.pass_context
def destinations(ctx: click.Context):
api_client = ctx.obj["API_CLIENT"]
Expand All @@ -80,29 +80,29 @@ def destinations(ctx: click.Context):

generate_source_or_destination("destination", api_client, workspace_id, definition_id, destination_name)

update_resource(Destination, api_client, workspace_id, destination_id)
import_resource(Destination, api_client, workspace_id, destination_id)


@update.command(cls=OctaviaCommand, name="destination", help="Get YAML for a destination")
@import_commands.command(cls=OctaviaCommand, name="destination", help="Get YAML for a destination")
@click.argument("resource_id", type=click.STRING)
@click.pass_context
def destination(ctx: click.Context, resource_id: str):
update_resource(Destination, ctx.obj["API_CLIENT"], ctx.obj["WORKSPACE_ID"], resource_id)
import_resource(Destination, ctx.obj["API_CLIENT"], ctx.obj["WORKSPACE_ID"], resource_id)


@update.command(cls=OctaviaCommand, name="connection", help="Get YAML for a connection")
@import_commands.command(cls=OctaviaCommand, name="connection", help="Get YAML for a connection")
@click.argument("resource_id", type=click.STRING)
@click.pass_context
def connection(ctx: click.Context, resource_id: str):
update_resource(Connection, ctx.obj["API_CLIENT"], ctx.obj["WORKSPACE_ID"], resource_id)
import_resource(Connection, ctx.obj["API_CLIENT"], ctx.obj["WORKSPACE_ID"], resource_id)


AVAILABLE_COMMANDS: List[click.Command] = [source, destination, connection]


def add_commands_to_list():
for command in AVAILABLE_COMMANDS:
update.add_command(command)
import_commands.add_command(command)


add_commands_to_list()
3 changes: 0 additions & 3 deletions octavia-cli/octavia_cli/update/__init__.py

This file was deleted.

2 changes: 1 addition & 1 deletion octavia-cli/unit_tests/test_entrypoint.py
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,7 @@ def test_available_commands():
assert entrypoint.AVAILABLE_COMMANDS == [
entrypoint.list_commands._list,
entrypoint.get_commands.get,
entrypoint.update_commands.update,
entrypoint.import_commands._import,
entrypoint.init_commands.init,
entrypoint.generate_commands.generate,
entrypoint.apply_commands.apply,
Expand Down
2 changes: 1 addition & 1 deletion octavia-cli/unit_tests/test_get/test_commands.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright (c) 2021 Airbyte, Inc., all rights reserved.
# Copyright (c) 2022 Airbyte, Inc., all rights reserved.
#

import pytest
Expand Down
2 changes: 1 addition & 1 deletion octavia-cli/unit_tests/test_get/test_resources.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright (c) 2021 Airbyte, Inc., all rights reserved.
# Copyright (c) 2022 Airbyte, Inc., all rights reserved.
#

import pytest
Expand Down