Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't drain protected services #1394

Merged
merged 2 commits into from
Sep 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENT_VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
v0.4.45
v0.4.46
1 change: 1 addition & 0 deletions assets/src/generated/graphql-plural.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3397,6 +3397,7 @@ export type RootMutationTypeLinkPublisherArgs = {


export type RootMutationTypeLoginArgs = {
captcha?: InputMaybe<Scalars['String']['input']>;
deviceToken?: InputMaybe<Scalars['String']['input']>;
email: Scalars['String']['input'];
password: Scalars['String']['input'];
Expand Down
2 changes: 2 additions & 0 deletions assets/src/generated/graphql.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7492,6 +7492,8 @@ export type ServiceTemplateAttributes = {
name?: InputMaybe<Scalars['String']['input']>;
/** the namespace for this service (optional for managed namespaces) */
namespace?: InputMaybe<Scalars['String']['input']>;
/** whether to protect this templated service from deletion */
protect?: InputMaybe<Scalars['Boolean']['input']>;
/** the id of a repository to source manifests for this service */
repositoryId?: InputMaybe<Scalars['ID']['input']>;
/** attributes to configure sync settings for this service */
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -449,6 +449,10 @@ spec:
description: Namespace the namespace for this service (optional
for managed namespaces)
type: string
protect:
description: Whether to protect this service from deletion. Protected
services are also not drained on cluster deletion.
type: boolean
repositoryRef:
description: |-
ObjectReference contains enough information to let you inspect or modify the referred object.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -366,6 +366,10 @@ spec:
description: Namespace the namespace for this service (optional
for managed namespaces)
type: string
protect:
description: Whether to protect this service from deletion. Protected
services are also not drained on cluster deletion.
type: boolean
repositoryRef:
description: |-
ObjectReference contains enough information to let you inspect or modify the referred object.
Expand Down
2 changes: 2 additions & 0 deletions go/client/models_gen.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions go/controller/api/v1alpha1/managednamespace_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,11 @@ type ServiceTemplate struct {
Templated *bool `json:"templated,omitempty"`
// +kubebuilder:validation:Optional
RepositoryRef *corev1.ObjectReference `json:"repositoryRef"`

// Whether to protect this service from deletion. Protected services are also not drained on cluster deletion.
// +kubebuilder:validation:Optional
Protect *bool `json:"protect,omitempty"`

// a list of context ids to add to this service
// +kubebuilder:validation:Optional
Contexts []string `json:"contexts,omitempty"`
Expand Down
5 changes: 5 additions & 0 deletions go/controller/api/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -449,6 +449,10 @@ spec:
description: Namespace the namespace for this service (optional
for managed namespaces)
type: string
protect:
description: Whether to protect this service from deletion. Protected
services are also not drained on cluster deletion.
type: boolean
repositoryRef:
description: |-
ObjectReference contains enough information to let you inspect or modify the referred object.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -366,6 +366,10 @@ spec:
description: Namespace the namespace for this service (optional
for managed namespaces)
type: string
protect:
description: Whether to protect this service from deletion. Protected
services are also not drained on cluster deletion.
type: boolean
repositoryRef:
description: |-
ObjectReference contains enough information to let you inspect or modify the referred object.
Expand Down
1 change: 1 addition & 0 deletions go/controller/docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -2189,6 +2189,7 @@ _Appears in:_
| `namespace` _string_ | Namespace the namespace for this service (optional for managed namespaces) | | Optional: {} <br /> |
| `templated` _boolean_ | | | Optional: {} <br /> |
| `repositoryRef` _[ObjectReference](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.29/#objectreference-v1-core)_ | | | Optional: {} <br /> |
| `protect` _boolean_ | Whether to protect this service from deletion. Protected services are also not drained on cluster deletion. | | Optional: {} <br /> |
| `contexts` _string array_ | a list of context ids to add to this service | | Optional: {} <br /> |
| `git` _[GitRef](#gitref)_ | Git settings to configure git for a service | | Optional: {} <br /> |
| `helm` _[ServiceHelm](#servicehelm)_ | Helm settings to configure helm for a service | | Optional: {} <br /> |
Expand Down
1 change: 1 addition & 0 deletions go/controller/internal/controller/common.go
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ func genServiceTemplate(ctx context.Context, c runtimeclient.Client, namespace s
Namespace: srv.Namespace,
Templated: lo.ToPtr(true),
RepositoryID: repositoryID,
Protect: srv.Protect,
SyncConfig: syncConf,
}
if len(srv.Dependencies) > 0 {
Expand Down
11 changes: 8 additions & 3 deletions lib/console/deployments/global.ex
Original file line number Diff line number Diff line change
Expand Up @@ -465,6 +465,7 @@ defmodule Console.Deployments.Global do
owner_id: owner_id,
configuration: Enum.map(Map.merge(dest_secrets, source_secrets), fn {k, v} -> %{name: k, value: v} end),
repository_id: source.repository_id,
protect: source.protect,
sync_config: clean(source.sync_config),
git: clean(source.git),
helm: clean(source.helm),
Expand Down Expand Up @@ -510,8 +511,8 @@ defmodule Console.Deployments.Global do
dest = Repo.preload(dest, [:dependencies])
with {:ok, source_secrets} <- configuration(spec),
{:ok, dest_secrets} <- Services.configuration(dest),
do: (spec.repository_id != dest.repository_id || spec.templated != dest.templated ||
specs_different?(spec, dest) || !contexts_equal?(spec, dest) || !deps_equal?(spec, dest) ||
do: (fields_different?(spec, dest) || specs_different?(spec, dest) ||
!contexts_equal?(spec, dest) || !deps_equal?(spec, dest) ||
missing_source?(source_secrets, dest_secrets))
end

Expand All @@ -526,7 +527,7 @@ defmodule Console.Deployments.Global do
def diff?(_, _), do: false

defp diff?(%Service{} = s, %Service{} = d, source, dest) do
missing_source?(source, dest) || !deps_equal?(s, d) || specs_different?(s, d) || s.repository_id != d.repository_id || s.namespace != d.namespace || s.templated != d.templated
missing_source?(source, dest) || !deps_equal?(s, d) || specs_different?(s, d) || fields_different?(s, d)
end

defp ensure_revision(%ServiceTemplate{} = template, config) do
Expand Down Expand Up @@ -572,6 +573,10 @@ defmodule Console.Deployments.Global do
|> MapSet.equal?(MapSet.new(deps || [], & &1.name))
end

defp fields_different?(svc1, svc2) do
Enum.any?(~w(repository_id namespace templated protect)a, & Map.get(svc1, &1) != Map.get(svc2, &1))
end

@spec svc_deps([ServiceDependency.t], [ServiceDependency.t]) :: [ServiceDependency.t]
defp svc_deps(dependencies, existing) do
existing_by_name = Map.new(existing, & {&1.name, &1})
Expand Down
2 changes: 1 addition & 1 deletion lib/console/deployments/pr/validation.ex
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ defmodule Console.Deployments.Pr.Validation do
end
end

defp do_validate(%Configuration{type: :string}, val) when is_binary(val), do: :ok
defp do_validate(%Configuration{type: :string}, val) when is_binary(val) and byte_size(val) > 0, do: :ok
defp do_validate(%Configuration{type: t}, val),
do: {:error, "value #{inspect(val)} does not match type #{String.upcase(to_string(t))}"}
end
1 change: 1 addition & 0 deletions lib/console/graphql/deployments/global.ex
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ defmodule Console.GraphQl.Deployments.Global do
field :name, :string, description: "the name for this service (optional for managed namespaces)"
field :namespace, :string, description: "the namespace for this service (optional for managed namespaces)"
field :templated, :boolean
field :protect, :boolean, description: "whether to protect this templated service from deletion"
field :repository_id, :id, description: "the id of a repository to source manifests for this service"
field :contexts, list_of(:id), description: "a list of context ids to add to this service"
field :configuration, list_of(:config_attributes), description: "a list of secure configuration that will be added to any services created by this template"
Expand Down
16 changes: 14 additions & 2 deletions lib/console/graphql/deployments/stack.ex
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,13 @@ defmodule Console.GraphQl.Deployments.Stack do
field :paused, :boolean, description: "whether the stack is actively tracking changes in git"
field :status, non_null(:stack_status), description: "The status of the last run of the stack"
field :job_spec, :job_gate_spec, description: "optional k8s job configuration for the job that will apply this stack"
field :configuration, non_null(:stack_configuration), description: "version/image config for the tool you're using"

@desc "version/image config for the tool you're using"
field :configuration, non_null(:stack_configuration), resolve: fn
%{configuration: %{} = conf}, _, _ -> {:ok, conf}
_, _, _ -> {:ok, %{hooks: []}}
end

field :approval, :boolean, description: "whether to require approval"
field :deleted_at, :datetime, description: "whether this stack was previously deleted and is pending cleanup"
field :cancellation_reason, :string, description: "why this run was cancelled"
Expand Down Expand Up @@ -229,7 +235,13 @@ defmodule Console.GraphQl.Deployments.Stack do
field :job_spec, :job_gate_spec,
description: "optional k8s job configuration for the job that will apply this stack",
resolve: &Deployments.job_spec/3
field :configuration, non_null(:stack_configuration), description: "version/image config for the tool you're using"

@desc "version/image config for the tool you're using"
field :configuration, non_null(:stack_configuration), resolve: fn
%{configuration: %{} = conf}, _, _ -> {:ok, conf}
_, _, _ -> {:ok, %{hooks: []}}
end

field :approval, :boolean, description: "whether to require approval"
field :message, :string, description: "the commit message"
field :approved_at, :datetime, description: "when this run was approved"
Expand Down
11 changes: 7 additions & 4 deletions lib/console/mailer.ex
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ defmodule Console.Mailer do
:ok
else
{:error, err} ->
Logger.info "not delivering email, reason: #{err}"
Logger.info "not delivering email, reason: #{inspect(err)}"
{:error, err}
end
end
Expand All @@ -24,11 +24,14 @@ defmodule Console.Mailer do
end
end

defp smtp_config() do
def smtp_config() do
case Settings.cached() do
%DeploymentSettings{smtp: %DeploymentSettings.SMTP{} = smtp} ->
{:ok, Map.take(smtp, DeploymentSettings.smtp_config())}
%DeploymentSettings{smtp: %DeploymentSettings.SMTP{} = config} -> {:ok, smtp(config)}
_ -> {:error, "smtp not configured"}
end
end

defp smtp(%DeploymentSettings.SMTP{user: u, password: p, server: s, port: port, ssl: ssl}) do
%{username: u, password: p, server: s, port: port, ssl: ssl, auth: :always}
end
end
2 changes: 1 addition & 1 deletion lib/console/schema/service.ex
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ defmodule Console.Schema.Service do
end

def drainable(query \\ __MODULE__) do
from(s in query, where: s.name != "deploy-operator")
from(s in query, where: s.name != "deploy-operator" and (is_nil(s.protect) or not s.protect))
end

def nonsystem(query \\ __MODULE__) do
Expand Down
1 change: 1 addition & 0 deletions lib/console/schema/service_template.ex
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ defmodule Console.Schema.ServiceTemplate do
field :name, :string
field :namespace, :string
field :templated, :boolean, default: true
field :protect, :boolean, default: false
field :contexts, {:array, :string}
field :configuration, {:array, :map}, virtual: true
field :ignore_sync, :boolean, virtual: true
Expand Down
2 changes: 1 addition & 1 deletion lib/console_web/views/email_view.ex
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
defmodule ConsoleWeb.EmailView do
use ConsoleWeb, :view

def url(path), do: "#{Console.conf(:host)}#{path}"
def url(path), do: Console.url(path)

def row(assigns \\ %{}, do: block), do: render_template("row.html", assigns, block)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -449,6 +449,10 @@ spec:
description: Namespace the namespace for this service (optional
for managed namespaces)
type: string
protect:
description: Whether to protect this service from deletion. Protected
services are also not drained on cluster deletion.
type: boolean
repositoryRef:
description: |-
ObjectReference contains enough information to let you inspect or modify the referred object.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -366,6 +366,10 @@ spec:
description: Namespace the namespace for this service (optional
for managed namespaces)
type: string
protect:
description: Whether to protect this service from deletion. Protected
services are also not drained on cluster deletion.
type: boolean
repositoryRef:
description: |-
ObjectReference contains enough information to let you inspect or modify the referred object.
Expand Down
2 changes: 1 addition & 1 deletion priv/notifications/secret.txt.eex
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
<%= @user.name %> shared a secret with you

It can be downloaded at a one-time url here: <%= Console.url("/secrets/share/#{@handle}") %>
It can be downloaded at a one-time url here: <%= Console.url("/secrets/#{@handle}") %>
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
defmodule Console.Repo.Migrations.AddProtectServiceTemplate do
use Ecto.Migration

def change do
alter table(:service_templates) do
add :protect, :boolean, default: false
end
end
end
3 changes: 3 additions & 0 deletions schema/schema.graphql
Original file line number Diff line number Diff line change
Expand Up @@ -1808,6 +1808,9 @@ input ServiceTemplateAttributes {

templated: Boolean

"whether to protect this templated service from deletion"
protect: Boolean

"the id of a repository to source manifests for this service"
repositoryId: ID

Expand Down
22 changes: 22 additions & 0 deletions test/console/deployments/git_test.exs
Original file line number Diff line number Diff line change
Expand Up @@ -318,6 +318,28 @@ defmodule Console.Deployments.GitTest do
assert err =~ "does not match regex"
end

test "it will reject a pull request w/ empty string configs" do
user = insert(:user)
conn = insert(:scm_connection, token: "some-pat")
pra = insert(:pr_automation,
identifier: "pluralsh/console",
cluster: build(:cluster),
connection: conn,
updates: %{regexes: ["regex"], match_strategy: :any, files: ["file.yaml"], replace_template: "replace"},
write_bindings: [%{user_id: user.id}],
create_bindings: [%{user_id: user.id}],
configuration: [
%{name: "first", type: :int},
%{name: "second", type: :string}
]
)

{:error, _} = Git.create_pull_request(%{
"first" => 10,
"second" => ""
}, pra.id, "pr-test", user)
end

test "it can create a pull request with a github app" do
user = insert(:user)
{:ok, pem_string, _} = Console.keypair("console@plural.sh")
Expand Down
7 changes: 5 additions & 2 deletions test/console/deployments/global_test.exs
Original file line number Diff line number Diff line change
Expand Up @@ -539,14 +539,17 @@ defmodule Console.Deployments.GlobalTest do
test "it returns false if targets have all the same relevant config" do
svc = insert(:service,
helm: %{chart: "test", version: "0.4.0", repository: %{name: "chart", namespace: "infra"}},
templated: true
templated: true,
protect: false,
namespace: "test"
)

template = insert(:service_template,
repository: svc.repository,
helm: %{chart: "test", version: "0.4.0", repository: %{name: "chart", namespace: "infra"}},
templated: true,
git: svc.git
git: svc.git,
namespace: "test"
)

refute Global.diff?(template, Console.Repo.preload(svc, [:contexts]))
Expand Down
Loading