enterprise/providers/google: initial account sync to google workspace (…

…#9384)

* providers/google: initial account sync to google workspace

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* start separating scim sync client

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* generalize more...ish

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* set dispatch_uid

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* start generalizing task

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fully separate tasks

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix more

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix signals...?

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* start google dedupe

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* drawing the rest of the owl

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* more

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* juse use a whole lot less magic

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* member sync, better implement conflict/retry-able exceptions

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* max wizards taller

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* gen api, basic UI

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix some bugs

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix a bunch more bugs

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* generalize sync status API

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* rework sync chart

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* add slugify to evaluator

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* add test property mappings

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* rename to google workspace

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* handle existing objects

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix credential render

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* verify email has correct domain before syncing user

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix missing docstring

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix lock not being used

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* abstract more common stuff away

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* backport time limit fix

#9546
Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* start discovery

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* implement discover for google

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* prevent same issue as with #9557

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix sync status

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* make group name unique in API

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix reference to old wrapper

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* start adding tests

man this api client is awful

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* add SkipObject

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* dont use weak ref

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* add group tests

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* add user and group delete options

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* set user agent

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* if the api's testing tools are awful, let's just make our own

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* add more tests and already fix some more bugs

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* add discover

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* add preview banner

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* add group import test

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* only import users/groups in the correct parent group

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix conflicting args

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix missing schedule

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* fix web ui

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

* add default_group_email_domain

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

---------

Signed-off-by: Jens Langhammer <jens@goauthentik.io>

authentik/core/api/groups.py

@@ -17,6 +17,7 @@
 from rest_framework.request import Request
 from rest_framework.response import Response
 from rest_framework.serializers import ListSerializer, ModelSerializer, ValidationError
+from rest_framework.validators import UniqueValidator
 from rest_framework.viewsets import ModelViewSet
 
 from authentik.core.api.used_by import UsedByMixin
@@ -100,7 +101,10 @@ class Meta:
         extra_kwargs = {
             "users": {
                 "default": list,
-            }
+            },
+            # TODO: This field isn't unique on the database which is hard to backport
+            # hence we just validate the uniqueness here
+            "name": {"validators": [UniqueValidator(Group.objects.all())]},
         }
 
 

authentik/core/expression/evaluator.py

@@ -6,6 +6,7 @@
 from django.http import HttpRequest
 from prometheus_client import Histogram
 
+from authentik.core.expression.exceptions import SkipObjectException
 from authentik.core.models import User
 from authentik.events.models import Event, EventAction
 from authentik.lib.expression.evaluator import BaseEvaluator
@@ -47,6 +48,7 @@ def __init__(
         self._context["request"] = req
         req.context.update(**kwargs)
         self._context.update(**kwargs)
+        self._globals["SkipObject"] = SkipObjectException
         self.dry_run = dry_run
 
     def handle_error(self, exc: Exception, expression_source: str):

authentik/core/expression/exceptions.py

@@ -0,0 +1,13 @@
+"""authentik core exceptions"""
+
+from authentik.lib.sentry import SentryIgnoredException
+
+
+class PropertyMappingExpressionException(SentryIgnoredException):
+    """Error when a PropertyMapping Exception expression could not be parsed or evaluated."""
+
+
+class SkipObjectException(PropertyMappingExpressionException):
+    """Exception which can be raised in a property mapping to skip syncing an object.
+    Only applies to Property mappings which sync objects, and not on mappings which transitively
+    apply to a single user"""

authentik/core/migrations/0029_provider_backchannel_applications_and_more.py

@@ -7,9 +7,10 @@
 
 
 def backport_is_backchannel(apps: Apps, schema_editor: BaseDatabaseSchemaEditor):
-    from authentik.core.models import BackchannelProvider
+    from authentik.providers.ldap.models import LDAPProvider
+    from authentik.providers.scim.models import SCIMProvider
 
-    for model in BackchannelProvider.__subclasses__():
+    for model in [LDAPProvider, SCIMProvider]:
         try:
             for obj in model.objects.only("is_backchannel"):
                 obj.is_backchannel = True

authentik/core/models.py

@@ -22,7 +22,7 @@
 from structlog.stdlib import get_logger
 
 from authentik.blueprints.models import ManagedModel
-from authentik.core.exceptions import PropertyMappingExpressionException
+from authentik.core.expression.exceptions import PropertyMappingExpressionException
 from authentik.core.types import UILoginButton, UserSettingSerializer
 from authentik.lib.avatars import get_avatar
 from authentik.lib.generators import generate_id

authentik/core/tests/test_property_mapping.py

@@ -3,7 +3,7 @@
 from django.test import RequestFactory, TestCase
 from guardian.shortcuts import get_anonymous_user
 
-from authentik.core.exceptions import PropertyMappingExpressionException
+from authentik.core.expression.exceptions import PropertyMappingExpressionException
 from authentik.core.models import PropertyMapping
 from authentik.core.tests.utils import create_test_admin_user
 from authentik.events.models import Event, EventAction

authentik/enterprise/providers/google_workspace/api/property_mappings.py

@@ -0,0 +1,39 @@
+"""google Property mappings API Views"""
+
+from django_filters.filters import AllValuesMultipleFilter
+from django_filters.filterset import FilterSet
+from drf_spectacular.types import OpenApiTypes
+from drf_spectacular.utils import extend_schema_field
+from rest_framework.viewsets import ModelViewSet
+
+from authentik.core.api.propertymappings import PropertyMappingSerializer
+from authentik.core.api.used_by import UsedByMixin
+from authentik.enterprise.providers.google_workspace.models import GoogleWorkspaceProviderMapping
+
+
+class GoogleProviderMappingSerializer(PropertyMappingSerializer):
+    """GoogleProviderMapping Serializer"""
+
+    class Meta:
+        model = GoogleWorkspaceProviderMapping
+        fields = PropertyMappingSerializer.Meta.fields
+
+
+class GoogleProviderMappingFilter(FilterSet):
+    """Filter for GoogleProviderMapping"""
+
+    managed = extend_schema_field(OpenApiTypes.STR)(AllValuesMultipleFilter(field_name="managed"))
+
+    class Meta:
+        model = GoogleWorkspaceProviderMapping
+        fields = "__all__"
+
+
+class GoogleProviderMappingViewSet(UsedByMixin, ModelViewSet):
+    """GoogleProviderMapping Viewset"""
+
+    queryset = GoogleWorkspaceProviderMapping.objects.all()
+    serializer_class = GoogleProviderMappingSerializer
+    filterset_class = GoogleProviderMappingFilter
+    search_fields = ["name"]
+    ordering = ["name"]

authentik/enterprise/providers/google_workspace/api/providers.py

@@ -0,0 +1,54 @@
+"""Google Provider API Views"""
+
+from rest_framework.viewsets import ModelViewSet
+
+from authentik.core.api.providers import ProviderSerializer
+from authentik.core.api.used_by import UsedByMixin
+from authentik.enterprise.api import EnterpriseRequiredMixin
+from authentik.enterprise.providers.google_workspace.models import GoogleWorkspaceProvider
+from authentik.enterprise.providers.google_workspace.tasks import google_workspace_sync
+from authentik.lib.sync.outgoing.api import OutgoingSyncProviderStatusMixin
+
+
+class GoogleProviderSerializer(EnterpriseRequiredMixin, ProviderSerializer):
+    """GoogleProvider Serializer"""
+
+    class Meta:
+        model = GoogleWorkspaceProvider
+        fields = [
+            "pk",
+            "name",
+            "property_mappings",
+            "property_mappings_group",
+            "component",
+            "assigned_backchannel_application_slug",
+            "assigned_backchannel_application_name",
+            "verbose_name",
+            "verbose_name_plural",
+            "meta_model_name",
+            "delegated_subject",
+            "credentials",
+            "scopes",
+            "exclude_users_service_account",
+            "filter_group",
+            "user_delete_action",
+            "group_delete_action",
+            "default_group_email_domain",
+        ]
+        extra_kwargs = {}
+
+
+class GoogleProviderViewSet(OutgoingSyncProviderStatusMixin, UsedByMixin, ModelViewSet):
+    """GoogleProvider Viewset"""
+
+    queryset = GoogleWorkspaceProvider.objects.all()
+    serializer_class = GoogleProviderSerializer
+    filterset_fields = [
+        "name",
+        "exclude_users_service_account",
+        "delegated_subject",
+        "filter_group",
+    ]
+    search_fields = ["name"]
+    ordering = ["name"]
+    sync_single_task = google_workspace_sync

authentik/enterprise/providers/google_workspace/apps.py

@@ -0,0 +1,9 @@
+from authentik.enterprise.apps import EnterpriseConfig
+
+
+class AuthentikEnterpriseProviderGoogleConfig(EnterpriseConfig):
+
+    name = "authentik.enterprise.providers.google_workspace"
+    label = "authentik_providers_google_workspace"
+    verbose_name = "authentik Enterprise.Providers.Google Workspace"
+    default = True

authentik/enterprise/providers/google_workspace/clients/base.py

@@ -0,0 +1,71 @@
+from django.db.models import Model
+from django.http import HttpResponseNotFound
+from google.auth.exceptions import GoogleAuthError, TransportError
+from googleapiclient.discovery import build
+from googleapiclient.errors import Error, HttpError
+from googleapiclient.http import HttpRequest
+from httplib2 import HttpLib2Error, HttpLib2ErrorWithResponse
+
+from authentik.enterprise.providers.google_workspace.models import GoogleWorkspaceProvider
+from authentik.lib.sync.outgoing import HTTP_CONFLICT
+from authentik.lib.sync.outgoing.base import BaseOutgoingSyncClient
+from authentik.lib.sync.outgoing.exceptions import (
+    NotFoundSyncException,
+    ObjectExistsSyncException,
+    StopSync,
+    TransientSyncException,
+)
+
+
+class GoogleWorkspaceSyncClient[TModel: Model, TConnection: Model, TSchema: dict](
+    BaseOutgoingSyncClient[TModel, TConnection, TSchema, GoogleWorkspaceProvider]
+):
+    """Base client for syncing to google workspace"""
+
+    domains: list
+
+    def __init__(self, provider: GoogleWorkspaceProvider) -> None:
+        super().__init__(provider)
+        self.directory_service = build(
+            "admin",
+            "directory_v1",
+            cache_discovery=False,
+            **provider.google_credentials(),
+        )
+        self.__prefetch_domains()
+
+    def __prefetch_domains(self):
+        self.domains = []
+        domains = self._request(self.directory_service.domains().list(customer="my_customer"))
+        for domain in domains.get("domains", []):
+            domain_name = domain.get("domainName")
+            self.domains.append(domain_name)
+
+    def _request(self, request: HttpRequest):
+        try:
+            response = request.execute()
+        except GoogleAuthError as exc:
+            if isinstance(exc, TransportError):
+                raise TransientSyncException(f"Failed to send request: {str(exc)}") from exc
+            raise StopSync(exc) from exc
+        except HttpLib2Error as exc:
+            if isinstance(exc, HttpLib2ErrorWithResponse):
+                self._response_handle_status_code(exc.response.status, exc)
+            raise TransientSyncException(f"Failed to send request: {str(exc)}") from exc
+        except HttpError as exc:
+            self._response_handle_status_code(exc.status_code, exc)
+            raise TransientSyncException(f"Failed to send request: {str(exc)}") from exc
+        except Error as exc:
+            raise TransientSyncException(f"Failed to send request: {str(exc)}") from exc
+        return response
+
+    def _response_handle_status_code(self, status_code: int, root_exc: Exception):
+        if status_code == HttpResponseNotFound.status_code:
+            raise NotFoundSyncException("Object not found") from root_exc
+        if status_code == HTTP_CONFLICT:
+            raise ObjectExistsSyncException("Object exists") from root_exc
+
+    def check_email_valid(self, *emails: str):
+        for email in emails:
+            if not any(email.endswith(f"@{domain_name}") for domain_name in self.domains):
+                raise TransientSyncException(f"Invalid email domain: {email}")