Sync groups and group membership through auth strategies

[signalkraft]

First off: It seems there is already a few feature requests on https://wiki.js.org/feedback/?search=group for this, so I added a ticket because I'm thinking about helping with the first implementation here.

Actual behavior

Currently authenticating with different strategies does not update group membership, even if the strategy supports that (i.e. roles in Keycloak, groups in LDAP). Manually adding users to groups is cumbersome and makes it difficult to use Wiki.js in larger teams where you want give some sub-teams their own private section.

Expected behavior

Signing in with a strategy that supports group / role memberships should create a group on Wiki.js, if it doesn't exist yet, and then add the user to the group during authentication. There should be settings in the admin UI of the different strategies that support groups, to control this behavior. My guess would be:

• Toggle "Synchronize groups"
• Toggle "Synchronize group membership"
• Group search query for LDAP

You could get infinitely more complex with custom group mappings, background sync of groups from LDAP, nested groups, permission mapping, etc, but as a first version the above seems useful.

--

I'd be happy to dig into the code and try to contribute a PR for LDAP and/or Keycloak, if you agree that this is a useful feature @NGPixel - it seems widely requested on Canny.

[NGPixel]

Sounds good 👍

[baodrate]

@signalkraft have you started on this? I'd like to give it a shot, but don't want to repeat any work you might've already done

[signalkraft]

@qubidt I looked into the underlying auth lib (passport) but couldn't figure out a good way to get groups out of it. Its main purpose is authentication, so maybe I also went at this from the wrong angle.

I fixed my own issue with groups by building a small python service that syncs users and groups back and forth over the (excellent) GraphQL API. So go for it!

[baodrate]

@signalkraft thanks, I looked into it a bit and also came to a similar conclusion. Not sure I'm familiar enough with the codebase to make the wide changes this would require. Your solution sounds like it could work well for me so I'll give it a shot, thanks!

[signalkraft]

@qubidt Here's my two scripts as a starting point for anyone in a similar position. get_ipa_client is just a thin wrapper around FreeIPAs Python SDK.

import os

import requests


def run_query(query, variables=None):
    headers = {"Authorization": f"Bearer {os.environ.get('WIKIJS_API_KEY')}"}
    json = {'query': query}
    if variables:
        json['variables'] = variables
    request = requests.post(os.environ.get('WIKIJS_API'), json=json, headers=headers)
    if request.status_code == 200:
        return request.json()
    else:
        raise Exception("Query failed to run by returning code of {}. {}".format(request.status_code, query))


def get_groups():
    query = """
    {
      groups {
        list {
          id
          name
          isSystem
          userCount
        }
      }
    }
    """
    return run_query(query)['data']['groups']['list']


def get_users():
    query = """
    {
      users {
        list {
          id
          name
          email
          providerKey
          isSystem
          createdAt
          __typename
        }
        __typename
      }
    }
    """
    return run_query(query)['data']['users']['list']


def create_group(name: str):
    query = """
    mutation($name: String!) {
      groups {
        create(name: $name) {
          responseResult {
            succeeded
            errorCode
            slug
            message
            __typename
          }
          group {
            id
            name
            createdAt
            updatedAt
            __typename
          }
          __typename
        }
        __typename
      }
    }

    """
    return run_query(query, {'name': name})['data']['groups']['create']['group']


def assign_user(group_id: int, user_id: int):
    query = """
    mutation($groupId: Int!, $userId: Int!) {
      groups {
        assignUser(groupId: $groupId, userId: $userId) {
          responseResult {
            succeeded
            errorCode
            slug
            message
            __typename
          }
          __typename
        }
        __typename
      }
    }
    """
    return run_query(query, {'groupId': group_id, 'userId': user_id})

#!/usr/bin/env python3
import logging
import os
from typing import List

from ipa import get_ipa_client
from wiki import get_groups, get_users, create_group, assign_user


# Set up logging to console
logger = logging.getLogger(__name__)
logger.setLevel(os.environ.get('LOG_LEVEL', 'INFO'))
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
ch.setFormatter(formatter)
logger.addHandler(ch)


if __name__ == '__main__':
    ipa_client = get_ipa_client()
    wikijs_groups = get_groups()  # type: List
    wikijs_users = get_users()  # type: List

    for ipa_group in ipa_client.list_groups():
        group_name = ipa_group['cn'][0]
        logger.info("Synchronizing group %s...", group_name)
        if group_name not in [g['name'] for g in wikijs_groups]:
            # Create group on Wiki.js
            wikijs_group = create_group(group_name)
            wikijs_groups.append(wikijs_group)
            logger.info("Created new Wiki.js group %s", group_name)
        else:
            # Get group by name
            wikijs_group = [g for g in wikijs_groups if g['name'] == group_name][0]
            logger.debug("Using existing group %s", group_name)

        for ipa_user in ipa_group.get('member_user', []):
            if ipa_user in [u['name'] for u in wikijs_users]:
                # Try to assign user to Wiki.js group
                wikijs_user = [u for u in wikijs_users if u['name'] == ipa_user][0]
                result = assign_user(wikijs_group['id'], wikijs_user['id'])
                if 'errors' in result and 'already assigned' in result['errors'][0]['message']:
                    # Ignore already exists errors
                    logger.debug("IPA User %s already in group %s", ipa_user, group_name)
                else:
                    logger.info("Added IPA user %s to group %s", ipa_user, group_name)
            else:
                logger.warning("Skipping IPA user %s, because it wasn't found on Wiki.js", ipa_user)

Bunch of ways to improve this still (nested loops, referencing nested dicts without null checks) but it runs reliably for me.

[drehelis]

@NGPixel is this something that is being looked into for the upcoming major releases?

[NGPixel]

@NGPixel is this something that is being looked into for the upcoming major releases?

It is planned for 3.x yes

[warthy]

It is planned for 3.x yes

is it worth it to open an MR with a potential "quick" implementation or V3 we be release soon, so there wouldn't any point ?

[devksingh4]

Hello,
Has there been any progress on implementing this for v3.x? I am hoping to use it with Azure AD. I didn't see any posted updates here or on the feedback page within the last year.

[uberspot]

+1 on this issue. Reviving this thread in the hope that someone would implement this.
It's a very useful feature.

[jtagcat]

GitHub Etiquette

• Please use the 👍 reaction to show that you are affected by the same issue.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

[fionera]

I would love tho see this happen. In the best case with a mapping of provider role name => group. I sadly dont know how the wikijs Codebase works and how much work implementing this would be

[aelgasser]

@NGPixel et al. I just submitted a PR to implement group sync with SAML: #6299

@fionera if you're still interested, this can be a starting point for you to implement it in another strategy.