feat(bootstrap): provision third-party auth providers and service tokens by rhysyngsun · Pull Request #311 · mitodl/lehrer

rhysyngsun · 2026-10-05T20:12:00Z

The bootstrap spec modelled users, oauth_applications and waffle_flags. Two more kinds of record have to exist before an LMS can federate login to an upstream identity provider and accept enrollments pushed back from it, and neither could be declared.

What was missing

An OAuth2ProviderConfig row. Without it, /auth/login/<backend>/ 404s even when the backend class is installed and listed in AUTHENTICATION_BACKENDS.

A long-lived AccessToken for the service account the upstream authenticates as. Without it the enrollment push 401s silently and learners sit at edx_enrolled=False with no courseware.

Both steps skip themselves when their *_env keys are unset, so a standalone install — which has no upstream in front of it — is unaffected and its provisioning run is unchanged.

Two implementation notes worth knowing

OAuth2ProviderConfig is a ConfigurationModel — append-only history keyed on (site_id, backend_name). Its .current() is overridden to take only the non-site key fields and resolve site_id from the active request, so the obvious current(site.id, backend_name) raises TypeError. The code uses a direct ordered queryset instead, which also avoids depending on request context a provisioning job does not have. The unchanged-check compares every declared field, so a re-run appends no history row.

The provider's URLs travel through other_settings rather than Django settings because that is the only channel there is: OAuth2ProviderConfig.get_setting serves KEY and SECRET from the row and every other name out of the other_settings JSON. It is canonicalised with sorted keys so an unchanged declaration does not read as a diff.

local-dev defaults

The committed defaults are the pair mitxonline's own local-dev registers for its edx-oauth-app, so a caller composing both stacks into one cluster gets a working pair with no extra configuration.

_{Stack created with GitHub Stacks CLI • Give Feedback 💬}

Copilot

Note

Copilot was unable to run its full agentic suite in this review.

Copilot review overview

Review effort: Lite
Findings: 1 · 3

Open (4)

Storing other_settings as json.dumps(...) will persist a JSON string rather than a JSON… · New
_report() interprets created=False as updated, but in the unchanged case this step performs… · New
update_or_create(name=...) can raise MultipleObjectsReturned if the database already contains… · New
Using update_or_create(token=token, defaults=...) will silently reassign an existing token string… · New

What changed in this PR

Adds bootstrap support for configuring third-party OAuth login providers and long-lived service bearer tokens during provisioning.

Changes:

Extends the bootstrap spec/schema with third_party_auth_providers and service_access_tokens.
Implements two new apply steps to create OAuth2ProviderConfig history rows and OAuth Toolkit AccessToken/Application + service user resources.
Adds optional environment variable lookup to allow steps to be skipped when secrets are absent.

File	Description
src/lehrer/bootstrap/spec.py	Adds new spec steps/models and `from_env_optional()` helper.
src/lehrer/bootstrap/__main__.py	Implements apply functions for third-party auth providers and service access tokens; wires them into APPLY/STEPS.
local-dev/secret-defaults.yaml	Adds local-dev default secrets for upstream IdP credentials.
local-dev/provision/bootstrap.yaml	Declares example `third_party_auth_providers` and `service_access_tokens` config entries.
bootstrap.schema.json	Updates JSON schema to include the new spec sections/models.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+            # Canonicalised, so a reordered or reindented env value does not
+            # read as a change and append a new version on every apply.
+            "other_settings": json.dumps(parsed, sort_keys=True),


+        if not unchanged:
+            OAuth2ProviderConfig.objects.create(
+                site=site, backend_name=declared.backend_name, **fields
+            )
+        _report(
+            "third_party_auth_providers",
+            declared.backend_name,
+            created=not unchanged,


+        application, _ = application_model.objects.update_or_create(
+            name=declared.application_name,
+            defaults={
+                "user": user,
+                "client_type": application_model.CLIENT_CONFIDENTIAL,
+                "authorization_grant_type": application_model.GRANT_CLIENT_CREDENTIALS,
+            },
+        )


+        _, created = token_model.objects.update_or_create(
+            token=token,
+            defaults={
+                "user": user,
+                "application": application,
+                "expires": timezone.now() + timedelta(days=declared.expires_in_days),
+                "scope": " ".join(declared.scopes),
+            },
+        )


blarghmatey · 2026-10-05T20:57:16Z

+third_party_auth_providers:
+# Federated login against an upstream MIT Open Learning application (mitxonline
+# in the composed local-dev cluster), via ol-social-auth's OLOAuth2 backend.
+#
+# Inert unless the caller supplies OL_OAUTH2_OTHER_SETTINGS: a standalone
+# install has no upstream to federate to, and the endpoint URLs are a property
+# of that upstream rather than of this stack, so they arrive from the
+# environment instead of being pinned here. The credentials come from
+# openedx-secrets like every other secret in this file.
+#
+# Needs ol_social_auth.backends.OLOAuth2 in THIRD_PARTY_AUTH_BACKENDS and a
+# true ENABLE_THIRD_PARTY_AUTH, or the row exists and nothing reads it.
+- backend_name: ol-oauth2
+  slug: ol-oauth2
+  name: MIT Open Learning
+  client_id_env: OL_OAUTH2_CLIENT_ID
+  client_secret_env: OL_OAUTH2_CLIENT_SECRET    # pragma: allowlist secret
+  other_settings_env: OL_OAUTH2_OTHER_SETTINGS
+  # The upstream owns registration and has already verified the address, so
+  # the LMS provisions straight from its user data instead of prompting for a
+  # form it cannot complete and an email it cannot confirm.
+  skip_registration_form: true
+  skip_email_verification: true
+  sync_learner_profile_data: true
+
+service_access_tokens:
+# The bearer token an upstream MIT Open Learning application uses to drive the
+# LMS enrollment API on learners' behalf (MITx Online's
+# OPENEDX_SERVICE_WORKER_API_TOKEN). It is not exchanged for anything: the
+# upstream sends this exact value, so it has to exist here as an AccessToken or
+# every enrollment push fails and the upstream records edx_enrolled=False
+# without surfacing an error to the learner.
+#
+# Inert unless the caller supplies OPENEDX_SERVICE_WORKER_API_TOKEN; a
+# standalone install has no upstream pushing enrollments to it.
+- username: mitxonline_service_worker
+  email: mitxonline-service-worker@example.com
+  application_name: mitxonline-service-worker
+  token_env: OPENEDX_SERVICE_WORKER_API_TOKEN    # pragma: allowlist secret
+  # Enrolling somebody *else* through the enrollment API is staff-only.
+  staff: true
+


Is it possible for us to scope this to the ol-mit deployment, or layer it in our local-dev deploy inheriting from lehrer?

Three groups of settings that pydantic-settings was silently dropping, all for reasons the codegen cannot detect on its own. LEARNING_MICROFRONTEND_URL is defined only in edx-platform's openedx/envs/common.py, which lms/envs/common.py reaches by star-import. The codegen's static AST pass does not follow star-imports, so no field is generated and the env source drops the var -- the setting stays at the base None and the LMS keeps linking at the legacy /courses/<id>/course/ pages. It fails quietly (None, not the string), which is why it survived a whole MFE rollout: the value was set in the ConfigMap, present in the pod's environment, and still None in Django. Same class as the CELERY_BROKER_* note already in this file. MITX_REDIRECT_* comes from the openedx-companion-auth plugin, whose plugin_settings() hardcodes MITX_REDIRECT_ENABLED = True and an allow-list that does not include /heartbeat. The plugin's own override path (its production.py) reads settings.ENV_TOKENS, which does not exist under aqueduct, so without these fields there is no way to turn the redirect off at all -- and an undeclared env var would arrive as the string "false", which is truthy. Every real deployment already overrides these, so the plugin default is never the intended value. The last group is LMS-only settings the CMS also has to resolve, because ol-openedx-canvas-integration registers a cms.djangoapp entry point while its tasks.py imports lms.djangoapps.instructor_task -> bulk_email.tasks, which reads them at decorator-evaluation time. CMS legitimately lacks them: lms/envs/common.py defines them, cms/envs/common.py does not. Tests mirror the existing env-arrival coverage and assert the parsed types, not just truthiness -- a string "false" would pass a truthiness check and is exactly the failure being guarded against. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The bootstrap spec modelled users, oauth_applications and waffle_flags. Two more kinds of record have to exist before an LMS can federate login to an upstream identity provider and accept enrollments pushed back from it, and neither could be declared: - an OAuth2ProviderConfig row, without which /auth/login/<backend>/ 404s even when the backend class is installed and listed in AUTHENTICATION_BACKENDS; and - a long-lived AccessToken for the service account the upstream authenticates as, without which its enrollment push 401s silently and learners sit at edx_enrolled=False with no courseware. Both steps skip themselves when their *_env keys are unset, so a standalone install -- which has no upstream in front of it -- is unaffected and its provisioning run is unchanged. Two things worth knowing about the implementation: OAuth2ProviderConfig is a ConfigurationModel: append-only history keyed on (site_id, backend_name). Its .current() is overridden to take only the non-site key fields and resolve site_id from the active request, so the obvious current(site.id, backend_name) raises TypeError. The code uses a direct ordered queryset instead, which also avoids depending on request context a provisioning job does not have. The unchanged check compares every declared field so a re-run appends no history row. The provider's URLs travel through other_settings rather than Django settings because that is the only channel there is: OAuth2ProviderConfig.get_setting serves KEY and SECRET from the row and every other name out of the other_settings JSON. It is canonicalised with sorted keys so an unchanged declaration does not look like a diff. local-dev's defaults are the pair mitxonline's own local-dev registers for its edx-oauth-app, so a caller composing both stacks into one cluster gets a working pair with no extra configuration. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

rhysyngsun added this pull request to stack #313 October 5, 2026 20:12

rhysyngsun marked this pull request as ready for review October 5, 2026 20:21

Copilot AI balanced review requested due to automatic review settings October 5, 2026 20:21

rhysyngsun force-pushed the nl/bootstrap-sso-provisioning branch from 3b0120d to e36c748 Compare October 5, 2026 20:22

rhysyngsun mentioned this pull request Oct 5, 2026

feat(local-dev): federate LMS login to mitxonline over ol-oauth2 mitodl/ol-infrastructure#6214

Open

Copilot AI reviewed Oct 5, 2026

View reviewed changes

blarghmatey approved these changes Oct 5, 2026

View reviewed changes

rhysyngsun force-pushed the nl/bootstrap-sso-provisioning branch 2 times, most recently from 04ce02a to 9d45e70 Compare October 6, 2026 13:25

rhysyngsun force-pushed the nl/bootstrap-sso-provisioning branch 2 times, most recently from 696537b to 1f042a3 Compare October 6, 2026 15:03

rhysyngsun and others added 2 commits October 6, 2026 11:04

rhysyngsun force-pushed the nl/bootstrap-sso-provisioning branch from 1f042a3 to 3031497 Compare October 6, 2026 15:04

blarghmatey approved these changes Oct 15, 2026

View reviewed changes

Navigation Menu

Repository navigation

feat(bootstrap): provision third-party auth providers and service tokens - #311

feat(bootstrap): provision third-party auth providers and service tokens#311
rhysyngsun wants to merge 2 commits into
nl/declare-star-import-settingsfrom
nl/bootstrap-sso-provisioning

rhysyngsun commented Oct 5, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

blarghmatey Oct 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

rhysyngsun commented Oct 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What was missing

Two implementation notes worth knowing

local-dev defaults

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

Uh oh!

blarghmatey Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rhysyngsun commented Oct 5, 2026 •

edited

Loading