Skip to content

Dedupe MicroMasters certificates in tfact_certificate and fix email-case user resolution #2381

Closed

Description

@blarghmatey

Expected Behavior

A MicroMasters certificate earned on edX.org produces exactly one row in tfact_certificate, and MicroMasters users resolve to a user_fk regardless of email case.

Current Behavior

tfact_certificate unions int__edxorg__mitx_courserun_certificates (lines 45-61) and int__micromasters__course_certificates (lines 63-79). MicroMasters certs are earned on edX.org and both rows resolve courserun_fk to the same edxorg run (lines 247-252). The surrogate keys differ (user_id-based vs email-based), so the defensive dedup cannot catch the double-count.

Additionally, lines 241-243 join raw MicroMasters user_email against dim_user.email, which is lower()-ed — so mixed-case MM emails silently fail user_fk resolution (and the warn-severity test hides it).

Possible Solution

  • Dedupe on (user, courserun) across sources with a documented precedence (or drop one source).
  • lower() the fact-side email before joining.

Additional Details

dim_program has the same micromasters-vs-edxorg duality — check it alongside this fix.


P1. Part of #2375 (2026-07 dbt warehouse audit). Witan task: tk-dedupe-micromasters-certificates-in-tfact-certif-21659d.

Activity

  1. added 2 commits that reference this issue on Jul 11, 2026
    0e17574
    728c959
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions