Skip to content

Dimensional yml/doc alignment pass (OpenMetadata-verified drift) #2386

Open

Description

@blarghmatey

Description/Context

Batch of OpenMetadata-verified doc-vs-SQL mismatches in models/dimensional:

  1. afact_video_engagement.video_played_count / video_completed_count documented as 0/1 flags but are counts; afact_discussion_engagement "indicating if" columns are count_if.
  2. tfact_video_events yml lists event types not in the SQL filter (video_hide_transcript) and omits ones that are (complete_video, closed_captions); tfact_course_navigation_events yml lists edx.ui.lms.video.played which isn't filtered.
  3. dim_discussion_topic.discussion_type yml values don't match emitted 'discussion component' / 'course-wide discussion'.
  4. tfact_payment yml describes user_fk resolution via an order chain but the SQL joins staging orders directly (lines 46-58); also switch its watermark from > to >= (line 145; siblings use >=) and add the standard dedup guard.
  5. dim_video extracts $.edx_video_id twice as edx_video_id and video_edx_uuid, documented as different identifiers.
  6. OpenMetadata-confirmed type drift: dim_user.certificate_desired is VARCHAR but documented boolean; mitlearn_openedx_user_id / mitxonline_application_user_id are BIGINT but documented string.

Plan/Design

Reconcile each yml/doc entry to the actual SQL behavior (or fix the SQL where the doc reflects intent, e.g. the tfact_payment watermark). Mechanical documentation-alignment pass; verify each against OpenMetadata profiling.


P2. Part of #2375 (2026-07 dbt warehouse audit). Witan task: tk-dimensional-yml-doc-alignment-pass-openmetadata--1f9c93.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions