Skip to content

data_detail_problems and data_detail_video are empty in production #2868

Open

Description

@quazi-h

Part of #2072.

Expected Behavior

data_detail_problems and data_detail_video have one row per problem or video event, like data_detail_nav does (about 30M rows). The Data Detail Problems and Data Detail Video charts on the Learner Engagement dashboard show data.

Current Behavior

Both tables in ol_warehouse_production_reporting have 0 rows (checked 2026-10-06), so both charts are blank.

Both models inner-join int__combined__course_runs on courserun_readable_id and platform (data_detail_problems.sql#L26-L30, data_detail_video.sql#L20-L24). tfact_problem_events and tfact_video_events use platform codes (mitxonline, edxorg, mitxpro, residential). int__combined__course_runs uses display names (MITx Online, edX.org, xPro, Residential MITx). The join never matches. data_detail_nav joins on courserun_readable_id only, so it isn't affected.

Steps to Reproduce

  1. select count(*) from ol_warehouse_production_reporting.data_detail_problems returns 0.
  2. select distinct platform from ol_warehouse_production_intermediate.int__combined__course_runs returns display names.
  3. select distinct platform from ol_warehouse_production_dimensional.tfact_problem_events returns codes.

Possible Solution

int__combined__course_runs is only used here to look up course_title, and it is the only non-dimensional ref in either model. Replace it with dim_course_run (is_current, platform codes) joined to dim_course on course_fk, which fixes the join and finishes the #2072 migration for both models. #2686 did the same lookup swap in engagement_problem_completion_summary and handled the two SCD2 dims there.

  • Remove both models' lines from dimensional_layering_baseline.txt.
  • dim_course has no residential rows, so check course_title coverage per platform. dim_course_run.courserun_title is a possible fallback.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions