Skip to content

ETL Pipeline to keep current and historical data of Employee location

Notifications You must be signed in to change notification settings

faraaznx/SCD-2-Implementation-

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SCD-2-Implementation

alt_text

Introduction

A company X wants to keep its employee data up to date in their central database. Since, over the time, many employees prefer to change their base location, it is essential to update the employee information in the company's central database.

Problem statement

The field 'Location' is a Slowly changing Dimension ( a dimension whose attribute or attributes for a record (row) change slowly over time). We will be using SCD-2 type implementation to keep a full history of dimension data in the table. The Type 2 Dimension mapping filters source rows based on user-defined comparisons and inserts both new and changed dimensions into the target. Changes are tracked in the target table by looking up the primary key and creating a version number for each dimension in the table called a surrogate key.

ETL Process

We are sourcing the data from MS SQL Server, transforming it using Informatica and loading it to Oracle server.

Source Database

alt_text

Target Database

alt_text

Source Table

alt_text

Target Table

alt_text

ETL Pipeline

alt_text

About

ETL Pipeline to keep current and historical data of Employee location

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published