Skip to content

Added data engineering guides #842

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
AtmaMani merged 3 commits into from
Dec 14, 2020
Merged

Added data engineering guides #842

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1f33e2c
Added data engineering guides
mohi9282 Nov 25, 2020
2140b30
Updates per feedback from Breck
mohi9282 Dec 4, 2020
a49997e
Update part1_introduction_to_dataengineering.ipynb
mohi9282 Dec 14, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,624 changes: 1,624 additions & 0 deletions guide/16-data-engineering-guides/part1_introduction_to_dataengineering.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

2,981 changes: 2,981 additions & 0 deletions guide/16-data-engineering-guides/part2_introduction_to_numpy.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

8,981 changes: 8,981 additions & 0 deletions guide/16-data-engineering-guides/part3_introduction_to_pandas.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

7,863 changes: 7,863 additions & 0 deletions guide/16-data-engineering-guides/part4_productivity_with_pandas.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

4,248 changes: 4,248 additions & 0 deletions guide/16-data-engineering-guides/part5_time_series_in_pandas.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions guide/data/health.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"Number of Beds":{"0":156,"1":59,"2":25,"3":49,"4":55},"Name":{"0":"Facility 1","1":"Facility 2","2":"Facility 3","3":"Facility 4","4":"Facility 5"},"Address":{"0":"2468 SOUTH ST ANDREWS PLACE","1":"2300 W. WASHINGTON BLVD.","2":"4060 E. WHITTIER BLVD.","3":"6070 W. PICO BOULEVARD","4":"1480 S. LA CIENEGA BL"},"City":{"0":"LOS ANGELES","1":"LOS ANGELES","2":"LOS ANGELES","3":"LOS ANGELES","4":"LOS ANGELES"},"State":{"0":"CA","1":"CA","2":"CA","3":"CA","4":"CA"},"Zip Code":{"0":90018,"1":90018,"2":90023,"3":90035,"4":90035}}
Binary file added guide/data/health.xlsx
View file
Open in desktop
Binary file not shown.
125 changes: 125 additions & 0 deletions guide/data/sample_cms_data.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
Provider Name,Provider City,Provider State,Residents Weekly Admissions COVID-19,Residents Total Admissions COVID-19,Residents Weekly Confirmed COVID-19,Residents Total Confirmed COVID-19,Residents Weekly Suspected COVID-19,Residents Total Suspected COVID-19,Residents Weekly All Deaths,Residents Total All Deaths,Residents Weekly COVID-19 Deaths,Residents Total COVID-19 Deaths,Number of All Beds,Total Number of Occupied Beds,LONGITUDE,LATITUDE
GROSSE POINTE MANOR,NILES,IL,3,5,10,56,0,10,6,15,4,12,99,61,-87.792973,42.012012
MILLER'S MERRY MANOR,DUNKIRK,IN,0,0,0,0,0,0,0,0,0,0,46,43,-85.197651,40.392722
PARKWAY MANOR,MARION,IL,0,0,0,0,0,0,0,0,0,0,131,84,-88.982944,37.750143
AVANTARA LONG GROVE,LONG GROVE,IL,1,6,0,141,3,3,0,0,0,0,195,131,-87.986442,42.160843
HARMONY NURSING & REHAB CENTER,CHICAGO,IL,2,19,1,75,0,0,0,43,0,16,180,116,-87.726353,41.975505
TRANSCENDENT HEALTHCARE OF BOONVILLE - NORTH,BOONVILLE,IN,0,0,0,0,0,0,0,0,0,0,56,47,-87.269655,38.058503
HARCOURT TERRACE NURSING AND REHABILITATION,INDIANAPOLIS,IN,0,2,0,1,0,0,0,1,0,1,110,66,-86.193469,39.904128
GREENCROFT HEALTHCARE,GOSHEN,IN,0,3,0,65,0,65,0,36,0,13,153,155,-85.817798,41.561063
"WATERS OF MARTINSVILLE, THE",MARTINSVILLE,IN,0,2,0,33,0,2,1,9,0,8,103,44,-86.432593,39.407438
CATHEDRAL HEALTH CARE CENTER,JASPER,IN,0,0,0,0,0,0,0,1,0,0,65,37,-86.936473,38.393736
APERION CARE CAIRO,CAIRO,IL,0,0,0,0,0,0,0,0,0,0,83,53,-89.178966,37.002806
MAJESTIC CARE OF CONNERSVILLE,CONNERSVILLE,IN,0,0,0,0,0,8,0,0,0,0,166,68,-85.123770,39.639037
PARK POINTE HEALTHCARE & REHAB,MORRIS,IL,0,0,5,16,0,0,3,7,1,1,146,98,-88.447650,41.367484
MILLER'S MERRY MANOR,MOORESVILLE,IN,0,0,0,1,0,1,0,3,0,0,98,61,-86.381582,39.611876
APERION CARE BURBANK,BURBANK,IL,10,44,0,3,0,0,5,23,0,2,56,49,-87.763826,41.748753
HILLCREST HOME,GENESEO,IL,0,0,0,0,0,0,0,0,0,0,106,90,-90.165060,41.363867
STONEBROOKE REHABILITATION CENTER,NEW CASTLE,IN,1,19,0,53,0,0,0,19,0,11,117,62,-85.365021,39.941559
MAYFIELD HEALTH CENTER,CHICAGO,IL,0,12,0,112,0,0,0,21,0,18,156,110,-87.772013,41.882153
RIVER TERRACE HEALTH CARE CENTER,BLUFFTON,IN,0,0,0,0,0,1,0,0,0,0,30,26,-85.193590,40.777324
HERITAGE HEALTH-JACKSONVILLE,JACKSONVILLE,IL,0,0,0,0,0,0,0,0,0,0,175,96,-90.240928,39.730634
SIGNATURE HEALTHCARE AT PARKWOOD,LEBANON,IN,0,2,5,28,0,1,0,7,0,4,106,66,-86.460087,40.056879
HOMEVIEW CENTER OF FRANKLIN,FRANKLIN,IN,0,0,0,0,0,0,0,0,0,0,119,92,-86.045753,39.474115
"H & J VONDERLIETH LVG CTR, THE",MOUNT PULASKI,IL,0,0,0,0,1,6,0,0,0,0,90,63,-89.282268,40.021164
TWIN CITY HEALTH CARE,GAS CITY,IN,0,0,0,0,0,0,0,0,0,0,75,46,-85.604784,40.495148
HICKORY POINT CHRISTIAN VILLAGE,FORSYTH,IL,0,0,0,0,0,10,0,7,0,0,64,55,-88.963774,39.921646
PINCKNEYVILLE NURSING & REHAB,PINCKNEYVILLE,IL,0,0,0,0,0,0,0,3,0,0,60,44,-89.390771,38.072421
WATERFRONT TERRACE,CHICAGO,IL,0,14,0,48,0,0,0,8,0,10,118,96,-87.552222,41.755860
MILLER'S MERRY MANOR,DUNKIRK,IN,0,0,0,0,0,0,0,3,0,0,46,44,-85.197651,40.392722
CITADEL CARE CENTER-WILMETTE,WILMETTE,IL,9,18,0,22,0,5,1,5,0,2,60,49,-87.705245,42.072903
ELEVATE ST ANDREW LIVING COMM,NILES,IL,0,1,2,3,0,0,1,2,1,2,55,37,-87.801230,42.010046
MORNING BREEZE RETIREMENT COMMUNITY AND HEALTHCARE,GREENSBURG,IN,0,0,0,0,0,0,0,17,0,7,64,39,-85.496445,39.344599
"GROVE OF SKOKIE, THE",SKOKIE,IL,0,2,1,4,3,9,28,132,0,4,149,120,-87.751757,42.044763
LAURELS OF DEKALB,BUTLER,IN,0,0,0,0,0,0,0,0,0,0,101,80,-84.878247,41.432595
NEW ALBANY NURSING AND REHABILITATION CENTER,NEW ALBANY,IN,5,16,0,4,0,0,0,7,0,6,104,73,-85.821817,38.287688
NORTH AURORA CARE CENTER,NORTH AURORA,IL,0,0,0,0,0,0,0,0,0,0,129,98,-88.312443,41.815568
HICKORY CREEK AT NEW CASTLE,NEW CASTLE,IN,0,0,0,0,0,0,0,1,0,0,36,33,-85.365033,39.940616
UNION COUNTY HOSPITAL L T C,ANNA,IL,0,0,0,0,0,0,0,0,0,0,22,14,-89.245021,37.465383
ENFIELD REHAB & HEALTH CARE CT,ENFIELD,IL,0,0,0,0,0,0,0,0,0,0,47,20,-88.336849,38.102388
LAKE COOK REHAB & HEALTHCARE,NORTHBROOK,IL,0,5,0,75,0,23,0,43,0,19,147,75,-87.879275,42.152881
HEALTHWIN,SOUTH BEND,IN,0,0,40,40,10,10,8,8,7,7,153,111,-86.267748,41.730984
SOUTHPOINT NURSING & REHAB CENTER,CHICAGO,IL,0,31,0,38,0,16,0,12,0,8,228,165,-87.648591,41.721425
UNIVERSITY HEIGHTS HEALTH AND LIVING COMMUNITY,INDIANAPOLIS,IN,0,0,0,0,0,0,0,8,0,0,174,133,-86.135442,39.635530
GOLDEN LIVING CENTER-RICHMOND,RICHMOND,IN,0,0,0,2,0,0,0,1,0,3,96,52,-84.886313,39.846906
WESLEY MANOR HEALTH CENTER,FRANKFORT,IN,0,1,0,0,0,0,1,4,0,0,96,81,-86.511531,40.294507
PARK PLACE CHRISTIAN COMMUNITY,ELMHURST,IL,0,0,0,0,0,2,0,1,0,0,37,31,-87.938618,41.862090
WARREN BARR NORTH SHORE,HIGHLAND PARK,IL,0,52,1,1,0,0,0,16,0,16,215,110,-87.834407,42.202671
GOLDEN LIVING CENTER-WOODBRIDGE,EVANSVILLE,IN,0,0,0,0,0,0,0,0,0,0,67,55,-87.574945,37.985518
INDIANA VETERANS HOME,WEST LAFAYETTE,IN,0,0,0,0,0,0,1,5,0,0,331,136,-86.885096,40.471635
WASHINGTON NURSING CENTER,WASHINGTON,IN,0,0,0,0,0,6,7,14,0,0,140,38,-87.167504,38.652354
GOLDEN LIVING CENTER-GOLDEN RULE,RICHMOND,IN,0,0,0,0,0,0,0,0,0,0,170,82,-84.890002,39.797730
SANCTUARY AT HOLY CROSS,SOUTH BEND,IN,0,0,0,2,0,0,0,0,0,0,168,118,-86.206403,41.712877
"WESTWOOD MANOR, THE",CHICAGO,IL,1,15,0,45,0,10,0,3,0,2,115,84,-87.692142,42.012273
HERITAGE CENTER,EVANSVILLE,IN,0,0,0,0,0,0,0,0,0,0,155,135,-87.579248,38.014359
MAPLE MANOR CHRISTIAN HOME INC,SELLERSBURG,IN,0,0,0,0,0,6,0,2,0,0,57,47,-85.763934,38.405328
WEDGEWOOD HEALTHCARE CENTER,CLARKSVILLE,IN,0,0,0,68,0,0,0,25,0,23,115,84,-85.784553,38.327205
WESTMINSTER VILLAGE NORTH,INDIANAPOLIS,IN,0,1,0,2,0,3,1,18,0,1,148,95,-85.969776,39.872089
EASTGATE MANOR NURSING AND REHABILITATION,WASHINGTON,IN,0,0,0,0,0,4,1,8,0,0,62,39,-87.147707,38.653994
WILLOW ROSE REHAB & HEALTH,JERSEYVILLE,IL,0,0,0,0,0,0,0,1,0,0,98,45,-90.328330,39.112415
FOSTER HEALTH & REHAB CENTER,CHICAGO,IL,0,3,0,24,0,11,0,6,0,6,46,33,-87.700732,41.975803
YORKTOWN MANOR,YORKTOWN,IN,0,4,0,68,0,42,0,15,0,15,77,50,-85.480764,40.175349
"WATERS OF HUNTINGBURG, THE",HUNTINGBURG,IN,0,10,0,0,0,0,0,0,0,0,95,42,-86.959603,38.312308
APERION CARE JACKSONVILLE,JACKSONVILLE,IL,0,6,0,44,0,26,0,5,0,5,113,79,-90.233924,39.747165
MAJESTIC CARE OF SHERIDAN,SHERIDAN,IN,0,0,0,0,0,0,0,0,0,0,80,77,-86.225434,40.131148
"GROVE OF NORTHBROOK,THE",NORTHBROOK,IL,0,0,0,0,7,7,0,0,0,0,132,118,-87.798390,42.147850
MILLER'S MERRY MANOR,PLYMOUTH,IN,0,0,0,0,0,0,0,0,0,0,135,66,-86.315168,41.331038
EAGLE CREEK HEALTHCARE CENTER,INDIANAPOLIS,IN,0,0,0,0,0,0,1,1,0,0,100,95,-86.279920,39.828984
GREENHILL MANOR,FOWLER,IN,0,0,0,0,0,1,0,0,0,0,64,47,-87.316481,40.622005
ALHAMBRA CARE CENTER,ALHAMBRA,IL,0,0,0,0,0,0,0,0,0,0,81,39,-89.729439,38.888299
ALLURE OF PROPHETSTOWN,PROPHETSTOWN,IL,0,1,0,1,0,3,0,0,0,0,70,39,-89.939348,41.678111
MILLER'S MERRY MANOR,PERU,IN,0,0,0,0,0,0,0,16,0,0,130,68,-86.087531,40.758018
FORUM AT THE CROSSING,INDIANAPOLIS,IN,0,1,0,15,0,0,0,2,0,2,77,28,-86.121512,39.910761
COUNTRYSIDE MANOR HEALTH & LIVING COMMUNITY,ANDERSON,IN,0,0,0,0,0,0,0,2,0,0,109,75,-85.683290,40.086935
ABBINGTON REHAB & NURSING CTR,ROSELLE,IL,1,7,1,44,0,0,1,2,1,6,82,55,-88.081628,41.983439
LEMONT NURSING & REHAB CENTER,LEMONT,IL,1,23,6,28,0,42,2,32,0,10,173,100,-87.970911,41.663003
GOLDEN YEARS HOMESTEAD,FORT WAYNE,IN,0,5,0,0,0,0,0,0,0,0,110,104,-85.036651,41.107479
BRIA OF CHICAGO HEIGHTS,SOUTH CHICAGO HEIGHT,IL,0,0,2,5,0,0,0,0,0,0,112,85,-87.640575,41.491500
ROBINGS MANOR RHC,BRIGHTON,IL,0,0,0,0,0,0,0,11,0,0,72,43,-90.141195,39.043557
BERKELEY NURSING & REHAB CENTER,OAK PARK,IL,0,0,0,0,0,0,3,4,2,2,72,54,-87.799020,41.908940
TOWNE HOUSE RETIREMENT COMMUNITY,FORT WAYNE,IN,0,0,1,1,1,1,0,1,0,0,58,46,-85.111952,41.133477
SHARON HEALTH CARE PINES,PEORIA,IL,0,0,0,0,0,0,0,1,0,0,116,96,-89.643629,40.731764
COUNTRYVIEW CARE CENTER-MACOMB,MACOMB,IL,0,0,0,0,0,1,0,9,0,0,62,42,-90.676440,40.447738
UNION COUNTY HOSPITAL L T C,ANNA,IL,0,0,0,0,0,0,0,0,0,0,22,14,-89.245021,37.465383
RIVERSIDE VILLAGE,ELKHART,IN,1,1,0,2,0,0,0,0,0,0,97,78,-85.988504,41.677455
BROOKDALE PROSPECT HEIGHTS,PROSPECT HEIGHTS,IL,0,0,0,0,0,0,0,0,0,0,30,21,-87.918983,42.087858
BRIDGE CARE SUITES,SPRINGFIELD,IL,0,0,0,0,0,0,0,0,0,0,75,45,-89.712368,39.786753
"WATERS OF CLIFTY FALLS, THE",MADISON,IN,0,1,0,0,0,1,0,12,0,0,138,69,-85.407453,38.764628
ALPHA HOME - A WATERS COMMUNITY,INDIANAPOLIS,IN,0,13,0,46,0,48,15,52,15,46,84,40,-86.203158,39.804906
ASHTON CREEK HEALTH AND REHABILITATION CENTER,FORT WAYNE,IN,3,36,0,0,0,0,0,1,0,1,139,100,-85.092123,41.188488
SUNSET REHABILITATION & HLTH C,CANTON,IL,0,0,0,0,0,0,18,18,0,0,115,74,-90.033527,40.554240
MIDWAY NEUROLOGICAL / REHAB CENTER,BRIDGEVIEW,IL,0,3,0,132,0,20,0,16,0,16,404,371,-87.799224,41.735931
HEARTHSTONE MANOR,WOODSTOCK,IL,0,2,3,30,0,0,0,2,0,0,130,67,-88.443149,42.323875
MANOR COURT OF CLINTON,CLINTON,IL,0,0,0,0,1,10,2,6,0,0,134,84,-88.973518,40.144751
INVERNESS HEALTH & REHAB,INVERNESS,IL,0,0,0,4,0,0,0,1,0,1,140,107,-88.078009,42.082937
MILLER'S MERRY MANOR,PERU,IN,0,0,0,0,0,0,0,0,0,0,130,69,-86.087531,40.758018
MILLER'S MERRY MANOR,MARION,IN,0,0,0,0,0,0,0,0,0,0,176,93,-85.685577,40.565875
SYMPHONY AT MIDWAY,CHICAGO,IL,2,34,0,9,0,4,2,25,0,0,249,182,-87.743158,41.811916
CHALET VILLAGE HEALTH AND REHABILITATION CENTER,BERNE,IN,0,0,0,0,0,0,0,1,0,0,60,27,-84.961982,40.648407
WOODLAND HILLS CARE CENTER,LAWRENCEBURG,IN,0,0,0,0,0,0,2,11,0,0,75,47,-84.883661,39.116499
UNIVERSITY NSG & REHAB CENTER,EDWARDSVILLE,IL,0,0,2,2,0,0,0,0,0,0,105,85,-89.970763,38.796573
EXCEPTIONAL LIVING CENTER OF BRAZIL,BRAZIL,IN,0,0,0,0,0,0,0,0,0,0,105,60,-87.108203,39.521232
"WATERS OF RISING SUN, THE",RISING SUN,IN,0,0,0,0,0,0,0,0,0,0,58,43,-84.861849,38.943005
GOLDEN LIVING CENTER-BLOOMINGTON,BLOOMINGTON,IN,0,1,23,74,4,34,4,10,0,0,153,108,-86.529807,39.126665
HAMMOND-WHITING CARE CENTER,WHITING,IN,0,0,13,44,0,24,8,12,7,9,80,48,-87.504093,41.687993
CHESTERTON MANOR,CHESTERTON,IN,0,0,0,0,1,10,0,0,0,0,100,66,-87.049669,41.595877
GALLATIN MANOR,RIDGWAY,IL,0,0,0,0,0,0,0,0,0,0,71,39,-88.269689,37.793770
ELWOOD HEALTH AND LIVING,ELWOOD,IN,0,0,0,0,0,0,0,0,0,0,91,57,-85.832559,40.286925
HICKORY CREEK AT FRANKLIN,FRANKLIN,IN,0,0,0,0,0,0,0,0,0,0,36,30,-86.062635,39.491658
PARKER NURSING & REHAB CENTER,STREATOR,IL,0,0,0,0,0,0,0,0,0,0,102,45,-88.843063,41.140037
DANVILLE REGIONAL REHABILITATION,DANVILLE,IN,0,0,0,0,0,0,0,0,0,0,110,109,-86.499629,39.763723
PRESENCE NAZARETHVILLE,DES PLAINES,IL,0,1,0,17,0,22,0,0,0,0,68,54,-87.889623,42.058927
GOLDEN LIVING CENTER-INDIANAPOLIS,INDIANAPOLIS,IN,0,1,0,0,0,0,0,2,0,0,101,78,-86.111747,39.730386
MILLER'S MERRY MANOR,HARTFORD CITY,IN,0,0,0,0,0,0,0,0,0,0,65,41,-85.389109,40.442431
"WATERS OF SCOTTSBURG, THE",SCOTTSBURG,IN,0,0,0,16,0,0,16,31,16,33,85,54,-85.787005,38.701980
RIVERSIDE VILLAGE,ELKHART,IN,0,0,0,0,0,0,0,0,0,0,97,82,-85.988504,41.677455
EASTGATE MANOR NURSING AND REHABILITATION,WASHINGTON,IN,0,0,0,0,0,4,0,6,0,0,62,49,-87.147707,38.653994
HILLCREST RETIREMENT VILLAGE,ROUND LAKE BEACH,IL,0,0,0,4,0,0,1,19,0,0,138,119,-88.086090,42.380225
HICKORY CREEK AT KENDALLVILLE,KENDALLVILLE,IN,0,0,0,0,0,0,0,1,0,0,36,16,-85.264893,41.427192
"ADMIRAL AT THE LAKE, THE",CHICAGO,IL,0,11,0,11,0,3,1,2,1,6,36,32,-87.653425,41.976407
PRESENCE RESURRECTION LIFE CTR,CHICAGO,IL,0,0,0,2,0,0,0,9,0,0,162,139,-87.811841,41.988106
SMITH CROSSING,ORLAND PARK,IL,0,3,0,0,0,0,0,0,0,0,46,37,-87.873796,41.557067
"CITADEL OF NORTHBROOK, THE",NORTHBROOK,IL,0,8,0,11,0,3,0,2,0,2,148,138,-87.876582,42.093489
MORRISON WOODS HEALTH CAMPUS,MUNCIE,IN,0,0,0,2,0,1,1,7,0,0,68,44,-85.442732,40.226829
SELFHELP HOME OF CHICAGO,CHICAGO,IL,3,3,1,1,0,0,2,2,1,1,72,42,-87.653243,41.973292
"WATERS OF DILLSBORO-ROSS MANOR, THE",DILLSBORO,IN,0,4,0,0,0,0,0,7,0,0,123,75,-85.056649,39.018794