Skip to content

Commit f45dec5

Browse files
author
Will McGinnis
committed
added mean value example with custom reducer
1 parent 5663d3f commit f45dec5

File tree

7 files changed

+1079
-3
lines changed

7 files changed

+1079
-3
lines changed

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ __pycache__/
55
*.py[cod]
66
*$py.class
77
.idea
8+
.idea/
89
# C extensions
910
*.so
1011

@@ -60,3 +61,5 @@ docs/_build/
6061
target/
6162

6263

64+
65+
.idea/workspace.xml

README.md

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,11 @@ In this example, we have row-wise json in one file, with an attribute field that
4040
colors. So we load both datasets in, convert the json data into a ordered and typed tuple, and join then two together
4141
to get a nice dataset of cars and their colors.
4242

43+
Mean Values
44+
-----------
45+
46+
Takes in a csv with two columns and finds the mean of each column, using a custom reducer function. Afterwards, it
47+
formats a string nicely with the output and dumps that onto disk.
4348

4449
Features
4550
========
@@ -52,10 +57,11 @@ Text data-source (read\_text)
5257
* trending hashtags
5358
* data enrichment
5459

55-
CSV data-srouce (read\_csv)
60+
CSV data-source (read\_csv)
5661
---------------------------
5762

5863
* data enrichment
64+
* mean values
5965

6066
String data-source (from\_elements)
6167
-----------------------------------
@@ -67,6 +73,7 @@ Text output (write\_text)
6773

6874
* word count
6975
* data enrichment
76+
* mean values
7077

7178
CSV output (write\_csv)
7279
-----------------------
@@ -83,6 +90,7 @@ Transformations: Map
8390
* word count
8491
* trending hashtags
8592
* data enrichment
93+
* mean values
8694

8795
Transformations: FlatMap
8896
------------------------
@@ -102,7 +110,8 @@ Transformations: Filter
102110
Transformations: Reduce
103111
-----------------------
104112

105-
113+
* mean values
114+
106115
Transformations: ReduceGroup
107116
----------------------------
108117

mean_values/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
__author__ = 'willmcginnis'

0 commit comments

Comments
 (0)