Skip to content

Commit 98c2a00

Browse files
Ken Takagiwagiwa
Ken Takagiwa
authored andcommitted
added count operation but this implementation need double check
1 parent 58591d2 commit 98c2a00

File tree

1 file changed

+6
-0
lines changed

1 file changed

+6
-0
lines changed

python/pyspark/streaming/dstream.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,12 @@ def combineByKey(self, createCombiner, mergeValue, mergeCombiners,
139139
def combineLocally(iterator):
140140
combiners = {}
141141
for x in iterator:
142+
143+
#TODO for count operation make sure count implementation
144+
# This is different from what pyspark does
145+
if isinstance(x, int):
146+
x = ("", x)
147+
142148
(k, v) = x
143149
if k not in combiners:
144150
combiners[k] = createCombiner(v)

0 commit comments

Comments
 (0)