Skip to content

Commit 61389f7

Browse files
add boring words filter to wordcount
1 parent 482d747 commit 61389f7

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

project4/src/main/java/com/jobreadyprogrammer/spark/WordCount.java

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,8 @@ public void start() {
3737
" 'for', 'if', 'in', 'into', 'is', 'it',\r\n" +
3838
" 'no', 'not', 'of', 'on', 'or', 'such',\r\n" +
3939
" 'that', 'the', 'their', 'then', 'there', 'these',\r\n" +
40-
" 'they', 'this', 'to', 'was', 'will', 'with', 'he', 'she')";
40+
" 'they', 'this', 'to', 'was', 'will', 'with', 'he', 'she', 'your', 'you', 'I', "
41+
+ " 'i','[',']', '[]', 'his', 'him', 'our', 'we')";
4142

4243
Dataset<Row> df2 = lineDS.toDF();
4344
df2 = df2.groupBy("value").count();

0 commit comments

Comments
 (0)