-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathout_group_4.txt
executable file
·82 lines (82 loc) · 4.5 KB
/
out_group_4.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
2017-12-13 16:49:05,926 INFO org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl (main): Timeline service address: http://ip-172-31-87-71.ec2.internal:8188/ws/v1/timeline/
2017-12-13 16:49:05,942 INFO org.apache.hadoop.yarn.client.RMProxy (main): Connecting to ResourceManager at ip-172-31-87-71.ec2.internal/172.31.87.71:8032
2017-12-13 16:49:07,007 WARN org.apache.hadoop.mapreduce.JobResourceUploader (main): Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2017-12-13 16:49:07,795 INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat (main): Total input paths to process : 1
2017-12-13 16:49:07,806 INFO com.hadoop.compression.lzo.GPLNativeCodeLoader (main): Loaded native gpl library
2017-12-13 16:49:07,809 INFO com.hadoop.compression.lzo.LzoCodec (main): Successfully loaded & initialized native-lzo library [hadoop-lzo rev d73c901b4228f4e75d3a527ec2318ce7376036cb]
2017-12-13 16:49:08,000 INFO org.apache.hadoop.mapreduce.JobSubmitter (main): number of splits:1
2017-12-13 16:49:08,157 INFO org.apache.hadoop.mapreduce.JobSubmitter (main): Submitting tokens for job: job_1513183628261_0001
2017-12-13 16:49:08,535 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl (main): Submitted application application_1513183628261_0001
2017-12-13 16:49:08,584 INFO org.apache.hadoop.mapreduce.Job (main): The url to track the job: http://ip-172-31-87-71.ec2.internal:20888/proxy/application_1513183628261_0001/
2017-12-13 16:49:08,584 INFO org.apache.hadoop.mapreduce.Job (main): Running job: job_1513183628261_0001
2017-12-13 16:49:19,687 INFO org.apache.hadoop.mapreduce.Job (main): Job job_1513183628261_0001 running in uber mode : false
2017-12-13 16:49:19,688 INFO org.apache.hadoop.mapreduce.Job (main): map 0% reduce 0%
2017-12-13 16:49:27,756 INFO org.apache.hadoop.mapreduce.Job (main): map 100% reduce 0%
2017-12-13 16:49:40,856 INFO org.apache.hadoop.mapreduce.Job (main): map 100% reduce 14%
2017-12-13 16:49:42,866 INFO org.apache.hadoop.mapreduce.Job (main): map 100% reduce 43%
2017-12-13 16:49:45,879 INFO org.apache.hadoop.mapreduce.Job (main): map 100% reduce 71%
2017-12-13 16:49:46,883 INFO org.apache.hadoop.mapreduce.Job (main): map 100% reduce 86%
2017-12-13 16:49:47,888 INFO org.apache.hadoop.mapreduce.Job (main): map 100% reduce 100%
2017-12-13 16:49:47,895 INFO org.apache.hadoop.mapreduce.Job (main): Job job_1513183628261_0001 completed successfully
2017-12-13 16:49:48,125 INFO org.apache.hadoop.mapreduce.Job (main): Counters: 55
File System Counters
FILE: Number of bytes read=273
FILE: Number of bytes written=1018401
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=104
HDFS: Number of bytes written=0
HDFS: Number of read operations=1
HDFS: Number of large read operations=0
HDFS: Number of write operations=0
S3: Number of bytes read=140408
S3: Number of bytes written=110
S3: Number of read operations=0
S3: Number of large read operations=0
S3: Number of write operations=0
Job Counters
Killed reduce tasks=1
Launched map tasks=1
Launched reduce tasks=7
Data-local map tasks=1
Total time spent by all maps in occupied slots (ms)=275940
Total time spent by all reduces in occupied slots (ms)=7912080
Total time spent by all map tasks (ms)=6132
Total time spent by all reduce tasks (ms)=87912
Total vcore-milliseconds taken by all map tasks=6132
Total vcore-milliseconds taken by all reduce tasks=87912
Total megabyte-milliseconds taken by all map tasks=8830080
Total megabyte-milliseconds taken by all reduce tasks=253186560
Map-Reduce Framework
Map input records=6040
Map output records=6040
Map output bytes=48098
Map output materialized bytes=245
Input split bytes=104
Combine input records=6040
Combine output records=14
Reduce input groups=14
Reduce shuffle bytes=245
Reduce input records=14
Reduce output records=14
Spilled Records=28
Shuffled Maps =7
Failed Shuffles=0
Merged Map outputs=7
GC time elapsed (ms)=2743
CPU time spent (ms)=11970
Physical memory (bytes) snapshot=3493593088
Virtual memory (bytes) snapshot=34742456320
Total committed heap usage (bytes)=3572498432
Shuffle Errors
BAD_ID=0
CONNECTION=0
IO_ERROR=0
WRONG_LENGTH=0
WRONG_MAP=0
WRONG_REDUCE=0
File Input Format Counters
Bytes Read=140408
File Output Format Counters
Bytes Written=110