forked from P7h/IntroToHadoopAndMR__Udacity_Course
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Updated README; Added exec logs as well.
Added the execution log files for each question of the Course. Also updated README to reflect these changes; Also removed references and updated references for Input files.
- Loading branch information
Showing
15 changed files
with
642 additions
and
13 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| [training@udacity P1]$ time hadoop jar pur.jar org.p7h.udacityhadoop.p1.P1 pur pur_p1 | ||
| 13/11/25 16:15:34 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. | ||
| 13/11/25 16:15:34 INFO input.FileInputFormat: Total input paths to process : 1 | ||
| 13/11/25 16:15:34 WARN snappy.LoadSnappy: Snappy native library is available | ||
| 13/11/25 16:15:34 INFO snappy.LoadSnappy: Snappy native library loaded | ||
| 13/11/25 16:15:35 INFO mapred.JobClient: Running job: job_201311251433_0050 | ||
| 13/11/25 16:15:36 INFO mapred.JobClient: map 0% reduce 0% | ||
| 13/11/25 16:15:46 INFO mapred.JobClient: map 37% reduce 0% | ||
| 13/11/25 16:15:48 INFO mapred.JobClient: map 50% reduce 0% | ||
| 13/11/25 16:15:53 INFO mapred.JobClient: map 75% reduce 0% | ||
| 13/11/25 16:15:56 INFO mapred.JobClient: map 97% reduce 25% | ||
| 13/11/25 16:15:57 INFO mapred.JobClient: map 100% reduce 25% | ||
| 13/11/25 16:15:59 INFO mapred.JobClient: map 100% reduce 100% | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Job complete: job_201311251433_0050 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Counters: 32 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: File System Counters | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: FILE: Number of bytes read=44576 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: FILE: Number of bytes written=965870 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: FILE: Number of read operations=0 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: FILE: Number of large read operations=0 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: FILE: Number of write operations=0 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: HDFS: Number of bytes read=211325679 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: HDFS: Number of bytes written=2920 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: HDFS: Number of read operations=9 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: HDFS: Number of large read operations=0 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: HDFS: Number of write operations=1 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Job Counters | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Launched map tasks=4 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Launched reduce tasks=1 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Data-local map tasks=4 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Total time spent by all maps in occupied slots (ms)=34737 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Total time spent by all reduces in occupied slots (ms)=11166 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Map-Reduce Framework | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Map input records=4138476 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Map output records=4138476 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Map output bytes=72926554 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Input split bytes=464 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Combine input records=4140330 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Combine output records=2266 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Reduce input groups=103 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Reduce shuffle bytes=8108 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Reduce input records=412 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Reduce output records=103 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Spilled Records=2678 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: CPU time spent (ms)=25880 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Physical memory (bytes) snapshot=825233408 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1940406272 | ||
| 13/11/25 16:16:00 INFO mapred.JobClient: Total committed heap usage (bytes)=659308544 | ||
|
|
||
| real 0m26.566s | ||
| user 0m1.036s | ||
| sys 0m0.404s |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| [training@udacity P1]$ time hadoop jar pur.jar org.p7h.udacityhadoop.p1.P1Q1 pur pur_p1q1 | ||
| 13/11/25 16:16:49 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. | ||
| 13/11/25 16:16:49 INFO input.FileInputFormat: Total input paths to process : 1 | ||
| 13/11/25 16:16:49 WARN snappy.LoadSnappy: Snappy native library is available | ||
| 13/11/25 16:16:49 INFO snappy.LoadSnappy: Snappy native library loaded | ||
| 13/11/25 16:16:49 INFO mapred.JobClient: Running job: job_201311251433_0051 | ||
| 13/11/25 16:16:50 INFO mapred.JobClient: map 0% reduce 0% | ||
| 13/11/25 16:16:59 INFO mapred.JobClient: map 41% reduce 0% | ||
| 13/11/25 16:17:01 INFO mapred.JobClient: map 50% reduce 0% | ||
| 13/11/25 16:17:05 INFO mapred.JobClient: map 75% reduce 0% | ||
| 13/11/25 16:17:09 INFO mapred.JobClient: map 96% reduce 25% | ||
| 13/11/25 16:17:10 INFO mapred.JobClient: map 100% reduce 25% | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: map 100% reduce 100% | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Job complete: job_201311251433_0051 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Counters: 32 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: File System Counters | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: FILE: Number of bytes read=8342 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: FILE: Number of bytes written=923128 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: FILE: Number of read operations=0 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: FILE: Number of large read operations=0 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: FILE: Number of write operations=0 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: HDFS: Number of bytes read=211325679 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: HDFS: Number of bytes written=557 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: HDFS: Number of read operations=8 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: HDFS: Number of large read operations=0 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: HDFS: Number of write operations=1 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Job Counters | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Launched map tasks=4 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Launched reduce tasks=1 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Data-local map tasks=4 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Total time spent by all maps in occupied slots (ms)=32894 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Total time spent by all reduces in occupied slots (ms)=10707 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Map-Reduce Framework | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Map input records=4138476 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Map output records=4138476 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Map output bytes=77711522 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Input split bytes=464 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Combine input records=4138800 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Combine output records=396 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Reduce input groups=18 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Reduce shuffle bytes=1520 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Reduce input records=72 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Reduce output records=18 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Spilled Records=468 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: CPU time spent (ms)=24010 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Physical memory (bytes) snapshot=824774656 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1940406272 | ||
| 13/11/25 16:17:12 INFO mapred.JobClient: Total committed heap usage (bytes)=659308544 | ||
|
|
||
| real 0m24.417s | ||
| user 0m0.824s | ||
| sys 0m0.525s |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| [training@udacity P1]$ time hadoop jar pur.jar org.p7h.udacityhadoop.p1.P1Q2 pur pur_p1q2 | ||
| 13/11/25 16:18:14 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. | ||
| 13/11/25 16:18:14 INFO input.FileInputFormat: Total input paths to process : 1 | ||
| 13/11/25 16:18:14 WARN snappy.LoadSnappy: Snappy native library is available | ||
| 13/11/25 16:18:14 INFO snappy.LoadSnappy: Snappy native library loaded | ||
| 13/11/25 16:18:15 INFO mapred.JobClient: Running job: job_201311251433_0052 | ||
| 13/11/25 16:18:16 INFO mapred.JobClient: map 0% reduce 0% | ||
| 13/11/25 16:18:26 INFO mapred.JobClient: map 31% reduce 0% | ||
| 13/11/25 16:18:29 INFO mapred.JobClient: map 50% reduce 0% | ||
| 13/11/25 16:18:33 INFO mapred.JobClient: map 75% reduce 0% | ||
| 13/11/25 16:18:37 INFO mapred.JobClient: map 92% reduce 25% | ||
| 13/11/25 16:18:39 INFO mapred.JobClient: map 100% reduce 25% | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: map 100% reduce 100% | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Job complete: job_201311251433_0052 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Counters: 32 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: File System Counters | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: FILE: Number of bytes read=44576 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: FILE: Number of bytes written=965950 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: FILE: Number of read operations=0 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: FILE: Number of large read operations=0 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: FILE: Number of write operations=0 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: HDFS: Number of bytes read=211325679 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: HDFS: Number of bytes written=1712 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: HDFS: Number of read operations=9 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: HDFS: Number of large read operations=0 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: HDFS: Number of write operations=1 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Job Counters | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Launched map tasks=4 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Launched reduce tasks=1 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Data-local map tasks=4 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Total time spent by all maps in occupied slots (ms)=37263 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Total time spent by all reduces in occupied slots (ms)=11899 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Map-Reduce Framework | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Map input records=4138476 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Map output records=4138476 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Map output bytes=72926554 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Input split bytes=464 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Combine input records=4140330 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Combine output records=2266 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Reduce input groups=103 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Reduce shuffle bytes=8108 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Reduce input records=412 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Reduce output records=103 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Spilled Records=2678 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: CPU time spent (ms)=28360 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Physical memory (bytes) snapshot=825094144 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1940406272 | ||
| 13/11/25 16:18:41 INFO mapred.JobClient: Total committed heap usage (bytes)=659308544 | ||
|
|
||
| real 0m27.436s | ||
| user 0m0.828s | ||
| sys 0m0.545s |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| [training@udacity P1]$ time hadoop jar pur.jar org.p7h.udacityhadoop.p1.P1Q3 pur pur_p1q3 | ||
| 13/11/25 16:19:32 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. | ||
| 13/11/25 16:19:32 INFO input.FileInputFormat: Total input paths to process : 1 | ||
| 13/11/25 16:19:32 WARN snappy.LoadSnappy: Snappy native library is available | ||
| 13/11/25 16:19:32 INFO snappy.LoadSnappy: Snappy native library loaded | ||
| 13/11/25 16:19:33 INFO mapred.JobClient: Running job: job_201311251433_0053 | ||
| 13/11/25 16:19:34 INFO mapred.JobClient: map 0% reduce 0% | ||
| 13/11/25 16:19:44 INFO mapred.JobClient: map 42% reduce 0% | ||
| 13/11/25 16:19:46 INFO mapred.JobClient: map 50% reduce 0% | ||
| 13/11/25 16:19:50 INFO mapred.JobClient: map 75% reduce 0% | ||
| 13/11/25 16:19:54 INFO mapred.JobClient: map 93% reduce 25% | ||
| 13/11/25 16:19:56 INFO mapred.JobClient: map 100% reduce 25% | ||
| 13/11/25 16:20:00 INFO mapred.JobClient: map 100% reduce 33% | ||
| 13/11/25 16:20:03 INFO mapred.JobClient: map 100% reduce 85% | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: map 100% reduce 100% | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Job complete: job_201311251433_0053 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Counters: 32 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: File System Counters | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: FILE: Number of bytes read=193952736 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: FILE: Number of bytes written=294192115 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: FILE: Number of read operations=0 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: FILE: Number of large read operations=0 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: FILE: Number of write operations=0 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: HDFS: Number of bytes read=211325679 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: HDFS: Number of bytes written=29 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: HDFS: Number of read operations=8 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: HDFS: Number of large read operations=0 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: HDFS: Number of write operations=1 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Job Counters | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Launched map tasks=4 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Launched reduce tasks=1 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Data-local map tasks=4 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Total time spent by all maps in occupied slots (ms)=37028 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Total time spent by all reduces in occupied slots (ms)=20084 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Map-Reduce Framework | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Map input records=4138476 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Map output records=4138476 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Map output bytes=91046472 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Input split bytes=464 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Combine input records=0 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Combine output records=0 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Reduce input groups=1 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Reduce shuffle bytes=99323448 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Reduce input records=4138476 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Reduce output records=1 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Spilled Records=12219835 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: CPU time spent (ms)=31220 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Physical memory (bytes) snapshot=960282624 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Virtual memory (bytes) snapshot=1940406272 | ||
| 13/11/25 16:20:06 INFO mapred.JobClient: Total committed heap usage (bytes)=812261376 | ||
|
|
||
| real 0m34.663s | ||
| user 0m0.881s | ||
| sys 0m0.549s |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| [training@udacity P1]$ time hs P1_Mapper.py P1_Reducer.py pur pur_p1 | ||
| packageJobJar: [P1_Mapper.py, P1_Reducer.py, /tmp/hadoop-training/hadoop-unjar1202599279492459260/] [] /tmp/streamjob8266157901554607271.jar tmpDir=null | ||
| 13/11/25 16:03:26 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. | ||
| 13/11/25 16:03:26 WARN snappy.LoadSnappy: Snappy native library is available | ||
| 13/11/25 16:03:26 INFO snappy.LoadSnappy: Snappy native library loaded | ||
| 13/11/25 16:03:26 INFO mapred.FileInputFormat: Total input paths to process : 1 | ||
| 13/11/25 16:03:26 INFO streaming.StreamJob: getLocalDirs(): [/var/lib/hadoop-hdfs/cache/training/mapred/local] | ||
| 13/11/25 16:03:26 INFO streaming.StreamJob: Running job: job_201311251433_0043 | ||
| 13/11/25 16:03:26 INFO streaming.StreamJob: To kill this job, run: | ||
| 13/11/25 16:03:26 INFO streaming.StreamJob: UNDEF/bin/hadoop job -Dmapred.job.tracker=0.0.0.0:8021 -kill job_201311251433_0043 | ||
| 13/11/25 16:03:26 INFO streaming.StreamJob: Tracking URL: http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201311251433_0043 | ||
| 13/11/25 16:03:27 INFO streaming.StreamJob: map 0% reduce 0% | ||
| 13/11/25 16:03:37 INFO streaming.StreamJob: map 32% reduce 0% | ||
| 13/11/25 16:03:40 INFO streaming.StreamJob: map 50% reduce 0% | ||
| 13/11/25 16:03:46 INFO streaming.StreamJob: map 75% reduce 0% | ||
| 13/11/25 16:03:49 INFO streaming.StreamJob: map 93% reduce 25% | ||
| 13/11/25 16:03:52 INFO streaming.StreamJob: map 100% reduce 25% | ||
| 13/11/25 16:03:55 INFO streaming.StreamJob: map 100% reduce 33% | ||
| 13/11/25 16:03:58 INFO streaming.StreamJob: map 100% reduce 72% | ||
| 13/11/25 16:04:01 INFO streaming.StreamJob: map 100% reduce 79% | ||
| 13/11/25 16:04:04 INFO streaming.StreamJob: map 100% reduce 86% | ||
| 13/11/25 16:04:07 INFO streaming.StreamJob: map 100% reduce 93% | ||
| 13/11/25 16:04:11 INFO streaming.StreamJob: map 100% reduce 100% | ||
| 13/11/25 16:04:13 INFO streaming.StreamJob: Job complete: job_201311251433_0043 | ||
| 13/11/25 16:04:13 INFO streaming.StreamJob: Output: pur_p1 | ||
|
|
||
| real 0m47.573s | ||
| user 0m0.888s | ||
| sys 0m0.554s |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| [training@udacity P1]$ time hs P1Q1_Mapper.py P1Q1_Reducer.py pur pur_p1q1 | ||
| packageJobJar: [P1Q1_Mapper.py, P1Q1_Reducer.py, /tmp/hadoop-training/hadoop-unjar2600752056635677193/] [] /tmp/streamjob3117925569824943428.jar tmpDir=null | ||
| 13/11/25 16:05:09 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. | ||
| 13/11/25 16:05:09 WARN snappy.LoadSnappy: Snappy native library is available | ||
| 13/11/25 16:05:09 INFO snappy.LoadSnappy: Snappy native library loaded | ||
| 13/11/25 16:05:09 INFO mapred.FileInputFormat: Total input paths to process : 1 | ||
| 13/11/25 16:05:09 INFO streaming.StreamJob: getLocalDirs(): [/var/lib/hadoop-hdfs/cache/training/mapred/local] | ||
| 13/11/25 16:05:09 INFO streaming.StreamJob: Running job: job_201311251433_0044 | ||
| 13/11/25 16:05:09 INFO streaming.StreamJob: To kill this job, run: | ||
| 13/11/25 16:05:09 INFO streaming.StreamJob: UNDEF/bin/hadoop job -Dmapred.job.tracker=0.0.0.0:8021 -kill job_201311251433_0044 | ||
| 13/11/25 16:05:09 INFO streaming.StreamJob: Tracking URL: http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201311251433_0044 | ||
| 13/11/25 16:05:10 INFO streaming.StreamJob: map 0% reduce 0% | ||
| 13/11/25 16:05:19 INFO streaming.StreamJob: map 33% reduce 0% | ||
| 13/11/25 16:05:22 INFO streaming.StreamJob: map 50% reduce 0% | ||
| 13/11/25 16:05:31 INFO streaming.StreamJob: map 50% reduce 17% | ||
| 13/11/25 16:05:32 INFO streaming.StreamJob: map 75% reduce 17% | ||
| 13/11/25 16:05:34 INFO streaming.StreamJob: map 94% reduce 25% | ||
| 13/11/25 16:05:36 INFO streaming.StreamJob: map 100% reduce 25% | ||
| 13/11/25 16:05:43 INFO streaming.StreamJob: map 100% reduce 75% | ||
| 13/11/25 16:05:46 INFO streaming.StreamJob: map 100% reduce 82% | ||
| 13/11/25 16:05:49 INFO streaming.StreamJob: map 100% reduce 90% | ||
| 13/11/25 16:05:52 INFO streaming.StreamJob: map 100% reduce 100% | ||
| 13/11/25 16:05:55 INFO streaming.StreamJob: Job complete: job_201311251433_0044 | ||
| 13/11/25 16:05:55 INFO streaming.StreamJob: Output: pur_p1q1 | ||
|
|
||
| real 0m47.695s | ||
| user 0m0.811s | ||
| sys 0m0.662s |
Oops, something went wrong.