File tree Expand file tree Collapse file tree 1 file changed +20
-0
lines changed Expand file tree Collapse file tree 1 file changed +20
-0
lines changed Original file line number Diff line number Diff line change @@ -272,6 +272,26 @@ <h2 id="project_tagline">This page is a summary to keep the track of Hadoop rela
272
272
< br > < a href ="http://www.tinkerpop.com/ "> 2. TinkerPop site</ a > </ td >
273
273
</ tr >
274
274
275
+ < tr >
276
+ < td width ="20% "> Pachyderm MapReduce</ td >
277
+ < td > Pachyderm is a completely new MapReduce engine built on top Docker and CoreOS.
278
+ In Pachyderm MapReduce (PMR) a job is an HTTP server inside a Docker container
279
+ (a microservice). You give Pachyderm a Docker image and it will automatically
280
+ distribute it throughout the cluster next to your data. Data is POSTed to
281
+ the container over HTTP and the results are stored back in the file system.
282
+ You can implement the web server in any language you want and pull in any library.
283
+ Pachyderm also creates a DAG for all the jobs in the system and their dependencies
284
+ and it automatically schedules the pipeline such that each job isn’t run until it’s
285
+ dependencies have completed. Everything in Pachyderm “speaks in diffs” so it knows
286
+ exactly which data has changed and which subsets of the pipeline need to be rerun.
287
+ CoreOS is an open source lightweight operating system based on Chrome OS, actually
288
+ CoreOS is a fork of Chrome OS. CoreOS provides only the minimal functionality
289
+ required for deploying applications inside software containers, together with
290
+ built-in mechanisms for service discovery and configuration sharing</ td >
291
+ < td width ="20% "> < a href ="http://www.pachyderm.io/ "> 1. Pachyderm site</ a >
292
+ < br > < a href ="https://medium.com/pachyderm-data/lets-build-a-modern-hadoop-4fc160f8d74f "> 2. Pachyderm introduction article</ a > </ td >
293
+ </ tr >
294
+
275
295
276
296
<!---------------------------->
277
297
<!-- NoSQL ecosystem -->
You can’t perform that action at this time.
0 commit comments