A curated list of awesome System Design (A.K.A. Distributed Systems) resources.

DE직무에 필요한 모든 것

HadoopOffice - Analyze Office documents using the Hadoop ecosystem (Spark/Flink/Hive)

IBIS is a workflow creation-engine that abstracts the Hadoop internals of ingesting RDBMS data.

Life-cycle: Internal working of HDFS, SQOOP, HIVE, SPARK, HBASE, KAFKA with code.

Hadoop3.2 single/cluster mode with web terminal gotty, spark, jupyter pyspark, hive, eco etc.

Instructions on setting up Hadoop, HDFS, java, sbt, kafka, scala, spark and flume on Ubuntu 18.04

Dockerfile for running Apache Knox (http://knox.apache.org/) in Docker

This project sets up a Hadoop High Availability (HA) Cluster using Docker Compose with three master nodes and two worker nodes for fault-tolerant big data processing. It includes Zookeeper & JournalNodes for automatic NameNode failover, ensuring scalability & reliability.

The goal of this project is to identify the flood-prone areas with probabilities of flood in counties in a future date, using Spark MLLib.

Analysis of YouTube Data using Hadoop Mapreduce framework in Java.

Built a Large Scale Distributed Data Processing system for Streaming Analytics using Hadoop Ecosystem (Apache Spark and HDFS), in Cloud for real-time spatial analytics.

SparkSQL Quick Start Tutorial

Helm chart for Apache Knox

EMR 5.25.0 cluster single node Hadoop docker image. With Amazon Linux, Hadoop 2.8.5 and Hive 2.3.5

Spark Streaming & Kafka Quick Start Tutorial

This project focuses on analyzing movie data using Pyspark tailored for efficient data processing on Hadoop Distributed File System (HDFS)

[BigData] one year weblog analysis using PIG

Practise programs in hadoop ecosystem for refrence

Big Data is Stored and analyzed of various Customer using Hadoop and other tools like Hive, Zookeeper, Hbase and sqoop and all details of the customer is analyzed then result are given.This result is very useful for companies.