Demonstration of using Python to process the Common Crawl dataset with the mrjob framework
-
Updated
Jan 22, 2026 - Python
#
map-reduce
Here are 108 public repositories matching this topic...
Prosto is a data processing toolkit radically changing how data is processed by heavily relying on functions and operations with functions - an alternative to map-reduce and join-groupby
python workflow data-science spark pandas map-reduce business-intelligence olap data-wrangling data-preprocessing feature-engineering data-processing data-preparation
-
Updated
Nov 21, 2021 - Python
Efficient and scalable parallelism using the message passing interface (MPI) to handle big data and highly computational problems.
-
Updated
Nov 11, 2016 - Python
RedisGears python client
-
Updated
Jun 19, 2023 - Python
There are Python 2.7 codes and learning notes for Spark 2.1.1
spark map-reduce minhash tf-idf kmeans als cosine-similarity python27 kmeans-clustering minhash-lsh-algorithm apriori-algorithm alternating-least-squares uv-decomposition savasere-omiecinski-and-navathe apriori-son
-
Updated
Aug 21, 2018 - Python
Code for paper "Locally Distributed Deep Learning Inference on Edge Device Clusters"
deep-learning cluster parallel-computing distributed-computing inference dnn map-reduce vggnet parallel-algorithm speedup workload edge-computing
-
Updated
Aug 22, 2025 - Python
Iterable Java8 style Streams for Python
python streaming python3 java-streams map-reduce mapreduce python-streaming python-multiprocessing python-itertools python-multithreading python-iterables python-mapreduce
-
Updated
Dec 24, 2025 - Python
A tool that converts long audio files into a thorough, summarized report. Leverages OpenAI and its API (ChatGPT backend), Langchain for text processing, and Pinecone for vector database facilitation.
python chatbot artificial-intelligence map-reduce openai embedding-models object-oriented-programming pinecone vector-database openai-api langchain
-
Updated
Feb 27, 2024 - Python
Distributed encoding, second generation.
-
Updated
Aug 4, 2020 - Python
Source code of the numerical experiments presented in "Energy-Efficient Edge-Facilitated Wireless Collaborative Computing using Map-Reduce" by Antoine Paris, Hamed Mirghasemi, Ivan Stupia and Luc Vandendorpe (presented at SPAWC19).
distributed-computing internet-of-things map-reduce fog-computing sensor-network convex-optimization energy-efficiency wireless-communication mobile-computing cyber-physical-systems wireless-collaborative-computing
-
Updated
Nov 5, 2019 - Python
🎓Repository for masters labs on FCSN, BSUIR
machine-learning tcp aws-lambda study dsp oop blockchain uml map-reduce bsuir labs neural-networks plc hyperledger hyperledger-fabric digital-signal-processing tcp-chat traffic-light bsuir-labworks
-
Updated
Jan 7, 2023 - Python
A package for working with lists distributed over MPI
-
Updated
Sep 8, 2023 - Python
Implementation of Girvan-Newman Algorithm to detect communities in graphs using Yelp dataset
data-mining community-detection map-reduce betweenness breadth-first-search social-graph yelp-dataset girvan-newman spark-rdd
-
Updated
Jul 16, 2019 - Python
Scatter gather with AWS lambda
-
Updated
Oct 4, 2020 - Python
Parallel implementation of Breadth-First Search algorith in Java MapReduce and PySpark. This implementation finds degrees of separation between Twitter Users
map big-data spark hadoop bigdata parallel parallelism map-reduce pyspark reduce bfs hadoop-mapreduce hadoo
-
Updated
Oct 23, 2018 - Python
Learn Big Data tools/ framework by doing examples, POC, per projects.
java airflow scala kafka big-data spark hive hadoop bigdata hbase python3 map-reduce sqoop case-study hadoop-mapreduce
-
Updated
Jul 29, 2022 - Python
A case study on mining association rules between different factors related to deaths of people in the United States
python data-science machine-learning data-mining hadoop mining map-reduce mapreduce association-rules hadoop-mapreduce hadoop-streaming
-
Updated
Jun 24, 2017 - Python
This repository presents a Python-powered analysis tool using MRJob and MongoDB to identify top-selling music artists by decade. With a focus on big data processing, it serves as a valuable resource for understanding historical and current musical trends, providing a streamlined solution for music industry analytics.
-
Updated
Apr 13, 2024 - Python
Improve this page
Add a description, image, and links to the map-reduce topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the map-reduce topic, visit your repo's landing page and select "manage topics."