Skip to content

Simple data extraction pipeline using Python to consume a REST API and store data in Parquet format. Common pattern used by Data Engineers.

Notifications You must be signed in to change notification settings

Cardoso-leo/api-data-extraction

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 

Repository files navigation

API Data Extraction with Python

This project demonstrates a simple and common Data Engineering pattern:
extracting data from a REST API, transforming it into a structured format, and storing it as a Parquet file.

Technologies Used

  • Python
  • Requests
  • Pandas
  • PyArrow (Parquet)

What this project does

  • Consumes a public REST API
  • Validates the HTTP response
  • Converts JSON data into a Pandas DataFrame
  • Adds extraction metadata
  • Stores the data in Parquet format

About

Simple data extraction pipeline using Python to consume a REST API and store data in Parquet format. Common pattern used by Data Engineers.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages