Spark - Apache Iceberg Examples

This project shows how to manipulate Apache Iceberg Tables using data from CSV files

Data CSV Files

• ./minio-data/csv_files/airlines.csv: Version 1 of airlines companies
• ./minio-data/csv_files/airlines_v2.csv: Version 2 of airlines companies (Include Country field)
• ./minio-data/csv_files/airports.csv: Version 1 of airports
• ./minio-data/csv_files/flights.csv: Version 1 of fights

Used Technologies

• Java 11
• Docker
• Spark 3.5
• Apache Iceberg

How to Run

• Build project: ./gradlew clean build
• Start docker-compose dependencies: docker-compose up -d

Classes

• Example1 shows how to create Apache Iceberg tables from CSV files using Spark
• Example2 shows how to query on Iceberg tables using Spark SQL
• Example3 shows how to version an Apache Iceberg table when need to create new field
• Example4 shows how to purge data using delete command
• Example5 shows how to make table maintenance