Data Engineer (2.6+ years) — building scalable, production-grade data platforms on Azure and Databricks.

I design and deliver scalable data engineering solutions for analytics and ML. My focus areas are reliable ETL/ELT pipelines, Delta Lake-backed data lakes, PySpark performance tuning, and enterprise data delivery on Azure (ADF, Databricks, ADLS, Fabric). I combine engineering rigor (testing, monitoring, deployment) with a strong emphasis on data quality and cost-efficient architecture.

Key strengths:

• Scalable ETL/ELT (batch and micro-batch) using PySpark and Databricks
• Delta Lake architecture, partitioning, and schema evolution strategies
• Azure ecosystem: Data Factory, Databricks, ADLS Gen2, Fabric
• SQL performance optimization, data modeling (star schema), CDC and incremental loads
• Production ops: CI/CD, monitoring, cost tuning, and observability
• Integrating GenAI & RAG patterns into data workflows (embeddings, vector DBs)

Python • PySpark • SQL • Scala (basics) • Git • Bash

Apache Spark • Databricks • Delta Lake • Parquet • Spark SQL • Structured Streaming

Azure Data Factory • ADLS Gen2 • Azure Databricks • Microsoft Fabric

Data Lake (Bronze/Silver/Gold) • CDC & Incremental Loads • Data Modeling (Fact & Dimension) • ETL Testing • Observability & Instrumentation

Apache Kafka • Structured Streaming • Micro-batch processing • Checkpointing & Exactly-once patterns

Vector DBs • Embeddings • Prompt engineering • RAG pipeline design

• ETL / ELT pipeline design and implementation (PySpark & Databricks)
• Delta Lake design: partitioning, compaction, and schema evolution
• Incremental processing & CDC
• SQL query optimization & Spark performance tuning
• Data validation, reconciliation and testing frameworks
• CI/CD and operationalizing data pipelines

• Databricks Certified: Data Engineer Associate
• Databricks Accreditation - AI Agent Fundamentals
• Microsoft Certified: Fabric Data Engineer Associate
• Microsoft Certified: Azure AI Fundamentals
• Microsoft Certified: GitHub Copilot
• Google Cloud Certified: Generative AI Leader
• Google Cloud Certified: Cloud Digital Leader
• HackerRank Certified: SQL (Advanced)

• Email: takeswapnil8@gmail.com
• LinkedIn: https://linkedin.com/in/swapniltake1/
• Open to Data Engineering roles and consulting

⭐ If you find my work interesting, consider starring my repositories! Thank you for visiting :)