Back to Home
Data EngineeringCompleted
AWS E-commerce ETL Pipeline
End-to-end AWS data pipeline processing e-commerce data from CSV and APIs into a scalable data lake architecture.
Tech Stack
PythonAWS S3GlueLambdaAthenaSQL
Problem
Data from multiple sources (CSV transactions + product API) needed to be unified into a centralized analytics system.
Solution
Built a serverless AWS pipeline using S3, Lambda ingestion, Glue ETL, and Athena querying.
Architecture
- CSV + API ingestion layer
- AWS Lambda for API ingestion
- S3 data lake (raw + processed layers)
- AWS Glue ETL transformation
- Athena SQL analytics layer
Results
- Built full serverless AWS data pipeline
- Implemented star schema modeling
- Enabled scalable SQL analytics via Athena