/projects
AWS YouTube Analytics
An AWS-based ETL pipeline that automates the processing and analysis of YouTube trending videos data, converting raw files into optimized datasets for analytics-ready insights
Highlights
- Built a scalable AWS ETL pipeline for automating YouTube Analytics ingestion, transformation, and storage using a Medallion Architecture
- Optimized data processing with JSON to Parquet conversion, Hive-style regional partitioning (CA, GB, US), predicate pushdown, and centralized schema management
- Delivered analytics-ready datasets from the YouTube Trending Videos Statistics dataset to enable fast and reliable insights
Technologies
AWS GlueAWS LambdaAWS S3AWS Glue Data CatalogApache ParquetPythonPySparkAWS Data Wrangler