/projects
ShopIndex - Moroccan Vendor Discovery and Price Index
Python data pipeline that discovers Moroccan vendors, collects public business data from Facebook and websites, and builds a searchable price index.
Highlights
- Discovers Moroccan businesses through OpenStreetMap, Wikidata, search engines, seeds, and Facebook Pages
- Scrapes and normalizes vendor identity, contact details, locations, categories, audience metrics, and price signals
- Crawls vendor websites and merges website and Facebook data into one deduplicated dataset
Technologies
PythonSQLiteBeautifulSoupPlaywrightWeb ScrapingData PipelinesPytest