/projects
Revealing Key Insights from GitHub Topics with Web Scraping
Web scraping project that scrapes GitHub Topics and the top repositories per topic to produce structured CSV files for analysis. The notebook demonstrates scraping the topics, exporting results to per-topic CSV files.
Highlights
- Scrapes GitHub Topics index to collect topic title, URL and description
- For each topic, extracts top 25 repositories with name, owner, stars and URL
- Converts shorthand star counts (e.g. 12k) to integers for analysis
- Exports per-topic CSV files (one CSV per topic) and provides example analysis notebooks
- Uses requests + BeautifulSoup with polite rate-limit handling and optional authenticated API fallback
Technologies
PythonBeautifulSouppandasJupyter NotebookWeb Scraping