Google Maps Scraper Architecture
Searches German cities for businesses via VPN - Entry point of the lead generation pipeline
Internal Components
Service Configuration
Replicas20
VPNGluetun (Frankfurt)
Network Mode
service:vpnDatabase Outputs
resultsGoogle Maps business data stored as JSONB with place_id as unique key
Pipeline Position
Upstream
None (Entry point)
Downstream
Result Processor (polls every 30s)
Component Breakdown
Search Term Producer
Generates search queries by combining business categories with German cities from the shared cities list
City Queue Manager
Distributes scraping workload across replicas, tracks progress per city to avoid duplicates
VPN Health Monitor
Waits up to 40 attempts (~2 min) for Mullvad VPN DNS availability before scraping
Results Writer
Writes scraped data to PostgreSQL results table with deduplication on place_id