fuelprices/crawler
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(CRIT-010): add zone-to-county mapping and county_id to oil_prices

Browse Source 
Add ZONE_COUNTY_MAP for all 5 scraped states (42 zone-to-county entries).
Scraper now resolves county_id at startup and assigns it to each record.
Upsert logic deduplicates by (name, state, county_id) to prevent duplicates
when multiple zones map to the same county. Also adds County model for
DB lookups and fixes Rhode Island zone count (4, not 5).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Edwin Eames
2026-02-09 18:31:46 -05:00
parent 6daa706e5a
commit 8f45f4c209
9 changed files with 746 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
run.py
View File

@@ -6,7 +6,7 @@ import logging
# The 'import models' is crucial for init_db to know about the tables
import models
from database import init_db, SessionLocal
from fuel_scraper import main as run_scraper_main # Assuming your scraper's main is 'main'
from fuel_scraper import main as run_scraper_main # Import from modular package
# Configure basic logging for the run.py script itself if needed
# Your other modules (fuel_scraper, database) will have their own logging