refactor: replace fuel_scraper with newenglandoil + cheapestoil scrapers

- Add newenglandoil/ package as the primary scraper (replaces fuel_scraper)
- Add cheapestoil/ package as a secondary market price scraper
- Add app.py entry point for direct execution
- Update run.py: new scrape_cheapest(), migrate command, --state filter,
  --refresh-metadata flag for overwriting existing phone/URL data
- Update models.py with latest schema fields
- Update requirements.txt dependencies
- Update Dockerfile and docker-compose.yml for new structure
- Remove deprecated fuel_scraper module, test.py, and log file

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-03-06 11:34:21 -05:00
parent 8f45f4c209
commit 1592e6d685
26 changed files with 3221 additions and 1468 deletions

View File

@@ -1,22 +1,11 @@
services:
app:
build: . # Build the image from the Dockerfile in the current directory
build: .
container_name: fuel_scraper_app_service
ports:
- "9553:9553"
volumes:
# Mount current directory for live code updates during development
- .:/app
# If your app needs to connect to a DB on the host, and host.docker.internal
# isn't working, you might need to add it to the host network (less secure, platform-dependent)
# or use 'extra_hosts' on Linux.
# For host.docker.internal to work on Linux, you might need:
extra_hosts:
- "host.docker.internal:host-gateway"
#environment:
# You can pass DATABASE_URL here to override database.py if needed
# DATABASE_URL: "postgresql://your_user:your_password@host.docker.internal:5432/fuelprices"
# PYTHONUNBUFFERED: 1 # Already in Dockerfile, but good practice
# The default command comes from the Dockerfile's CMD
# To keep the container running after the script finishes (for debugging or exec):
# tty: true
# stdin_open: true