refactor: replace fuel_scraper with newenglandoil + cheapestoil scrapers

- Add newenglandoil/ package as the primary scraper (replaces fuel_scraper)
- Add cheapestoil/ package as a secondary market price scraper
- Add app.py entry point for direct execution
- Update run.py: new scrape_cheapest(), migrate command, --state filter,
  --refresh-metadata flag for overwriting existing phone/URL data
- Update models.py with latest schema fields
- Update requirements.txt dependencies
- Update Dockerfile and docker-compose.yml for new structure
- Remove deprecated fuel_scraper module, test.py, and log file

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-03-06 11:34:21 -05:00
parent 8f45f4c209
commit 1592e6d685
26 changed files with 3221 additions and 1468 deletions

View File

@@ -1,5 +1,5 @@
# Use an official Python runtime as a parent image
FROM python:3.9-slim-buster
FROM python:3.11-slim-bookworm
# Set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
@@ -24,5 +24,7 @@ RUN pip install --no-cache-dir -r requirements.txt
# Copy the rest of the application code into the container
# This will be overridden by the volume mount in docker-compose for development
COPY . .
#CMD ["python3", "run.py", "initdb"]
CMD ["python3", "run.py", "scrape"]
EXPOSE 9553
CMD ["python3", "run.py", "server"]