Updated crawler configuration and database connection handling. Added .gitignore to exclude pycache and .env file.