fix(cache): resolve scanning stop issue and add cache TTL management

- Fix Redis SCAN cursor type conversion (Buffer to String) to prevent early termination
- Add progress logging in initializeClubCache (every 100 activities with summary)
- Add Redis memory limits (512MB with LRU eviction policy)
- Implement cache TTL: 24h for normal data, 1h for error states (allows retry)
- Fix Docker permission issue by running app container as root
- Add TTL configuration to .env and example.env

Root cause: SCAN cursor comparison failed due to type mismatch (Buffer vs String)
Impact: Scanning now processes all 5000+ IDs instead of stopping at ~300
This commit is contained in:
JamesFlare1212
2026-04-06 21:03:30 -04:00
parent ee8cccc755
commit 32dee6b161
5 changed files with 91 additions and 18 deletions

View File

@@ -18,6 +18,11 @@ STAFF_UPDATE_INTERVAL_MINS=360
CLUB_UPDATE_INTERVAL_MINS=360
LOG_LEVEL=info # Example: 'debug', 'info', 'warn', 'error'
# Cache TTL Configuration (in seconds)
ACTIVITY_CACHE_TTL=86400 # 24 hours for normal activity data
STAFF_CACHE_TTL=86400 # 24 hours for staff data
ERROR_CACHE_TTL=3600 # 1 hour for error states (allows retry)
# Proxy Configuration (Optional)
# Set USE_PROXY=true to enable proxy for Playwright requests
USE_PROXY=false