Command Reference

EPGOAT Documentation - User Guides

EPGOAT Command Reference

Status: Active Last Updated: 2025-11-02 Related Docs: Quick Start, EPG Generation, Development Setup Code Location: backend/epgoat/

Complete guide to all commands, entry points, and their usage

Table of Contents

Quick Start

Setup Environment

# 1. Copy environment template
cp .env.example .env

# 2. Edit .env with your credentials
#    Required variables:
#    - TPS_M3U_URL: Your IPTV provider's M3U playlist URL
#    - THESPORTSDB_API_KEY: Your API key (use "1" for testing)

# 3. Install dependencies
pip install -r requirements.txt

# 4. Test your setup
python3 backend/epgoat/run_provider.py --provider tps --api-key YOUR_KEY --verbose

Primary Commands

1. Generate EPG with Provider Configuration

Purpose: Main entry point for EPG generation with AUTOMATED utility scheduling.

python3 backend/epgoat/run_provider.py --provider tps --api-key YOUR_KEY

🎯 NEW: Automated Pipeline

This command now automatically runs utilities based on staleness:

  1. Before EPG Generation:
  2. backend/epgoat/utilities/refresh_event_db_v2.py - Runs 3x daily (every 8 hours), fetches -1 to +3 days
  3. refresh_leagues.py - Runs weekly

  4. backfill_event_details.py - Runs daily with 100 event limit

  5. After EPG Generation:

  6. analyze_mismatches.py - Runs every time
  7. backend/epgoat/utilities/clone_m3u.py - Runs every time

Flags: - --provider PROVIDER (required): Provider ID (matches backend/config/providers/PROVIDER.yml) - --api-key KEY: TheSportsDB API key (overrides env var) - --date YYYY-MM-DD: Target date for EPG (default: today) - --tz TIMEZONE: Timezone (default: from provider config) - --max-channels N: Process only first N channels (for testing) - --force-refresh: Force all utilities to run regardless of staleness - --verbose: Enable debug logging - --disable-api: Skip API enrichment (faster but less accurate) - --debug-matching: Show detailed pattern matching diagnostics - --m3u URL/PATH: Override M3U source - --out-xmltv PATH: Override output XML path - --csv PATH: Override audit CSV path

When to use: - Production EPG generation (just run this!) - Scheduled/cron job execution - Provider-specific processing

Staleness Tracking: The system tracks when utilities last ran in dist/.scheduler_state.json. You can check the status:

python3 backend/epgoat/services/scheduler_state.py

Example with all options:

python3 backend/epgoat/run_provider.py \
    --provider tps \
    --api-key 181358 \
    --date 2025-10-30 \
    --tz "America/New_York" \
    --max-channels 100 \
    --verbose \
    --csv dist/audit.csv

2. Direct EPG Generation

Purpose: Generate EPG directly from M3U file without provider config.

python3 backend/epgoat/application/epg_generator.py \
    --m3u https://your-provider.com/playlist.m3u \
    --out-xmltv dist/epg.xml \
    --tz "America/Chicago"

Flags: - --m3u PATH/URL (required): M3U playlist source - --out-xmltv PATH (required): Output XMLTV file - --tz TIMEZONE: Timezone for scheduling - --date YYYY-MM-DD: Target date - --csv PATH: Generate audit CSV - --api-key KEY: TheSportsDB API key - --event-duration-min N: Default event duration in minutes - --max-event-duration-min N: Maximum event duration - --pre-event-block-min N: Pre-event block size (default: 120) - --post-event-block-min N: Post-event block size (default: 120) - --disable-api: Skip API enrichment - --verbose: Debug output

When to use: - Testing new M3U sources - One-off EPG generation - Development/debugging

Utility Commands

3. Refresh Event Database

Purpose: Update the local events cache from TheSportsDB API.

python3 backend/epgoat/utilities/refresh_event_db_v2.py \
    --date 2025-10-30 \
    --days 7 \
    --api-key YOUR_KEY

Flags: - --date YYYY-MM-DD: Starting date (default: today) - --days N: Number of days to fetch (default: 30) - --sports SPORT1,SPORT2: Specific sports to update - --api-key KEY: API key - --db PATH: Database file path - --force: Force refresh even if data exists - --verbose: Detailed output

When to use: - Before running EPG generation (improves accuracy) - Daily maintenance task - After API key renewal

4. Refresh League Information

Purpose: Update league metadata and team information.

python3 backend/epgoat/utilities/refresh_leagues.py --all

Flags: - --all: Refresh all leagues - --league LEAGUE_ID: Specific league to refresh - --api-key KEY: API key - --verbose: Debug output

When to use: - Weekly maintenance - When adding new sports - Team roster updates

5. Backfill Event Details

Purpose: Fill in missing event details for better EPG accuracy.

python3 backend/epgoat/utilities/backfill_event_details.py \
    --limit 100 \
    --sleep 0.1

Flags: - --limit N: Maximum events to process (0 = all) - --sleep SECONDS: Delay between API calls - --db PATH: Database file path - --verbose: Show progress

When to use: - After refresh_event_db - When you notice missing event details - Part of daily maintenance

6. Analyze Mismatches

Purpose: Review channels that failed to match patterns or API data.

python3 backend/epgoat/utilities/analyze_mismatches.py \
    --input dist/tps-audit.csv \
    --output dist/mismatches.xlsx

Flags: - --input PATH: Audit CSV from EPG generation - --output PATH: Excel report output - --threshold N: Minimum confidence threshold - --verbose: Detailed analysis

When to use: - After EPG generation - To improve pattern matching - Quality assurance

7. Clone M3U with Stable IDs

Purpose: Create M3U with consistent tvg-id attributes for IPTV players.

python3 backend/epgoat/utilities/clone_m3u.py \
    --input original.m3u \
    --output clone.m3u \
    --prefix "epgoat"

Flags: - --input PATH: Original M3U file - --output PATH: Output M3U with tvg-ids - --prefix PREFIX: ID prefix (default: "epg") - --preserve-existing: Keep existing tvg-ids

When to use: - Provider M3U lacks tvg-ids - Need stable channel IDs - IPTV player compatibility

8. Verify Channel Patterns

Purpose: Test pattern matching for specific channels.

python3 backend/epgoat/utilities/verify_channels.py \
    --channel "NBA 01: Lakers vs Celtics 7:30 PM ET"

Flags: - --channel TEXT: Channel name to test - --file PATH: Test all channels in M3U - --verbose: Show matching details

When to use: - Debugging pattern issues - Adding new patterns - Quality control

9. Fetch Specific Event Details

Purpose: Get detailed information for a specific event ID.

python3 backend/epgoat/utilities/fetch_event_details.py 2140080

Arguments: - Event ID(s) from TheSportsDB

When to use: - Debugging specific events - Manual verification - API testing

Development Commands

10. Run Tests

# All tests
make test

# Specific module
cd backend/epgoat
pytest tests/test_patterns.py -v

# With coverage
make test-coverage

# Single test
pytest tests/test_patterns.py::test_nba_pattern -v

11. Code Quality

# Format code
make format

# Check formatting
make format-check

# Lint code
make lint

# Type checking
make type-check

# All checks
make ci

12. Validate Configuration

make validate

Purpose: Verify all YAML configs and patterns are valid.

13. Test Provider Config Manager

cd backend/epgoat
python3 test_config_manager.py

Purpose: Test provider configuration loading and YAML cache generation.

What it does: 1. Loads provider config from Supabase database (with force refresh) 2. Saves config to YAML cache at backend/config/providers/{slug}.yml 3. Loads config from YAML cache (tests cache functionality) 4. Validates cache matches database content 5. Displays provider statistics (patterns, TVG-IDs, VOD filters)

Expected output:

TEST 1: Load TPS config from D1 (force refresh)
✓ Successfully loaded TPS config:
  Provider: TPS (tps)
  Patterns: 72
  TVG-ID mappings: 0
  VOD filters: 26

TEST 2: Load TPS config from YAML cache
✓ Successfully loaded from cache
✓ Cache validation passed

TEST 3: Verify YAML cache file
✓ Cache file exists: backend/config/providers/tps.yml
  Size: 24,729 bytes

When to use: - After database migration (verify config loading) - When adding new provider patterns - To regenerate YAML cache after DB changes - For debugging config manager issues

Performance notes: - Database load: ~8 seconds (first time) - Cache load: ~150ms (53x faster) - Cache TTL: 24 hours

Migration & Cleanup

14. System Cleanup

Purpose: Clean up duplicates, check for issues, set up environment.

python3 engineering/scripts/cleanup_migration.py

What it does: - Removes duplicate/backup files - Checks for hardcoded credentials - Creates .env.example - Updates .gitignore - Validates directory structure

14. Clean Build Artifacts

make clean

Removes: - __pycache__ directories - .pyc files - Test coverage reports - Build artifacts

Cron Job Setup

Local Cron (Linux/Mac)

# Edit crontab
crontab -e

# Add hourly EPG generation (adjust path to your repo)
# The script automatically handles utility scheduling - just run it!
0 * * * * cd /path/to/epgoat-internal && source .env && /usr/bin/python3 backend/epgoat/run_provider.py --provider tps --api-key $THESPORTSDB_API_KEY >> logs/epg.log 2>&1

# Or run every 4 hours if hourly is too frequent
0 */4 * * * cd /path/to/epgoat-internal && source .env && /usr/bin/python3 backend/epgoat/run_provider.py --provider tps --api-key $THESPORTSDB_API_KEY >> logs/epg.log 2>&1

Note: The automated scheduler ensures: - Events refresh runs max 3x daily (even if EPG runs hourly) - Leagues refresh runs max 1x weekly - Backfill runs max 1x daily - No wasted API calls!

GitHub Actions

Create .github/workflows/epg-generation.yml:

name: Generate EPG
on:
  schedule:
    - cron: '0 * * * *'  # Every hour
  workflow_dispatch:  # Manual trigger

jobs:
  generate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - uses: actions/setup-python@v4
        with:
          python-version: '3.11'

      - name: Install dependencies
        run: pip install -r requirements.txt

      - name: Generate EPG
        env:
          TPS_M3U_URL: ${{ secrets.TPS_M3U_URL }}
          THESPORTSDB_API_KEY: ${{ secrets.THESPORTSDB_API_KEY }}
        run: |
          python3 backend/epgoat/run_provider.py \
            --provider tps \
            --api-key $THESPORTSDB_API_KEY

      - name: Upload EPG
        uses: actions/upload-artifact@v3
        with:
          name: epg-files
          path: dist/*.xml

Environment Variables

Required

  • TPS_M3U_URL: Full M3U playlist URL with credentials
  • THESPORTSDB_API_KEY: API key (minimum: "1" for testing)

Optional

  • EPG_TIMEZONE: Default timezone (e.g., "America/New_York")
  • EPG_OUTPUT_DIR: Output directory (default: "dist")
  • DEBUG: Enable debug logging ("true"/"false")
  • CACHE_TTL: Cache duration in seconds
  • API_RATE_LIMIT: Max requests per second

Common Workflows

Daily EPG Generation

# 1. Refresh event database
python3 backend/epgoat/utilities/refresh_event_db_v2.py --days 7

# 2. Generate EPG
python3 backend/epgoat/run_provider.py --provider tps --api-key YOUR_KEY

# 3. Analyze any issues
python3 backend/epgoat/utilities/analyze_mismatches.py \
    --input dist/tps-audit.csv

Add New Sport/League

# 1. Edit configuration files
vim backend/config/sport_emojis.yml      # Add emoji
vim backend/config/sport_categories.yml  # Add category

# 2. Add pattern
vim backend/epgoat/domain/patterns.py      # Add to ALLOWED_CHANNEL_PATTERNS

# 3. Test pattern
python3 backend/epgoat/utilities/verify_channels.py \
    --channel "NEW_LEAGUE 01: Team A vs Team B"

# 4. Run tests
pytest backend/epgoat/tests/test_patterns.py -v

# 5. Generate EPG to verify
python3 backend/epgoat/run_provider.py --provider tps --max-channels 10

Debug Channel Matching

# Enable debug matching
python3 backend/epgoat/run_provider.py \
    --provider tps \
    --debug-matching \
    --max-channels 50 \
    --verbose > debug.log 2>&1

# Check audit CSV
grep "0%" dist/tps-audit.csv  # Find unmatched channels

Troubleshooting

Problem: "Module not found" errors

# Ensure you're in the repo root
cd /path/to/epgoat-internal

# Add to PYTHONPATH
export PYTHONPATH="${PYTHONPATH}:$(pwd)"

Problem: API key not working

# Test with free key
export THESPORTSDB_API_KEY="1"

# Verify key works
python3 backend/epgoat/utilities/fetch_event_details.py 652890

Problem: Wrong timezone in EPG

# List available timezones
python3 -c "from zoneinfo import available_timezones; print(sorted(available_timezones())[:20])"

# Common US timezones:
# America/New_York (Eastern)
# America/Chicago (Central)
# America/Denver (Mountain)
# America/Los_Angeles (Pacific)

Problem: Channels not matching patterns

# Test specific channel
python3 backend/epgoat/utilities/verify_channels.py \
    --channel "Your Channel Name Here" \
    --verbose

# Review audit CSV
cat dist/tps-audit.csv | column -t -s,

Performance Tips

  1. Use --max-channels for testing - Process subset first
  2. Disable API for speed - Add --disable-api flag
  3. Cache API responses - Reuse event database
  4. Run backfill separately - Don't block EPG generation
  5. Use verbose sparingly - Lots of output slows processing

Support

For issues or questions: 1. Check this command reference 2. Review audit CSV for errors 3. Enable verbose mode for debugging 4. Check GitHub issues