Commit Graph

6 Commits

Author SHA1 Message Date
Alihan
dec49a43f9 Add filesystem health monitoring and compression queue system
- Implement periodic filesystem write permission checks (60-minute intervals)
- Add real-time health status monitoring with SSE endpoints
- Display system health banner when storage issues detected
- Limit compression to 1 concurrent job with queue support
- Add max queue limit of 10 pending jobs
- Show queue positions for pending compression jobs
- Update button text dynamically (Start/Queue Compression)
- Enable write access to footage mount in Docker
- Add comprehensive logging for health checks and compression

Co-Authored-By: Alihan <alihan@example.com>
2025-10-12 22:54:21 +03:00
Alihan
b01fea34aa Refactor codebase: Fix vulnerabilities, improve performance, and eliminate technical debt
## Critical Security Fixes
- Fix path traversal vulnerability with proper sanitization and symlink resolution
- Add CORS configuration via ALLOWED_ORIGINS environment variable
- Validate all user-supplied path components before file operations

## Performance Improvements
- Replace synchronous file.stat() with async aiofiles.os.stat()
- Add TTL-based directory listing cache (60s) for locations/dates/files
- Optimize regex compilation (moved to class level, ~1000x fewer compilations)
- Consolidate duplicate SSE connections into shared useCompressionJobs hook

## Bug Fixes
- Fix race condition in SSE by adding async lock and snapshot method
- Fix memory leak with periodic job pruning (every 5 minutes, max 100 jobs)
- Fix ETA calculation double-counting in pass 1
- Fix video validation to check actual errors, not just stderr presence

## Code Quality
- Replace all print() with proper logging framework (INFO/WARNING/ERROR levels)
- Extract magic numbers to named constants (MAX_STORED_JOBS, WATCHDOG_TIMEOUT, etc)
- Remove dead code (unused CompressionPanel.jsx component)
- Create shared utility modules (formatters.js, useCompressionJobs.js)
- Eliminate duplicate functions (formatFileSize, formatETA across 3 files)

## Impact
- Security: Eliminated path traversal vulnerability
- Stability: Fixed race condition, memory leak, cancellation bugs
- Performance: 2-3x faster directory listings, non-blocking I/O
- Maintainability: Proper logging, DRY principles, configuration constants

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-12 20:06:31 +03:00
Alihan
7cd79216fe Improve compression queue: Add resource limits and security
- Add concurrency limiting with semaphore (max 2 concurrent jobs)
- Add job pruning to prevent unbounded memory growth (max 100 jobs)
- Add file path validation to ensure files within allowed directory
- Fix ffmpeg2pass log cleanup to use source file directory
- Add SSE reconnect handler to re-sync jobs on connection restore

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-12 19:48:46 +03:00
Alihan
752fa4eefd Fix compression job deadlock and add watchdog timer
Resolved critical issue where compression jobs would get stuck at random
progress percentages (e.g., 35.5%) due to pipe buffer deadlock.

**Root Cause:**
- Python code only read ffmpeg's stdout for progress updates
- ffmpeg's stderr pipe buffer (64KB) would fill with output
- ffmpeg blocked writing to stderr, Python blocked reading stdout
- Result: deadlock with job appearing stuck but ffmpeg still using CPU

**Fixes:**
- Read stdout and stderr concurrently using asyncio.gather()
- Prevents pipe buffer deadlock by consuming both streams
- Added watchdog timer to detect genuinely stuck jobs (5 min timeout)
- Improved error logging with stderr capture
- Better error messages showing exact failure reason

**Additional Improvements:**
- Watchdog sets job.error with informative message before killing
- Captures last 50 lines of stderr on failure for debugging
- Enhanced cancellation handling with multiple checkpoints

Tested with previously stuck video file - progress now updates
continuously throughout encoding process.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-12 03:05:14 +03:00
Alihan
334dc85609 Update service ports: frontend to 33768, backend internal-only
- Change frontend port from 9999 to 33768
- Remove backend port exposure for security (only accessible via Docker network)
2025-10-12 02:47:24 +03:00
Alihan
0d71830cfb Initial commit: Drone Footage Manager with Video Compression
- React frontend with video/image browser
- Python FastAPI backend with video compression
- Docker containerized setup
- Video compression with FFmpeg (two-pass encoding)
- Real-time job monitoring with SSE
- Global active jobs monitor
- Clickable header to reset navigation
- Toast notifications for user feedback
2025-10-12 02:22:12 +03:00