feat(UI): initial working frontend UI

This commit is contained in:
2025-10-09 17:12:55 +13:00
parent e8972cae38
commit 9e61d18bf6
26 changed files with 6885 additions and 3 deletions
+35
View File
@@ -0,0 +1,35 @@
__pycache__
*.pyc
*.pyo
*.pyd
.Python
env
pip-log.txt
pip-delete-this-directory.txt
.tox
.coverage
.coverage.*
.pytest_cache
nosetests.xml
coverage.xml
*.cover
*.log
.git
.mypy_cache
.pytest_cache
.hypothesis
*.egg-info/
dist/
build/
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
.DS_Store
Thumbs.db
*.swp
*.swo
*~
+17
View File
@@ -0,0 +1,17 @@
# BDFR Web Interface Configuration
# Copy this file to .env and update the values as needed
# Reddit OAuth Configuration
# You MUST set this to match your Reddit OAuth app settings
# Go to https://www.reddit.com/prefs/apps, create/edit your app, and use the exact redirect URI
BDFR_REDIRECT_URI=http://localhost:8000/auth/callback
# OAuth Credentials (from your Reddit OAuth app)
# Get these from: https://www.reddit.com/prefs/apps
BDFR_CLIENT_ID=your_client_id_here
BDFR_CLIENT_SECRET=your_client_secret_here
# Server Configuration (optional)
# HOST=0.0.0.0
# PORT=8000
# DEBUG=true
+35
View File
@@ -0,0 +1,35 @@
FROM python:3.11-slim
# Set working directory
WORKDIR /app
# Install system dependencies
RUN apt-get update && apt-get install -y \
gcc \
&& rm -rf /var/lib/apt/lists/*
# Copy requirements first for better caching
COPY requirements.txt .
# Install Python dependencies
RUN pip install --no-cache-dir -r requirements.txt
# Copy application code
COPY app/ ./app/
COPY templates/ ./templates/
COPY static/ ./static/
# Create non-root user
RUN useradd --create-home --shell /bin/bash app \
&& chown -R app:app /app
USER app
# Expose port
EXPOSE 8000
# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD curl -f http://localhost:8000/health || exit 1
# Run the application
CMD ["python", "/app/app/main.py"]
+202
View File
@@ -0,0 +1,202 @@
# BDFR Web Interface
A modern web interface for the Bulk Downloader for Reddit (BDFR) built with FastAPI, WebSockets, and vanilla JavaScript.
## Features
- **Modern UI**: Clean, responsive design with gradient backgrounds and smooth animations
- **Real-time Progress**: WebSocket-based progress updates for active downloads
- **Subreddit Downloads**: Download posts from any subreddit with customizable limits and sorting
- **User Downloads**: Download posts from specific users
- **Status Monitoring**: Real-time system status and connection monitoring
- **Form Validation**: Client-side validation with visual feedback
- **Error Handling**: Comprehensive error handling with user-friendly notifications
## Project Structure
```
web_interface/
├── app/
│ └── main.py # FastAPI application
├── static/
│ ├── css/
│ │ └── style.css # Modern CSS styling
│ └── js/
│ └── app.js # WebSocket client and form handling
├── templates/
│ └── index.html # Main web interface
└── requirements.txt # Python dependencies
```
## Installation
1. **Install Dependencies**:
```bash
cd web_interface
pip install -r requirements.txt
```
2. **Run the Application**:
```bash
cd app
python main.py
```
3. **Access the Interface**:
Open your browser and navigate to `http://localhost:8000`
## API Endpoints
### Download Endpoints
- `POST /api/download/subreddit` - Start subreddit download
- `POST /api/download/user` - Start user download
- `GET /api/downloads` - List all active downloads
- `GET /api/downloads/{download_id}` - Get specific download status
- `DELETE /api/downloads/{download_id}` - Cancel download
### WebSocket
- `ws://localhost:8000/ws/progress` - Real-time progress updates
### Status Endpoints
- `GET /` - Main web interface
- `GET /health` - Health check
- `GET /api/bdfr/status` - BDFR system status
## Configuration
The application uses the following default settings:
- **Host**: `0.0.0.0`
- **Port**: `8000`
- **WebSocket Path**: `/ws/progress`
- **Static Files**: Served from `/static`
## Docker Support
To run with Docker:
```bash
# Build the image
docker build -t bdfr-web-interface .
# Run the container
docker run -p 8000:8000 bdfr-web-interface
```
## Development
### Adding New Features
1. **Backend Changes**: Modify `app/main.py` to add new endpoints
2. **Frontend Changes**: Update `templates/index.html` for UI changes
3. **Styling**: Modify `static/css/style.css` for visual changes
4. **JavaScript**: Update `static/js/app.js` for client-side functionality
### WebSocket Integration
The WebSocket connection automatically handles:
- Connection establishment and reconnection
- Progress updates from the server
- Error handling and user notifications
- Real-time UI updates
### Form Handling
Both download forms include:
- Input validation
- Loading states
- Success/error notifications
- Automatic form reset on success
## Integration with BDFR
### Direct BDFR API Integration
This interface uses the direct BDFR API integration, eliminating the need for subprocess console parsing:
- `/api/download/subreddit` - Downloads from subreddits using `BDFRManager.download_subreddit()`
- `/api/download/user` - Downloads from users using `BDFRManager.download_user()`
- `/api/bdfr/status` - Returns BDFR system status and capabilities
- `/ws/progress` - Provides real-time progress updates via WebSocket
The web interface imports `BDFRManager` directly from `bdfr.api` and uses structured progress callbacks for seamless integration.
### Migration Notes
**Previous Approach (Subprocess-based)**:
- Used `subprocess.Popen` to start BDFR CLI
- Parsed console output with regex for progress updates
- Required `BDFRRunner` class for process management
- Used `threading.Thread` and `queue.Queue` for coordination
**Current Approach (Direct API)**:
- Direct integration with `BDFRManager` from `bdfr.api`
- Structured `ProgressEvent` callbacks instead of console parsing
- Thread-safe progress tracking with `ProgressCallback` interface
- No subprocess overhead or console output parsing required
The migration provides better error handling, structured progress events, and eliminates console parsing complexity.
## Browser Support
- Modern browsers with WebSocket support
- Chrome 60+
- Firefox 55+
- Safari 11+
- Edge 79+
## Security Considerations
- CORS is enabled for all origins (configure for production)
- Input validation on both client and server
- No authentication implemented (add as needed)
- WebSocket connections are not secured (use WSS in production)
## Production Deployment
For production deployment:
1. Configure CORS for specific origins
2. Add authentication/authorization
3. Use HTTPS/WSS for secure connections
4. Configure proper logging
5. Set up reverse proxy (nginx recommended)
6. Add rate limiting
7. Configure environment variables
## Troubleshooting
### Common Issues
1. **WebSocket Connection Failed**:
- Check if the server is running
- Verify firewall settings
- Check browser console for errors
2. **Downloads Not Starting**:
- Verify BDFR integration is configured
- Check server logs for errors
- Ensure form data is valid
3. **Static Files Not Loading**:
- Verify static file paths
- Check file permissions
- Ensure proper MIME types
### Debug Mode
Run with debug logging:
```bash
python main.py --log-level debug
```
## Contributing
1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test thoroughly
5. Submit a pull request
## License
This project is part of the BDFR ecosystem. See the main project license for details.
+177
View File
@@ -0,0 +1,177 @@
# BDFR Web Interface Startup Scripts
This directory contains simple startup scripts to easily run the BDFR web interface application.
## Available Scripts
### 🚀 Quick Start
Choose the appropriate script for your operating system:
- **`start.py`** - Cross-platform Python script (recommended)
- **`start.sh`** - Unix/Linux/macOS shell script
- **`start.bat`** - Windows batch script
## Usage
### Option 1: Python Script (Cross-platform)
```bash
# Navigate to the web_interface directory
cd web_interface
# Run the startup script
python start.py
```
### Option 2: Shell Script (Unix/Linux/macOS)
```bash
# Navigate to the web_interface directory
cd web_interface
# Make sure the script is executable
chmod +x start.sh
# Run the startup script
./start.sh
```
### Option 3: Batch Script (Windows)
```cmd
REM Navigate to the web_interface directory
cd web_interface
REM Run the startup script
start.bat
```
## What the Scripts Do
1. **Check Python version** - Ensures Python 3.8+ is installed
2. **Install dependencies** - Automatically installs required packages from `requirements.txt`
3. **Verify BDFR module** - Checks if the BDFR module is available
4. **Start the server** - Launches the FastAPI application with uvicorn
## Server Information
Once started, the web interface will be available at:
- **Main interface**: http://localhost:8000
- **API documentation**: http://localhost:8000/docs
- **Health check**: http://localhost:8000/health
## Features
- ✅ Automatic dependency management
- ✅ Cross-platform compatibility
- ✅ Colored output for better user experience
- ✅ Error handling and informative messages
- ✅ Graceful server shutdown
- ✅ BDFR module availability checking
## Requirements
- Python 3.8 or higher
- Internet connection (for installing dependencies)
- BDFR module in Python path (parent directory should contain the BDFR package)
- Reddit OAuth application (for authentication features)
## Reddit OAuth Setup
To use the authentication features, you need to:
1. **Create a Reddit OAuth Application**:
- Go to [Reddit App Preferences](https://www.reddit.com/prefs/apps)
- Click "Create App" or "Create Another App"
- Choose "web app" as the application type
- Set a name (e.g., "BDFR Web Interface")
- Set redirect URI to: `http://localhost:8000/auth/callback`
2. **Configure the Redirect URI** (if using a different port or domain):
- Run the OAuth setup helper: `python setup_oauth.py`
- Or manually copy `.env.example` to `.env`
- Update the following in the `.env` file:
- `BDFR_REDIRECT_URI` - Your OAuth redirect URI
- `BDFR_CLIENT_ID` - Your OAuth client ID (from Reddit app)
- `BDFR_CLIENT_SECRET` - Your OAuth client secret (from Reddit app)
- Make sure the redirect URI matches exactly what you set in your Reddit OAuth app
3. **Update BDFR Configuration**:
- The web interface uses the same OAuth credentials as BDFR
- You need to either update your existing Reddit OAuth app or create a new one
## Option A: Update Existing Reddit OAuth App
If you want to use the same OAuth app for both BDFR CLI and web interface:
1. Go to [Reddit App Preferences](https://www.reddit.com/prefs/apps)
2. Find your existing app (the one with client ID `U-6gk4ZCh3IeNQ`)
3. Click "edit" and add your redirect URI to the "redirect uris" field:
- `http://localhost:8000/auth/callback`
4. Save the changes
## Option B: Create a New Reddit OAuth App (Recommended)
For better separation between CLI and web interface:
1. Go to [Reddit App Preferences](https://www.reddit.com/prefs/apps)
2. Click "Create App" or "Create Another App"
3. Fill in the details:
- **Name**: `BDFR Web Interface` (or your preferred name)
- **App type**: `web app`
- **Description**: `Web interface for BDFR (Bulk Downloader for Reddit)`
- **About URL**: (optional)
- **Redirect URI**: `http://localhost:8000/auth/callback`
4. Click "Create app"
5. Copy the client ID and client secret
6. Update `bdfr/default_config.cfg` with the new credentials:
```
client_id = YOUR_NEW_CLIENT_ID
client_secret = YOUR_NEW_CLIENT_SECRET
```
## Troubleshooting
### "BDFR module not found"
Make sure you're running the script from the correct directory, or ensure the parent directory containing the BDFR package is in your Python path.
### "Python 3.8+ required"
Install Python 3.8 or higher from the official Python website.
### "Permission denied" (Unix/Linux/macOS)
Make sure the shell script has execute permissions:
```bash
chmod +x start.sh
```
### "invalid redirect_uri parameter" (OAuth Error)
This error occurs when the redirect URI doesn't match what you configured in your Reddit OAuth app:
1. **Verify your Reddit OAuth app settings**:
- Go to [Reddit App Preferences](https://www.reddit.com/prefs/apps)
- Find your app and check the redirect URI
- Make sure it exactly matches what you're using
2. **Update the redirect URI**:
- Copy `.env.example` to `.env`
- Set `BDFR_REDIRECT_URI` to match your Reddit OAuth app
- Example: `BDFR_REDIRECT_URI=http://localhost:8000/auth/callback`
3. **Common redirect URI formats**:
- Local development: `http://localhost:8000/auth/callback`
- With custom port: `http://localhost:3000/auth/callback`
- Production: `https://yourdomain.com/auth/callback`
4. **Recreate your OAuth app if needed**:
- Delete the existing app in Reddit
- Create a new one with the correct redirect URI
## Manual Alternative
If you prefer to run the server manually:
```bash
cd web_interface
pip install -r requirements.txt
python -m uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
+311
View File
@@ -0,0 +1,311 @@
"""
OAuth2 Authentication module for BDFR Web Interface
This module handles OAuth2 authentication flow for the web interface,
integrating with BDFR's existing OAuth2 system.
"""
import asyncio
import json
import logging
import secrets
import time
from datetime import datetime, timedelta
from typing import Dict, Optional, Any
from urllib.parse import urlencode
import httpx
from fastapi import HTTPException, status
# Try to import BDFR modules, but handle gracefully if not available
try:
from bdfr.oauth2 import OAuth2Authenticator, OAuth2TokenManager
from bdfr.exceptions import RedditAuthenticationError
BDFR_AVAILABLE = True
except ImportError:
BDFR_AVAILABLE = False
# Create mock classes for when BDFR is not available
class OAuth2Authenticator:
pass
class OAuth2TokenManager:
pass
class RedditAuthenticationError(Exception):
pass
logger = logging.getLogger(__name__)
class WebOAuth2Manager:
"""OAuth2 manager for web interface authentication"""
def __init__(self, client_id: str, client_secret: str, scopes: list = None):
self.client_id = client_id
self.client_secret = client_secret
self.scopes = scopes or ["identity", "history", "read", "save", "mysubreddits"]
# In-memory storage for OAuth2 states and tokens
# In production, this should be replaced with a proper database
self.oauth_states = {}
self.refresh_tokens = {}
self.access_tokens = {}
# Store Reddit usernames per session state
self.usernames = {}
# Reddit OAuth2 endpoints
self.reddit_auth_url = "https://www.reddit.com/api/v1/authorize"
self.reddit_token_url = "https://www.reddit.com/api/v1/access_token"
self.reddit_user_info_url = "https://oauth.reddit.com/api/v1/me"
# Token expiration tracking
self.token_expiry = {}
def generate_state(self) -> str:
"""Generate a secure random state for OAuth2"""
state = secrets.token_urlsafe(32)
self.oauth_states[state] = {
"created_at": time.time(),
"used": False
}
return state
def validate_state(self, state: str) -> bool:
"""Validate OAuth2 state parameter"""
if state not in self.oauth_states:
return False
state_data = self.oauth_states[state]
if state_data["used"]:
return False
# States expire after 10 minutes
if time.time() - state_data["created_at"] > 600:
del self.oauth_states[state]
return False
return True
def mark_state_used(self, state: str):
"""Mark OAuth2 state as used"""
if state in self.oauth_states:
self.oauth_states[state]["used"] = True
def get_authorization_url(self, redirect_uri: str) -> Dict[str, str]:
"""Generate OAuth2 authorization URL"""
state = self.generate_state()
params = {
"client_id": self.client_id,
"response_type": "code",
"state": state,
"redirect_uri": redirect_uri,
"scope": " ".join(self.scopes),
"duration": "permanent"
}
auth_url = f"{self.reddit_auth_url}?{urlencode(params)}"
return {
"authorization_url": auth_url,
"state": state
}
async def exchange_code_for_token(self, code: str, state: str, redirect_uri: str) -> Dict[str, Any]:
"""Exchange authorization code for access token"""
if not self.validate_state(state):
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Invalid or expired state parameter"
)
self.mark_state_used(state)
data = {
"grant_type": "authorization_code",
"code": code,
"redirect_uri": redirect_uri
}
headers = {
"User-Agent": "BDFR-Web-Interface/1.0"
}
# Use HTTP Basic Auth for client credentials
auth = (self.client_id, self.client_secret)
async with httpx.AsyncClient() as client:
try:
response = await client.post(
self.reddit_token_url,
data=data,
auth=auth,
headers=headers,
timeout=30.0
)
if response.status_code != 200:
error_detail = response.json() if response.headers.get("content-type", "").startswith("application/json") else response.text
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Token exchange failed: {error_detail}"
)
token_data = response.json()
# Store tokens
access_token = token_data["access_token"]
refresh_token = token_data.get("refresh_token")
if refresh_token:
self.refresh_tokens[state] = refresh_token
self.access_tokens[state] = access_token
# Set expiry (Reddit tokens typically last 1 hour)
self.token_expiry[state] = time.time() + token_data.get("expires_in", 3600)
# Attempt to fetch and store the Reddit username for this session
username = None
try:
user_info = await self.get_user_info(access_token)
username = user_info.get("name")
except Exception as e:
logger.warning(f"Failed to fetch user info during token exchange: {e}")
if username:
self.usernames[state] = username
return {
"access_token": access_token,
"refresh_token": refresh_token,
"expires_in": token_data.get("expires_in", 3600),
"token_type": token_data.get("token_type", "bearer"),
"state": state,
"username": username
}
else:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="No refresh token received"
)
except httpx.TimeoutException:
raise HTTPException(
status_code=status.HTTP_408_REQUEST_TIMEOUT,
detail="Token exchange timed out"
)
except Exception as e:
logger.error(f"Token exchange error: {str(e)}")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Internal server error during token exchange"
)
async def get_user_info(self, access_token: str) -> Dict[str, Any]:
"""Get user information using access token"""
headers = {
"Authorization": f"Bearer {access_token}",
"User-Agent": "BDFR-Web-Interface/1.0"
}
async with httpx.AsyncClient() as client:
try:
response = await client.get(
self.reddit_user_info_url,
headers=headers,
timeout=30.0
)
if response.status_code != 200:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid access token"
)
return response.json()
except httpx.TimeoutException:
raise HTTPException(
status_code=status.HTTP_408_REQUEST_TIMEOUT,
detail="User info request timed out"
)
except Exception as e:
logger.error(f"User info error: {str(e)}")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Error retrieving user information"
)
def is_token_expired(self, state: str) -> bool:
"""Check if access token is expired"""
if state not in self.token_expiry:
return True
return time.time() > self.token_expiry[state]
def get_valid_token(self, state: str) -> Optional[str]:
"""Get valid access token, refreshing if necessary"""
if state not in self.access_tokens:
return None
if self.is_token_expired(state):
# Token expired, would need refresh logic here
# For now, just return None to indicate re-auth needed
return None
return self.access_tokens[state]
def revoke_session(self, state: str):
"""Revoke OAuth2 session"""
if state in self.oauth_states:
del self.oauth_states[state]
if state in self.refresh_tokens:
del self.refresh_tokens[state]
if state in self.access_tokens:
del self.access_tokens[state]
if state in self.token_expiry:
del self.token_expiry[state]
if state in self.usernames:
del self.usernames[state]
def get_auth_status(self, state: str = None) -> Dict[str, Any]:
"""Get authentication status"""
if not state:
return {
"authenticated": False,
"message": "No active session"
}
if state not in self.access_tokens:
return {
"authenticated": False,
"message": "No tokens found for session"
}
access_token = self.get_valid_token(state)
if not access_token:
return {
"authenticated": False,
"message": "Token expired or invalid"
}
return {
"authenticated": True,
"expires_at": self.token_expiry.get(state, 0),
"scopes": self.scopes,
"username": self.usernames.get(state)
}
# Global OAuth2 manager instance
oauth_manager = None
def init_oauth_manager(client_id: str, client_secret: str, scopes: list = None):
"""Initialize the global OAuth2 manager"""
global oauth_manager
oauth_manager = WebOAuth2Manager(client_id, client_secret, scopes)
def get_oauth_manager() -> WebOAuth2Manager:
"""Get the global OAuth2 manager instance"""
if oauth_manager is None:
raise RuntimeError("OAuth2 manager not initialized")
return oauth_manager
+960
View File
@@ -0,0 +1,960 @@
from fastapi import FastAPI, WebSocket, WebSocketDisconnect, HTTPException, Form, File, UploadFile, Query, status
from fastapi.responses import HTMLResponse, FileResponse, RedirectResponse
from fastapi.staticfiles import StaticFiles
from fastapi.templating import Jinja2Templates
from fastapi.middleware.cors import CORSMiddleware
from fastapi import Request
import json
import asyncio
import os
import configparser
import logging
# Load environment variables from .env file if available
try:
from dotenv import load_dotenv
load_dotenv()
except ImportError:
# python-dotenv not installed, use os.environ directly
pass
from typing import List, Dict, Any, Optional
from datetime import datetime
from urllib.parse import urlencode
# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
# Import authentication module
from .auth import init_oauth_manager, get_oauth_manager
# Import BDFR API layer
import sys
import os
# Add the parent directory (BDFR root) to Python path
bdfr_root = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))), 'bdfr')
sys.path.insert(0, bdfr_root)
# Also add the current bdfr directory to path
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))), ''))
try:
from bdfr.api import BDFRManager, ProgressEvent, ProgressCallback, get_bdfr_manager
BDFR_AVAILABLE = True
except ImportError as e:
logger.warning(f"Failed to import BDFR API: {e}. BDFR modules may not be available.")
BDFR_AVAILABLE = False
# Create a mock BDFRManager for when BDFR is not available
class MockBDFRManager:
def __init__(self, *args, **kwargs):
pass
def download_subreddit(self, *args, **kwargs):
raise NotImplementedError("BDFR not available")
def download_user(self, *args, **kwargs):
raise NotImplementedError("BDFR not available")
def get_download_status(self, *args, **kwargs):
return None
def cancel_download(self, *args, **kwargs):
return False
class MockProgressCallback:
pass
class MockProgressEvent:
def __init__(self, *args, **kwargs):
pass
BDFRManager = MockBDFRManager
ProgressEvent = MockProgressEvent
ProgressCallback = MockProgressCallback
def get_bdfr_manager(*args, **kwargs):
return MockBDFRManager()
app = FastAPI(title="BDFR Web Interface", version="1.0.0")
# Initialize OAuth2 manager
def init_oauth():
"""Initialize OAuth2 manager with credentials from environment or BDFR config"""
try:
# Try to get credentials from environment variables first
client_id = os.getenv("BDFR_CLIENT_ID")
client_secret = os.getenv("BDFR_CLIENT_SECRET")
if client_id and client_secret:
# Use environment credentials
logger.info("Using OAuth credentials from environment variables")
scopes = ["identity", "history", "read", "save", "mysubreddits"]
else:
# Fall back to BDFR config file
logger.info("Using OAuth credentials from BDFR config file")
config = configparser.ConfigParser()
config.read("../bdfr/default_config.cfg")
client_id = config.get("DEFAULT", "client_id")
client_secret = config.get("DEFAULT", "client_secret")
scopes_str = config.get("DEFAULT", "scopes", fallback="identity,read")
# Parse scopes
scopes = [scope.strip() for scope in scopes_str.split(",")]
init_oauth_manager(client_id, client_secret, scopes)
logger.info("OAuth2 manager initialized successfully")
except Exception as e:
logger.error(f"Failed to initialize OAuth2 manager: {e}")
# Use default credentials if config fails
init_oauth_manager("U-6gk4ZCh3IeNQ", "7CZHY6AmKweZME5s50SfDGylaPg")
# Initialize OAuth2 on startup
# Use a configurable redirect URI - this should match your Reddit OAuth app settings
redirect_uri = os.getenv("BDFR_REDIRECT_URI", "http://localhost:8000/auth/callback")
init_oauth()
# Enable CORS
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# Mount static files
import os
current_dir = os.path.dirname(os.path.abspath(__file__))
static_dir = os.path.join(current_dir, "..", "static")
template_dir = os.path.join(current_dir, "..", "templates")
# Ensure directories exist
os.makedirs(static_dir, exist_ok=True)
os.makedirs(template_dir, exist_ok=True)
app.mount("/static", StaticFiles(directory=static_dir), name="static")
templates = Jinja2Templates(directory=template_dir)
# WebSocket connection manager
class ConnectionManager:
def __init__(self):
self.active_connections: List[WebSocket] = []
async def connect(self, websocket: WebSocket):
logger.info(f"[WEBSOCKET] WebSocket connection attempt from {websocket.client}")
await websocket.accept()
self.active_connections.append(websocket)
logger.info(f"[WEBSOCKET] New connection established from {websocket.client}. Total connections: {len(self.active_connections)}")
def disconnect(self, websocket: WebSocket):
if websocket in self.active_connections:
self.active_connections.remove(websocket)
logger.info(f"[WEBSOCKET] Connection disconnected from {websocket.client}. Total connections: {len(self.active_connections)}")
async def send_personal_message(self, message: str, websocket: WebSocket):
try:
await websocket.send_text(message)
except Exception as e:
logger.warning(f"Failed to send personal message: {e}")
self.disconnect(websocket)
async def broadcast(self, message: str):
"""Broadcast message to all active connections - simplified version"""
logger.debug(f"[WEBSOCKET] Broadcasting message to {len(self.active_connections)} connections")
if not self.active_connections:
logger.debug("[WEBSOCKET] No active connections to broadcast to!")
return
# Simple approach: try to send to all connections, remove failed ones immediately
alive_connections = []
for connection in self.active_connections:
try:
await connection.send_text(message)
alive_connections.append(connection)
logger.debug(f"[WEBSOCKET] Message sent successfully to connection {id(connection)}")
except Exception as e:
logger.warning(f"[WEBSOCKET] Removing failed connection {id(connection)}: {e}")
# Connection failed, don't add it to alive_connections
# Update active connections to only include successful ones
self.active_connections = alive_connections
logger.debug(f"[WEBSOCKET] Broadcast complete. Active connections: {len(self.active_connections)}")
async def is_connection_alive(self, websocket: WebSocket) -> bool:
"""Check if a WebSocket connection is still alive"""
try:
# Try to send a ping frame (this is a WebSocket protocol ping)
await websocket.ping()
return True
except Exception:
return False
async def cleanup_dead_connections(self):
"""Simplified cleanup - just log the current state"""
logger.info(f"[WEBSOCKET] Cleanup check: {len(self.active_connections)} active connections")
manager = ConnectionManager()
# Initialize BDFR Manager
bdfr_manager = get_bdfr_manager("./downloads")
# Store auth token for BDFR manager if available
_bdfr_auth_token = None
# Active downloads tracking (now managed by BDFRManager, but kept for WebSocket compatibility)
active_downloads = {}
# WebSocket-compatible progress callback
class WebSocketProgressCallback(ProgressCallback if BDFR_AVAILABLE else MockProgressCallback):
"""Progress callback that sends updates to WebSocket clients"""
def __init__(self, download_id: str, connection_manager: ConnectionManager):
self.download_id = download_id
self.connection_manager = connection_manager
async def on_progress(self, event: ProgressEvent):
"""Send progress update to WebSocket clients"""
logger.info(f"[WEBSOCKET-PROGRESS] Received progress event for download {event.download_id}: {event.message} ({event.progress}%)")
logger.info(f"[WEBSOCKET-PROGRESS] Active connections before broadcast: {len(self.connection_manager.active_connections)}")
try:
# Convert BDFR API event to web interface format
progress_data = {
"type": "progress",
"id": event.download_id,
"download_id": event.download_id,
"status": "running",
"message": event.message,
"progress": event.progress or 0,
"data": event.data,
"timestamp": event.timestamp.isoformat()
}
# Update active_downloads for WebSocket compatibility
# Need to find the web interface download ID that corresponds to this BDFR download ID
web_download_id = None
for download_id, download_info in active_downloads.items():
if download_info.get("bdfr_download_id") == event.download_id:
web_download_id = download_id
break
if web_download_id:
# Keep server-side state in sync
active_downloads[web_download_id]["status"] = "running"
active_downloads[web_download_id]["progress"] = event.progress or 0
if "items_processed" in event.data:
active_downloads[web_download_id]["items_processed"] = event.data["items_processed"]
if "items_found" in event.data:
active_downloads[web_download_id]["items_found"] = event.data["items_found"]
if "current_item" in event.data:
active_downloads[web_download_id]["current_item"] = event.data["current_item"]
if "phase" in event.data:
active_downloads[web_download_id]["phase"] = event.data["phase"]
# Unify IDs for frontend to prevent duplicate cards
progress_data["id"] = web_download_id
progress_data["download_id"] = web_download_id
progress_data["bdfr_download_id"] = event.download_id
progress_data["web_download_id"] = web_download_id
progress_data["subreddit"] = active_downloads[web_download_id].get("subreddit")
progress_data["username"] = active_downloads[web_download_id].get("username")
progress_data["limit"] = active_downloads[web_download_id].get("limit")
else:
# No mapping yet; include bdfr id for debugging
progress_data["bdfr_download_id"] = event.download_id
progress_data["limit"] = (event.data or {}).get("limit")
logger.info(f"[WEBSOCKET-PROGRESS] Broadcasting progress data: {progress_data}")
logger.info(f"[WEBSOCKET-PROGRESS] About to broadcast progress message for download {event.download_id}")
await self.connection_manager.broadcast(json.dumps(progress_data))
logger.info(f"[WEBSOCKET-PROGRESS] Progress message broadcast completed for download {event.download_id}")
except Exception as e:
logger.warning(f"Failed to send progress update for {event.download_id}: {e}")
async def on_error(self, event: ProgressEvent):
"""Send error update to WebSocket clients"""
logger.info(f"[WEBSOCKET-ERROR] Received error event for download {event.download_id}: {event.message}")
try:
error_data = {
"type": "error",
"id": event.download_id,
"download_id": event.download_id,
"status": "failed",
"message": event.message,
"data": event.data,
"timestamp": event.timestamp.isoformat()
}
# Update active_downloads - need to find the web interface download ID
# that corresponds to this BDFR download ID
web_download_id = None
for download_id, download_info in active_downloads.items():
if download_info.get("bdfr_download_id") == event.download_id:
web_download_id = download_id
break
if web_download_id:
active_downloads[web_download_id]["status"] = "failed"
active_downloads[web_download_id]["error"] = event.message
active_downloads[web_download_id]["end_time"] = datetime.now().isoformat()
# Persist failure phase for status_update broadcasting (e.g., 'rate_limited')
try:
if isinstance(event.data, dict) and event.data.get("phase"):
active_downloads[web_download_id]["phase"] = event.data.get("phase")
except Exception:
pass
logger.info(f"Updated download {web_download_id} to failed status")
# Unify IDs for frontend and include mapping
error_data["id"] = web_download_id
error_data["download_id"] = web_download_id
error_data["bdfr_download_id"] = event.download_id
error_data["web_download_id"] = web_download_id
error_data["subreddit"] = active_downloads[web_download_id].get("subreddit")
error_data["username"] = active_downloads[web_download_id].get("username")
# Also surface phase at top-level for clients that check data.phase or phase
try:
if isinstance(event.data, dict) and event.data.get("phase"):
error_data["phase"] = event.data.get("phase")
except Exception:
pass
else:
error_data["bdfr_download_id"] = event.download_id
await self.connection_manager.broadcast(json.dumps(error_data))
except Exception as e:
logger.warning(f"Failed to send error update for {event.download_id}: {e}")
logger.warning(f"Error data type: {type(event.data.get('exception'))}")
async def on_completed(self, event: ProgressEvent):
"""Send completion update to WebSocket clients"""
logger.info(f"[WEBSOCKET-COMPLETED] Received completion event for download {event.download_id}: {event.message}")
try:
completed_data = {
"type": "completed",
"id": event.download_id,
"download_id": event.download_id,
"status": "completed",
"message": event.message,
"progress": 100.0,
"data": event.data,
"timestamp": event.timestamp.isoformat()
}
# Update active_downloads - need to find the web interface download ID
# that corresponds to this BDFR download ID
web_download_id = None
for download_id, download_info in active_downloads.items():
if download_info.get("bdfr_download_id") == event.download_id:
web_download_id = download_id
break
if web_download_id:
active_downloads[web_download_id]["status"] = "completed"
active_downloads[web_download_id]["progress"] = 100.0
active_downloads[web_download_id]["end_time"] = datetime.now().isoformat()
logger.info(f"Updated download {web_download_id} to completed status")
# Unify IDs and include mapping data
completed_data["id"] = web_download_id
completed_data["download_id"] = web_download_id
completed_data["bdfr_download_id"] = event.download_id
completed_data["web_download_id"] = web_download_id
completed_data["subreddit"] = active_downloads[web_download_id].get("subreddit")
completed_data["username"] = active_downloads[web_download_id].get("username")
else:
completed_data["bdfr_download_id"] = event.download_id
await self.connection_manager.broadcast(json.dumps(completed_data))
except Exception as e:
logger.warning(f"Failed to send completion update for {event.download_id}: {e}")
# Helper function to create download with BDFR API
async def create_download_with_bdfr_api(download_type: str, name: str, **kwargs):
"""Create a download using the new BDFR API layer"""
# Check if BDFR is available
if not hasattr(bdfr_manager, 'download_subreddit'):
# BDFR not available, create a mock failed download
download_id = f"{download_type}_{name.replace('/', '_').replace(' ', '_')}_{datetime.now().strftime('%Y%m%d_%H%M%S')}"
active_downloads[download_id] = {
"id": download_id,
"type": download_type,
"status": "failed",
"progress": 0,
"start_time": datetime.now().isoformat(),
"authenticated": False,
"items_processed": 0,
"items_found": 0,
"current_item": None,
"current_item_type": None,
"phase": "failed",
"error": "BDFR API not available"
}
# Add type-specific fields
if download_type in ["subreddit", "archive", "clone"]:
active_downloads[download_id]["subreddit"] = name
elif download_type == "user":
active_downloads[download_id]["username"] = name
return download_id
# Get global auth token if available
global _bdfr_auth_token
auth_token = _bdfr_auth_token
logger.info(f"[DEBUG] Global auth token available: {auth_token is not None}")
logger.info(f"[DEBUG] Auth token for BDFR: {auth_token[:10]}..." if auth_token else "None")
# Create unique download ID
download_id = f"{download_type}_{name.replace('/', '_').replace(' ', '_')}_{datetime.now().strftime('%Y%m%d_%H%M%S')}"
# Get auth token if provided
auth_token = None
logger.info(f"[DEBUG] Checking auth_state: {kwargs.get('auth_state')}")
if kwargs.get('auth_state'):
try:
oauth_manager = get_oauth_manager()
logger.info(f"[DEBUG] OAuth manager exists: {oauth_manager is not None}")
auth_token = oauth_manager.get_valid_token(kwargs['auth_state'])
logger.info(f"[DEBUG] Auth token retrieved for state {kwargs['auth_state']}: {auth_token[:10]}..." if auth_token else "None")
except Exception as e:
logger.warning(f"Failed to get auth token: {e}")
# Initialize download tracking for WebSocket compatibility
active_downloads[download_id] = {
"id": download_id,
"type": download_type,
"status": "queued",
"progress": 0,
"start_time": datetime.now().isoformat(),
"limit": kwargs.get('limit'),
"authenticated": auth_token is not None,
"items_processed": 0,
"items_found": 0,
"current_item": None,
"current_item_type": None,
"phase": "queued"
}
# Add type-specific fields
if download_type in ["subreddit", "archive", "clone"]:
active_downloads[download_id]["subreddit"] = name
elif download_type == "user":
active_downloads[download_id]["username"] = name
# Create progress callback
callback = WebSocketProgressCallback(download_id, manager)
# Start download based on type
if download_type == "subreddit":
bdfr_download_id = bdfr_manager.download_subreddit(
name,
limit=kwargs.get('limit'),
sort=kwargs.get('sort', 'hot'),
time_filter=kwargs.get('time_filter', 'all'),
no_dupes=kwargs.get('no_dupes', False),
progress_callbacks=[callback]
)
elif download_type == "archive":
# Archive mode - metadata only
bdfr_download_id = bdfr_manager.archive_subreddit(
name,
format_type=kwargs.get('format', 'json'),
limit=kwargs.get('limit'),
progress_callbacks=[callback]
)
elif download_type == "clone":
# Clone mode - both download and archive
bdfr_download_id = bdfr_manager.clone_subreddit(
name,
limit=kwargs.get('limit'),
format_type=kwargs.get('format', 'json'),
no_dupes=kwargs.get('no_dupes', False),
progress_callbacks=[callback]
)
elif download_type == "user":
bdfr_download_id = bdfr_manager.download_user(
name,
limit=kwargs.get('limit'),
submitted=kwargs.get('submitted', True),
upvoted=kwargs.get('upvoted', False),
saved=kwargs.get('saved', False),
no_dupes=kwargs.get('no_dupes', False),
progress_callbacks=[callback]
)
else:
raise ValueError(f"Unsupported download type: {download_type}")
# Store BDFR download ID for tracking
active_downloads[download_id]["bdfr_download_id"] = bdfr_download_id
return download_id
@app.get("/", response_class=HTMLResponse)
async def home(request: Request):
"""Serve the main interface"""
return templates.TemplateResponse("index.html", {"request": request})
@app.get("/health")
async def health_check():
"""Health check endpoint"""
return {"status": "healthy", "timestamp": datetime.now().isoformat()}
# OAuth2 Authentication Endpoints
@app.get("/auth/login")
async def oauth_login(redirect_uri: str = None):
"""Initiate OAuth2 login flow"""
try:
# Use provided redirect_uri or fall back to configured default
if redirect_uri is None:
redirect_uri = os.getenv("BDFR_REDIRECT_URI", "http://localhost:8000/auth/callback")
oauth_manager = get_oauth_manager()
auth_data = oauth_manager.get_authorization_url(redirect_uri)
return {
"authorization_url": auth_data["authorization_url"],
"state": auth_data["state"],
"redirect_uri": redirect_uri,
"message": "Redirect user to the authorization URL"
}
except Exception as e:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to initiate OAuth2 login: {str(e)}"
)
@app.get("/auth/callback")
async def oauth_callback(
request: Request,
code: str = Query(..., description="Authorization code from Reddit"),
state: str = Query(..., description="State parameter for security"),
error: Optional[str] = Query(None, description="Error from OAuth2 provider")
):
"""Handle OAuth2 callback"""
try:
if error:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"OAuth2 error: {error}"
)
if not code or not state:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Missing code or state parameter"
)
oauth_manager = get_oauth_manager()
# Use the same redirect URI that was used for authorization
redirect_uri = os.getenv("BDFR_REDIRECT_URI", "http://localhost:8000/auth/callback")
token_data = await oauth_manager.exchange_code_for_token(code, state, redirect_uri)
# Get user info
user_info = await oauth_manager.get_user_info(token_data["access_token"])
# Return success page instead of JSON
return templates.TemplateResponse("auth_success.html", {"request": request})
except HTTPException:
raise
except Exception as e:
logger.error(f"OAuth2 callback error: {str(e)}")
# Return error page for unexpected errors
return templates.TemplateResponse("auth_error.html", {
"request": request,
"error": "Authentication failed",
"details": str(e)
})
except HTTPException:
raise
except Exception as e:
logger.error(f"OAuth2 callback error: {str(e)}")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Authentication failed"
)
@app.get("/auth/status")
async def auth_status(state: Optional[str] = Query(None)):
"""Get current authentication status"""
try:
oauth_manager = get_oauth_manager()
status_data = oauth_manager.get_auth_status(state)
return {
"authenticated": status_data["authenticated"],
"message": status_data.get("message", "Authenticated" if status_data["authenticated"] else "Not authenticated"),
"expires_at": status_data.get("expires_at"),
"scopes": status_data.get("scopes"),
"username": status_data.get("username")
}
except Exception as e:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to get auth status: {str(e)}"
)
@app.get("/api/auth/status")
async def api_auth_status(state: Optional[str] = Query(None)):
"""Get current authentication status (JSON API)"""
try:
oauth_manager = get_oauth_manager()
# If no state provided, check if there are any active sessions
if not state and oauth_manager:
# Check if there are any stored access tokens
if oauth_manager.access_tokens:
# Use the first available state for checking
state = next(iter(oauth_manager.access_tokens.keys()))
logger.info(f"No state provided, using first available: {state}")
status_data = oauth_manager.get_auth_status(state) if oauth_manager else {"authenticated": False, "message": "No OAuth manager"}
# Debug logging
logger.info(f"Auth status check - State: {state}, Authenticated: {status_data['authenticated']}")
# Store auth token for BDFR if authenticated
global _bdfr_auth_token
if status_data['authenticated'] and state:
_bdfr_auth_token = oauth_manager.get_valid_token(state)
logger.info(f"Stored auth token for BDFR manager: {_bdfr_auth_token is not None}")
elif not status_data['authenticated']:
_bdfr_auth_token = None
logger.info("Cleared auth token for BDFR manager")
return {
"authenticated": status_data["authenticated"],
"message": status_data.get("message", "Authenticated" if status_data["authenticated"] else "Not authenticated"),
"expires_at": status_data.get("expires_at", 0),
"scopes": status_data.get("scopes", []),
"username": status_data.get("username"),
"debug_info": {
"state_provided": state is not None,
"state_used": state,
"oauth_manager_exists": oauth_manager is not None,
"available_states": len(oauth_manager.oauth_states) if oauth_manager else 0,
"available_tokens": len(oauth_manager.access_tokens) if oauth_manager else 0,
"bdfr_auth_token_set": _bdfr_auth_token is not None
}
}
except Exception as e:
logger.error(f"Auth status error: {str(e)}")
return {
"authenticated": False,
"message": f"Error checking auth status: {str(e)}",
"error": True,
"debug_info": {
"error_details": str(e)
}
}
@app.post("/auth/logout")
async def auth_logout(state: str = Form(...)):
"""Logout and revoke OAuth2 session"""
try:
oauth_manager = get_oauth_manager()
oauth_manager.revoke_session(state)
return {
"message": "Successfully logged out",
"state": state
}
except Exception as e:
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Logout failed: {str(e)}"
)
@app.post("/api/download/subreddit")
async def download_subreddit(
subreddit: str = Form(...),
limit: int = Form(10),
sort: str = Form("hot"),
time_filter: str = Form(""),
min_score: str = Form(""),
no_dupes: bool = Form(False),
simple_check: bool = Form(False),
make_hard_links: bool = Form(False),
download_mode: str = Form("download"),
auth_state: str = Form(None)
):
"""Download from subreddit using BDFR API"""
# Map download_mode to appropriate BDFR operation
if download_mode == "archive":
# Use archive mode
download_id = await create_download_with_bdfr_api(
"archive",
subreddit,
limit=limit,
sort=sort,
time_filter=time_filter or "all",
format="json",
simple_check=simple_check,
auth_state=auth_state
)
message = f"Starting archive for r/{subreddit}"
elif download_mode == "clone":
# Use clone mode (download + archive)
download_id = await create_download_with_bdfr_api(
"clone",
subreddit,
limit=limit,
sort=sort,
time_filter=time_filter or "all",
no_dupes=no_dupes,
simple_check=simple_check,
format="json",
auth_state=auth_state
)
message = f"Starting clone for r/{subreddit}"
else:
# Default: download mode
download_id = await create_download_with_bdfr_api(
"subreddit",
subreddit,
limit=limit,
sort=sort,
time_filter=time_filter or "all",
no_dupes=no_dupes,
simple_check=simple_check,
auth_state=auth_state
)
message = f"Starting download for r/{subreddit}"
return {
"download_id": download_id,
"message": message,
"mode": download_mode,
"estimated_time": "2-3 minutes"
}
@app.post("/api/download/user")
async def download_user(
username: str = Form(...),
limit: int = Form(10),
submitted: bool = Form(True),
sort: str = Form("hot"),
time_filter: str = Form(""),
no_dupes: bool = Form(False),
simple_check: bool = Form(False),
make_hard_links: bool = Form(False),
download_mode: str = Form("download"),
auth_state: str = Form(None)
):
"""Download from user using BDFR API"""
# Map download_mode to appropriate BDFR operation
if download_mode == "archive":
# Use archive mode
download_id = await create_download_with_bdfr_api(
"archive",
username,
limit=limit,
sort=sort,
time_filter=time_filter or "all",
format="json",
submitted=submitted,
simple_check=simple_check,
auth_state=auth_state
)
message = f"Starting archive for u/{username}"
elif download_mode == "clone":
# Use clone mode (download + archive)
download_id = await create_download_with_bdfr_api(
"clone",
username,
limit=limit,
sort=sort,
time_filter=time_filter or "all",
no_dupes=no_dupes,
simple_check=simple_check,
format="json",
submitted=submitted,
auth_state=auth_state
)
message = f"Starting clone for u/{username}"
else:
# Default: download mode
download_id = await create_download_with_bdfr_api(
"user",
username,
limit=limit,
sort=sort,
submitted=submitted,
no_dupes=no_dupes,
simple_check=simple_check,
auth_state=auth_state
)
message = f"Starting download for u/{username}"
return {
"download_id": download_id,
"message": message,
"mode": download_mode,
"estimated_time": "1-2 minutes"
}
@app.get("/api/downloads")
async def get_downloads():
"""Get all active downloads"""
return {"downloads": active_downloads}
@app.get("/api/downloads/{download_id}")
async def get_download_status(download_id: str):
"""Get specific download status"""
if download_id not in active_downloads:
raise HTTPException(status_code=404, detail="Download not found")
return {"download": active_downloads[download_id]}
@app.delete("/api/downloads/{download_id}")
async def cancel_download(download_id: str):
"""Cancel a download"""
if download_id not in active_downloads:
raise HTTPException(status_code=404, detail="Download not found")
# Cancel in BDFR manager if we have the BDFR download ID
if "bdfr_download_id" in active_downloads[download_id]:
bdfr_manager.cancel_download(active_downloads[download_id]["bdfr_download_id"])
active_downloads[download_id]["status"] = "cancelled"
return {"message": "Download cancelled"}
@app.websocket("/ws/progress")
async def websocket_endpoint(websocket: WebSocket):
"""WebSocket endpoint for real-time progress updates"""
logger.info(f"[WEBSOCKET-ENDPOINT] New WebSocket connection attempt from {websocket.client}")
await manager.connect(websocket)
logger.info(f"[WEBSOCKET-ENDPOINT] WebSocket connection established successfully")
logger.info(f"[WEBSOCKET-ENDPOINT] Connection count: {len(manager.active_connections)}")
# Track last broadcast state to avoid redundant messages
last_broadcast_state = {}
cleanup_counter = 0
keepalive_counter = 0
try:
while True:
# Clean up dead connections periodically (every 10 iterations = 20 seconds)
cleanup_counter += 1
if cleanup_counter >= 10:
await manager.cleanup_dead_connections()
cleanup_counter = 0
# Send keepalive ping every 15 iterations (30 seconds) to prevent timeout
keepalive_counter += 1
if keepalive_counter >= 15:
try:
await websocket.send_json({"type": "keepalive", "timestamp": datetime.now().isoformat()})
logger.debug(f"[WEBSOCKET-ENDPOINT] Sent keepalive ping")
except Exception as e:
logger.warning(f"[WEBSOCKET-ENDPOINT] Failed to send keepalive: {e}")
break
keepalive_counter = 0
# Check for changes in download states
has_changes = False
for download_id, download in list(active_downloads.items()):
# Only check active downloads
if download.get("status") in ["running", "queued", "failed"]:
# Create a state snapshot for comparison
current_state = {
"status": download.get("status"),
"progress": download.get("progress", 0),
"phase": download.get("phase", "unknown"),
"items_processed": download.get("items_processed", 0),
"items_found": download.get("items_found", 0),
"current_item": download.get("current_item"),
}
# Compare with last broadcast state
if download_id not in last_broadcast_state or last_broadcast_state[download_id] != current_state:
has_changes = True
last_broadcast_state[download_id] = current_state
logger.debug(f"[WEBSOCKET-ENDPOINT] State change detected for {download_id}: {download.get('status')}")
status_message = {
"type": "status_update",
"id": download_id,
"download_id": download_id,
"status": download["status"],
"progress": download.get("progress", 0),
"phase": download.get("phase", "unknown"),
"items_processed": download.get("items_processed", 0),
"items_found": download.get("items_found", 0),
"limit": download.get("limit", 0),
"current_item": download.get("current_item"),
"current_item_type": download.get("current_item_type"),
"current_subreddit": download.get("current_subreddit"),
"username": download.get("username"),
"subreddit": download.get("subreddit"),
"message": f"{download.get('subreddit', download.get('username', 'content'))} - {download['status']}"
}
try:
await manager.broadcast(json.dumps(status_message))
except Exception as e:
logger.warning(f"Error broadcasting status update: {e}")
if not has_changes:
logger.debug(f"[WEBSOCKET-ENDPOINT] No state changes detected, skipping broadcast")
await asyncio.sleep(2) # Check every 2 seconds
except WebSocketDisconnect:
logger.info(f"[WEBSOCKET-ENDPOINT] WebSocket disconnected normally from {websocket.client}")
manager.disconnect(websocket)
except Exception as e:
logger.error(f"[WEBSOCKET-ENDPOINT] WebSocket error from {websocket.client}: {e}")
manager.disconnect(websocket)
@app.get("/api/bdfr/status")
async def bdfr_status():
"""Get BDFR status and available options"""
return {
"bdfr_available": BDFR_AVAILABLE,
"version": "2.0.0",
"supported_operations": [
"subreddit_download",
"user_download",
"custom_filter"
],
"output_formats": ["json", "csv", "xml"]
}
@app.get("/api/websocket/status")
async def websocket_status():
"""Get WebSocket connection status for debugging"""
return {
"active_connections": len(manager.active_connections),
"connection_failures": manager.connection_failures,
"active_downloads": len(active_downloads),
"websocket_endpoint": "/ws/progress",
"server_host": "0.0.0.0",
"server_port": 8000
}
@app.get("/api/oauth/debug")
async def oauth_debug():
"""Debug endpoint to check OAuth state"""
try:
oauth_manager = get_oauth_manager()
return {
"oauth_manager_initialized": oauth_manager is not None,
"oauth_states_count": len(oauth_manager.oauth_states) if oauth_manager else 0,
"access_tokens_count": len(oauth_manager.access_tokens) if oauth_manager else 0,
"refresh_tokens_count": len(oauth_manager.refresh_tokens) if oauth_manager else 0,
"client_id_configured": oauth_manager.client_id != "U-6gk4ZCh3IeNQ" if oauth_manager else False,
"redirect_uri": os.getenv("BDFR_REDIRECT_URI", "http://localhost:8000/auth/callback")
}
except Exception as e:
return {
"error": str(e),
"oauth_manager_initialized": False
}
if __name__ == "__main__":
import uvicorn
uvicorn.run("app.main:app", host="0.0.0.0", port=8000, reload=True)
+11
View File
@@ -0,0 +1,11 @@
fastapi>=0.100.0
uvicorn[standard]>=0.20.0
websockets>=10.0
jinja2>=3.1.0
python-multipart>=0.0.6
aiofiles>=0.23.0
python-dotenv>=1.0.0
pydantic>=2.0.0
pydantic-settings>=2.0.0
requests>=2.25.0
httpx>=0.24.0
+123
View File
@@ -0,0 +1,123 @@
#!/usr/bin/env python3
"""
BDFR Web Interface OAuth Setup Helper
This script helps you set up Reddit OAuth for the BDFR web interface.
Run this script to configure your OAuth credentials and redirect URI.
"""
import os
import sys
from pathlib import Path
def create_env_file():
"""Create .env file from template"""
env_example = Path(__file__).parent / ".env.example"
env_file = Path(__file__).parent / ".env"
if not env_example.exists():
print("❌ Error: .env.example not found")
return False
if env_file.exists():
print("⚠️ .env file already exists")
response = input("Do you want to overwrite it? (y/N): ").lower().strip()
if response != 'y':
print("Setup cancelled")
return False
# Copy .env.example to .env
with open(env_example, 'r') as src, open(env_file, 'w') as dst:
dst.write(src.read())
print("✅ Created .env file from template")
return True
def get_oauth_instructions():
"""Display OAuth setup instructions"""
print("\n" + "="*60)
print("🔐 REDDIT OAUTH SETUP INSTRUCTIONS")
print("="*60)
print()
print("To use the BDFR Web Interface authentication features, you need to:")
print()
print("1. 📱 CREATE OR UPDATE REDDIT OAUTH APP:")
print(" • Go to: https://www.reddit.com/prefs/apps")
print(" • Find your app or click 'Create App'")
print(" • Set the redirect URI to: http://localhost:8000/auth/callback")
print()
print("2. 📝 COPY YOUR CREDENTIALS:")
print(" • After creating/editing the app, copy the client ID and secret")
print(" • These are the values that look like: 7CZHY6AmKweZME5s50SfDGylaPg")
print()
print("3. ✏️ EDIT YOUR CONFIGURATION:")
print(" • Open the .env file that was just created")
print(" • Update BDFR_REDIRECT_URI if using a different port/domain")
print(" • Update BDFR_CLIENT_ID with your OAuth client ID")
print(" • Update BDFR_CLIENT_SECRET with your OAuth client secret")
print(" • OR update bdfr/default_config.cfg with your OAuth credentials")
print()
print("💡 TIP: Use the .env file for web interface configuration")
print(" and bdfr/default_config.cfg for CLI tool configuration")
print()
print("="*60)
print()
input("Press Enter to open the .env file for editing...")
return True
def open_env_file():
"""Open .env file in default editor"""
env_file = Path(__file__).parent / ".env"
if not env_file.exists():
print("❌ Error: .env file not found")
return False
print(f"📝 Opening {env_file} for editing...")
# Try to open with default editor
editor = os.getenv('EDITOR', 'notepad' if os.name == 'nt' else 'nano')
try:
if os.name == 'nt': # Windows
os.startfile(env_file)
else: # Unix-like
os.system(f"{editor} {env_file}")
return True
except Exception as e:
print(f"❌ Error opening editor: {e}")
print(f"📍 Please manually edit the file: {env_file}")
return False
def main():
"""Main setup function"""
print("🚀 BDFR Web Interface OAuth Setup")
print("=" * 40)
# Create .env file
if not create_env_file():
return 1
# Show instructions
if not get_oauth_instructions():
return 1
# Open .env file for editing
if not open_env_file():
print("📝 Please manually edit the .env file with your OAuth settings")
print("📍 File location:", Path(__file__).parent / ".env")
print("\n✅ OAuth setup initiated!")
print("📖 Check STARTUP.md for detailed setup instructions")
print("🚀 Run 'python start.py' to start the web interface after configuration")
return 0
if __name__ == "__main__":
sys.exit(main())
+105
View File
@@ -0,0 +1,105 @@
@echo off
REM BDFR Web Interface Startup Script for Windows
REM This script provides an easy way to start the BDFR web interface on Windows
setlocal enabledelayedexpansion
REM Colors for output (Windows 10+)
set "RED=[91m"
set "GREEN=[92m"
set "YELLOW=[93m"
set "BLUE=[94m"
set "NC=[0m"
REM Function to print colored output (simplified for Windows)
echo 🌟 BDFR Web Interface Startup
echo ==================================================
REM Check if we're in the right directory
if not exist "requirements.txt" (
echo ❌ Error: Please run this script from the web_interface directory
echo Usage: start.bat
pause
exit /b 1
)
if not exist "app" (
echo ❌ Error: app directory not found
echo Please make sure you're in the web_interface directory
pause
exit /b 1
)
echo ️ Checking Python version...
REM Check Python version
python --version > temp_python_version.txt 2>&1
set /p PYTHON_VERSION=<temp_python_version.txt
del temp_python_version.txt
echo ✅ Python version: %PYTHON_VERSION%
REM Check if Python 3.8+ is available (simplified check)
echo %PYTHON_VERSION% | findstr /C:"Python 3." >nul
if errorlevel 1 (
echo ❌ Error: Python 3 is required
echo Current version: %PYTHON_VERSION%
pause
exit /b 1
)
REM Install dependencies
echo ️ Checking and installing dependencies...
if exist requirements.txt (
echo ️ Installing Python dependencies...
python -m pip install -r requirements.txt
if !errorlevel! neq 0 (
echo ❌ Failed to install dependencies
pause
exit /b 1
)
echo ✅ Dependencies installed successfully
) else (
echo ❌ requirements.txt not found
pause
exit /b 1
)
REM Check if BDFR module is available
echo ️ Checking BDFR module availability...
python -c "import sys; sys.path.insert(0, '../bdfr'); import bdfr.api; print('BDFR API imported successfully')" >nul 2>&1
if !errorlevel! neq 0 (
echo ⚠️ Warning: BDFR module not found in Python path
echo Make sure the parent directory is in your Python path
echo Or run this script from the project root directory
echo Attempting to install BDFR...
cd ..
python -m pip install -e .
cd web_interface
if !errorlevel! neq 0 (
echo ❌ Failed to install BDFR
pause
exit /b 1
)
echo ✅ BDFR installed successfully
) else (
echo ✅ BDFR module found
)
REM Start the server
echo.
echo ==================================================
echo ️ Starting BDFR Web Interface...
echo ️ Server will be available at: http://localhost:8000
echo ️ API documentation at: http://localhost:8000/docs
echo ️ Press Ctrl+C to stop the server
echo ==================================================
REM Start uvicorn server
python -m uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
REM This code runs when the server is stopped
echo.
echo ️ BDFR Web Interface stopped
pause
+119
View File
@@ -0,0 +1,119 @@
#!/usr/bin/env python3
"""
BDFR Web Interface Startup Script
This script provides an easy way to start the BDFR web interface with
proper dependency management and error handling.
"""
import os
import sys
import subprocess
import importlib.util
from pathlib import Path
def check_python_version():
"""Check if Python version is compatible (3.8+)"""
if sys.version_info < (3, 8):
print("ERROR: Python 3.8 or higher is required")
print(f"Current version: {sys.version}")
sys.exit(1)
def install_dependencies():
"""Install required dependencies if missing"""
requirements_path = Path(__file__).parent / "requirements.txt"
if not requirements_path.exists():
print("❌ Error: requirements.txt not found")
sys.exit(1)
print("Checking and installing dependencies...")
try:
# Try to import required modules first
required_modules = [
'fastapi',
'uvicorn',
'websockets',
'jinja2'
]
missing_modules = []
for module in required_modules:
if not importlib.util.find_spec(module):
missing_modules.append(module)
if missing_modules:
print(f"Installing missing modules: {', '.join(missing_modules)}")
subprocess.check_call([
sys.executable, '-m', 'pip', 'install', '-r', str(requirements_path)
])
else:
print("All dependencies are already installed")
except subprocess.CalledProcessError as e:
print(f"❌ Error installing dependencies: {e}")
sys.exit(1)
except Exception as e:
print(f"❌ Error checking dependencies: {e}")
sys.exit(1)
def check_bdfr_module():
"""Check if BDFR module is available"""
try:
importlib.util.find_spec('bdfr')
print("BDFR module found")
except ImportError:
print("⚠️ Warning: BDFR module not found in Python path")
print("Make sure the parent directory is in your Python path or run from project root")
def start_server():
"""Start the FastAPI server"""
print("Starting BDFR Web Interface...")
print("Server will be available at: http://localhost:8000")
print("API documentation at: http://localhost:8000/docs")
print("Press Ctrl+C to stop the server")
print("-" * 50)
try:
# Start uvicorn server
subprocess.call([
sys.executable, '-m', 'uvicorn',
'app.main:app',
'--host', '0.0.0.0',
'--port', '8000',
'--reload'
])
except KeyboardInterrupt:
print("\n🛑 Server stopped by user")
except Exception as e:
print(f"❌ Error starting server: {e}")
sys.exit(1)
def main():
"""Main startup function"""
print("BDFR Web Interface Startup")
print("=" * 40)
# Change to web_interface directory
web_interface_dir = Path(__file__).parent
os.chdir(web_interface_dir)
# Pre-flight checks
check_python_version()
install_dependencies()
check_bdfr_module()
print("\n" + "=" * 40)
# Start the server
start_server()
if __name__ == "__main__":
main()
+105
View File
@@ -0,0 +1,105 @@
#!/bin/bash
# BDFR Web Interface Startup Script
# Compatible with Linux, macOS, and other Unix-like systems
set -e # Exit on any error
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Function to print colored output
print_info() {
echo -e "${BLUE}$1${NC}"
}
print_success() {
echo -e "${GREEN}$1${NC}"
}
print_warning() {
echo -e "${YELLOW}⚠️ $1${NC}"
}
print_error() {
echo -e "${RED}$1${NC}"
}
# Check if we're in the right directory
if [[ ! -f "requirements.txt" ]] || [[ ! -d "app" ]]; then
print_error "Error: Please run this script from the web_interface directory"
echo "Usage: ./start.sh"
exit 1
fi
print_info "BDFR Web Interface Startup"
echo "=================================================="
# Check Python version
print_info "Checking Python version..."
PYTHON_VERSION=$(python3 --version 2>&1 | awk '{print $2}')
print_success "Python version: $PYTHON_VERSION"
# Check if Python 3.8+ is available
PYTHON_MAJOR=$(echo $PYTHON_VERSION | cut -d. -f1)
PYTHON_MINOR=$(echo $PYTHON_VERSION | cut -d. -f2)
if [[ $PYTHON_MAJOR -lt 3 ]] || [[ $PYTHON_MAJOR -eq 3 && $PYTHON_MINOR -lt 8 ]]; then
print_error "Python 3.8 or higher is required"
print_error "Current version: $PYTHON_VERSION"
exit 1
fi
# Install dependencies
print_info "Checking and installing dependencies..."
if [[ -f "requirements.txt" ]]; then
# Check if pip is available
if ! command -v pip3 &> /dev/null; then
print_error "pip3 is not installed. Please install Python 3 and pip first."
exit 1
fi
# Install/update requirements
print_info "Installing Python dependencies..."
pip3 install -r requirements.txt
if [[ $? -eq 0 ]]; then
print_success "Dependencies installed successfully"
else
print_error "Failed to install dependencies"
exit 1
fi
else
print_error "requirements.txt not found"
exit 1
fi
# Check if BDFR module is available
print_info "Checking BDFR module availability..."
if python3 -c "import bdfr" 2>/dev/null; then
print_success "BDFR module found"
else
print_warning "BDFR module not found in Python path"
print_warning "Make sure the parent directory is in your Python path"
print_warning "Or run this script from the project root directory"
fi
# Start the server
echo ""
echo "=================================================="
print_info "Starting BDFR Web Interface..."
print_info "Server will be available at: http://localhost:8000"
print_info "API documentation at: http://localhost:8000/docs"
print_info "Press Ctrl+C to stop the server"
echo "=================================================="
# Start uvicorn server with proper error handling
python3 -m uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
# This code runs when the server is stopped
echo ""
print_info "BDFR Web Interface stopped"
+817
View File
@@ -0,0 +1,817 @@
/* Reset and base styles */
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, sans-serif;
line-height: 1.6;
color: #333;
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
min-height: 100vh;
}
.container {
max-width: 1200px;
margin: 0 auto;
padding: 20px;
}
/* Header */
header {
text-align: center;
margin-bottom: 40px;
color: white;
position: relative;
}
header h1 {
font-size: 3rem;
font-weight: 700;
margin-bottom: 10px;
text-shadow: 0 2px 4px rgba(0,0,0,0.3);
}
.subtitle {
font-size: 1.2rem;
opacity: 0.9;
font-weight: 300;
}
/* Authentication section */
.auth-section {
margin-top: 20px;
padding: 15px;
background: rgba(255, 255, 255, 0.1);
border-radius: 10px;
backdrop-filter: blur(10px);
border: 1px solid rgba(255, 255, 255, 0.2);
}
.auth-status {
display: flex;
justify-content: space-between;
align-items: center;
flex-wrap: wrap;
gap: 15px;
}
.auth-info {
display: flex;
align-items: center;
gap: 10px;
flex-wrap: wrap;
}
.auth-label {
font-weight: 600;
color: rgba(255, 255, 255, 0.9);
}
.auth-user {
font-weight: 500;
color: white;
background: rgba(255, 255, 255, 0.2);
padding: 4px 8px;
border-radius: 4px;
font-size: 0.9rem;
}
.auth-status-indicator {
font-weight: 600;
padding: 4px 8px;
border-radius: 4px;
font-size: 0.85rem;
}
.auth-status-indicator.connected {
background: rgba(40, 167, 69, 0.2);
color: #28a745;
border: 1px solid rgba(40, 167, 69, 0.3);
}
.auth-status-indicator.disconnected {
background: rgba(220, 53, 69, 0.2);
color: #dc3545;
border: 1px solid rgba(220, 53, 69, 0.3);
}
.auth-actions {
display: flex;
gap: 10px;
}
/* Main content */
main {
background: white;
border-radius: 15px;
padding: 30px;
box-shadow: 0 10px 30px rgba(0,0,0,0.2);
margin-bottom: 30px;
}
/* Download section */
.download-section {
margin-bottom: 40px;
}
.form-container {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(400px, 1fr));
gap: 30px;
margin-bottom: 30px;
}
.form-container-unified {
max-width: 900px;
margin: 0 auto;
}
.form-card {
background: #f8f9fa;
padding: 25px;
border-radius: 10px;
border-left: 4px solid #667eea;
box-shadow: 0 2px 10px rgba(0,0,0,0.1);
}
.form-card-unified {
background: #f8f9fa;
padding: 30px;
border-radius: 10px;
border-left: 4px solid #667eea;
box-shadow: 0 2px 10px rgba(0,0,0,0.1);
}
.form-card h2,
.form-card-unified h2 {
color: #2c3e50;
margin-bottom: 20px;
font-size: 1.5rem;
}
.form-card:nth-child(2) {
border-left-color: #764ba2;
}
/* Form styles */
.download-form {
display: flex;
flex-direction: column;
gap: 15px;
}
.form-group {
display: flex;
flex-direction: column;
}
.form-group label {
margin-bottom: 5px;
font-weight: 600;
color: #555;
}
.form-group input,
.form-group select {
padding: 12px;
border: 2px solid #e1e8ed;
border-radius: 6px;
font-size: 1rem;
transition: border-color 0.3s ease;
}
.form-group input:focus,
.form-group select:focus {
outline: none;
border-color: #667eea;
box-shadow: 0 0 0 3px rgba(102, 126, 234, 0.1);
}
/* Form layout enhancements */
.form-row {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 15px;
}
.form-section {
margin-top: 25px;
padding-top: 20px;
border-top: 1px solid #e9ecef;
}
.form-section h4 {
color: #2c3e50;
font-size: 1.1rem;
margin-bottom: 15px;
display: flex;
align-items: center;
gap: 8px;
}
.form-help {
font-size: 0.85rem;
color: #666;
margin-top: 4px;
line-height: 1.4;
}
/* Checkbox styles */
.checkbox-group {
display: flex;
flex-direction: column;
gap: 12px;
}
.checkbox-label {
display: flex;
align-items: center;
cursor: pointer;
font-weight: 500;
padding: 8px;
border-radius: 6px;
transition: background-color 0.2s ease;
}
.checkbox-label:hover {
background-color: #f0f2f5;
}
.checkbox-label input[type="checkbox"] {
display: none;
}
.checkmark {
width: 20px;
height: 20px;
border: 2px solid #ddd;
border-radius: 4px;
margin-right: 10px;
position: relative;
transition: all 0.2s ease;
}
.checkbox-label input[type="checkbox"]:checked + .checkmark {
background-color: #667eea;
border-color: #667eea;
}
.checkbox-label input[type="checkbox"]:checked + .checkmark::after {
content: '✓';
position: absolute;
top: 50%;
left: 50%;
transform: translate(-50%, -50%);
color: white;
font-size: 14px;
font-weight: bold;
}
/* Radio button styles */
.radio-group {
display: flex;
flex-direction: column;
gap: 10px;
}
.radio-label {
display: flex;
align-items: center;
cursor: pointer;
font-weight: 500;
padding: 8px;
border-radius: 6px;
transition: background-color 0.2s ease;
}
.radio-label:hover {
background-color: #f0f2f5;
}
.radio-label input[type="radio"] {
display: none;
}
.radio-custom {
width: 20px;
height: 20px;
border: 2px solid #ddd;
border-radius: 50%;
margin-right: 10px;
position: relative;
transition: all 0.2s ease;
}
.radio-label input[type="radio"]:checked + .radio-custom {
border-color: #667eea;
}
.radio-label input[type="radio"]:checked + .radio-custom::after {
content: '';
position: absolute;
top: 50%;
left: 50%;
transform: translate(-50%, -50%);
width: 10px;
height: 10px;
background-color: #667eea;
border-radius: 50%;
}
/* Mode Selection Styles */
.mode-section {
background: white;
padding: 20px;
border-radius: 8px;
margin-bottom: 25px;
border: 2px solid #e1e8ed;
}
.mode-radio-group {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(150px, 1fr));
gap: 15px;
}
.mode-option {
background: white;
border: 2px solid #e1e8ed;
border-radius: 8px;
padding: 15px 12px;
transition: all 0.3s ease;
position: relative;
}
.mode-option:hover {
background-color: #f8f9fa;
border-color: #667eea;
transform: translateY(-2px);
box-shadow: 0 4px 12px rgba(102, 126, 234, 0.15);
}
.mode-option input[type="radio"]:checked + .radio-custom + .mode-label {
color: #667eea;
}
.mode-option input[type="radio"]:checked {
& ~ * {
border-color: #667eea;
}
}
.mode-option input[type="radio"]:checked + .radio-custom {
border-color: #667eea;
background-color: rgba(102, 126, 234, 0.1);
}
.mode-label {
display: flex;
flex-direction: column;
gap: 4px;
margin-left: 8px;
}
.mode-label strong {
font-size: 1rem;
color: #2c3e50;
}
.mode-label small {
font-size: 0.85rem;
color: #666;
font-weight: normal;
}
/* Tooltip Styles */
[data-tooltip] {
position: relative;
cursor: help;
}
[data-tooltip]::before {
content: attr(data-tooltip);
position: absolute;
bottom: calc(100% + 10px);
left: 50%;
transform: translateX(-50%) scale(0.95);
padding: 10px 15px;
background: #2c3e50;
color: white;
border-radius: 6px;
font-size: 0.85rem;
font-weight: 400;
line-height: 1.4;
white-space: normal;
width: max-content;
max-width: 280px;
opacity: 0;
pointer-events: none;
transition: all 0.2s ease;
z-index: 1000;
box-shadow: 0 4px 12px rgba(0,0,0,0.15);
}
[data-tooltip]::after {
content: '';
position: absolute;
bottom: calc(100% + 4px);
left: 50%;
transform: translateX(-50%) scale(0.95);
border: 6px solid transparent;
border-top-color: #2c3e50;
opacity: 0;
pointer-events: none;
transition: all 0.2s ease;
z-index: 1000;
}
[data-tooltip]:hover::before,
[data-tooltip]:hover::after {
opacity: 1;
transform: translateX(-50%) scale(1);
}
/* Button variations */
.btn-small {
padding: 8px 16px;
font-size: 0.9rem;
}
.btn-outline {
background: transparent;
border: 2px solid #6c757d;
color: #6c757d;
}
.btn-outline:hover {
background: #6c757d;
color: white;
}
/* Empty state styling */
.empty-state {
text-align: center;
padding: 40px 20px;
color: #666;
}
.empty-icon {
font-size: 3rem;
margin-bottom: 15px;
opacity: 0.5;
}
.empty-state p {
margin-bottom: 8px;
}
/* Downloads header */
.downloads-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 15px;
}
/* Buttons */
.btn {
padding: 12px 24px;
border: none;
border-radius: 6px;
font-size: 1rem;
font-weight: 600;
cursor: pointer;
transition: all 0.3s ease;
text-transform: uppercase;
letter-spacing: 0.5px;
}
.btn-primary {
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
color: white;
}
.btn-primary:hover {
transform: translateY(-2px);
box-shadow: 0 5px 15px rgba(102, 126, 234, 0.4);
}
.btn-secondary {
background: linear-gradient(135deg, #764ba2 0%, #667eea 100%);
color: white;
}
.btn-secondary:hover {
transform: translateY(-2px);
box-shadow: 0 5px 15px rgba(118, 75, 162, 0.4);
}
/* Progress section */
.progress-section {
margin-bottom: 40px;
}
.progress-section h2 {
color: #2c3e50;
margin-bottom: 20px;
font-size: 1.8rem;
}
.progress-container {
background: #f8f9fa;
border-radius: 10px;
padding: 20px;
min-height: 100px;
display: flex;
align-items: center;
justify-content: center;
}
.no-downloads {
text-align: center;
color: #666;
}
.no-downloads p {
margin-bottom: 10px;
}
/* Downloads list */
.downloads-list {
background: #f8f9fa;
border-radius: 10px;
padding: 20px;
}
.downloads-list h3 {
margin-bottom: 15px;
color: #2c3e50;
}
.downloads-items {
display: flex;
flex-direction: column;
gap: 15px;
}
/* Progress card */
.progress-card {
background: white;
padding: 20px;
border-radius: 8px;
border-left: 4px solid #28a745;
box-shadow: 0 2px 8px rgba(0,0,0,0.1);
}
.progress-card.downloading {
border-left-color: #ffc107;
animation: pulse 2s infinite;
}
.progress-card.running {
border-left-color: #ffc107;
animation: pulse 2s infinite;
}
.progress-card.completed {
border-left-color: #28a745;
}
.progress-card.failed {
border-left-color: #dc3545;
}
.progress-card.error {
border-left-color: #dc3545;
}
.progress-header {
display: flex;
justify-content: between;
align-items: center;
margin-bottom: 15px;
}
.progress-info h4 {
color: #2c3e50;
margin-bottom: 5px;
}
.progress-meta {
font-size: 0.9rem;
color: #666;
}
.progress-status {
font-weight: 600;
padding: 4px 8px;
border-radius: 4px;
font-size: 0.8rem;
}
.status-starting {
background: #e7f3ff;
color: #0066cc;
}
.status-downloading {
background: #fff3cd;
color: #856404;
}
.status-completed {
background: #d4edda;
color: #155724;
}
.status-error {
background: #f8d7da;
color: #721c24;
}
/* Progress bar */
.progress-bar-container {
margin-bottom: 10px;
}
.progress-bar {
width: 100%;
height: 8px;
background: #e9ecef;
border-radius: 4px;
overflow: hidden;
}
.progress-fill {
height: 100%;
background: linear-gradient(90deg, #667eea 0%, #764ba2 100%);
border-radius: 4px;
transition: width 0.3s ease;
}
.progress-text {
font-size: 0.9rem;
color: #666;
margin-bottom: 8px;
}
.progress-details {
font-size: 0.8rem;
color: #888;
margin-top: 8px;
}
.progress-details div {
margin-bottom: 2px;
}
.progress-phase {
font-size: 0.85rem;
color: #555;
font-weight: 500;
margin-top: 4px;
}
.current-item {
font-size: 0.85rem;
color: #007bff;
font-weight: 500;
margin-top: 4px;
padding: 4px 8px;
background-color: rgba(0, 123, 255, 0.1);
border-radius: 4px;
border-left: 3px solid #007bff;
}
/* Progress controls buttons */
.progress-controls {
display: flex;
gap: 8px;
align-items: center;
}
.btn-retry {
padding: 6px 12px;
background: #ffc107;
color: #000;
border: none;
border-radius: 4px;
font-size: 0.85rem;
font-weight: 600;
cursor: pointer;
transition: all 0.2s ease;
text-transform: uppercase;
letter-spacing: 0.5px;
}
.btn-retry:hover {
background: #ffca2c;
transform: translateY(-1px);
box-shadow: 0 2px 8px rgba(255, 193, 7, 0.4);
}
.btn-retry:active {
transform: translateY(0);
}
/* Status section */
.status-section {
margin-bottom: 30px;
}
.status-card {
background: #f8f9fa;
padding: 20px;
border-radius: 10px;
border-left: 4px solid #17a2b8;
}
.status-card h3 {
color: #2c3e50;
margin-bottom: 15px;
}
.status-item {
display: flex;
justify-content: space-between;
margin-bottom: 10px;
padding: 8px 0;
border-bottom: 1px solid #e9ecef;
}
.status-item:last-child {
border-bottom: none;
}
.status-label {
font-weight: 600;
color: #555;
}
.status-value {
font-weight: 500;
}
.status-online {
color: #28a745;
}
.status-offline {
color: #dc3545;
}
/* Footer */
footer {
text-align: center;
color: white;
opacity: 0.8;
}
/* Animations */
@keyframes pulse {
0% { box-shadow: 0 2px 8px rgba(0,0,0,0.1); }
50% { box-shadow: 0 4px 16px rgba(255, 193, 7, 0.3); }
100% { box-shadow: 0 2px 8px rgba(0,0,0,0.1); }
}
/* Responsive design */
@media (max-width: 768px) {
.container {
padding: 15px;
}
header h1 {
font-size: 2.5rem;
}
.form-container {
grid-template-columns: 1fr;
}
.form-card {
margin-bottom: 20px;
}
.status-item {
flex-direction: column;
align-items: flex-start;
gap: 5px;
}
}
@media (max-width: 480px) {
header h1 {
font-size: 2rem;
}
main {
padding: 20px;
}
.form-card {
padding: 20px;
}
}
File diff suppressed because it is too large Load Diff
+86
View File
@@ -0,0 +1,86 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Authentication Failed - BDFR Web Interface</title>
<style>
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
background: linear-gradient(135deg, #ff6b6b 0%, #ee5a52 100%);
margin: 0;
padding: 0;
min-height: 100vh;
display: flex;
align-items: center;
justify-content: center;
}
.error-container {
background: white;
padding: 2rem;
border-radius: 10px;
box-shadow: 0 10px 25px rgba(0,0,0,0.1);
text-align: center;
max-width: 400px;
width: 90%;
}
.error-icon {
font-size: 3rem;
color: #f44336;
margin-bottom: 1rem;
}
.error-title {
color: #333;
margin-bottom: 1rem;
font-size: 1.5rem;
}
.error-message {
color: #666;
margin-bottom: 1.5rem;
line-height: 1.5;
}
.error-details {
background: #f5f5f5;
padding: 1rem;
border-radius: 5px;
font-family: monospace;
font-size: 0.8rem;
color: #888;
margin-bottom: 1.5rem;
text-align: left;
white-space: pre-wrap;
word-break: break-all;
}
.retry-button {
background: #667eea;
color: white;
padding: 0.75rem 1.5rem;
border: none;
border-radius: 5px;
text-decoration: none;
display: inline-block;
margin-top: 1rem;
transition: background 0.3s;
}
.retry-button:hover {
background: #5a67d8;
}
</style>
</head>
<body>
<div class="error-container">
<div class="error-icon"></div>
<h1 class="error-title">Authentication Failed</h1>
<p class="error-message">
There was an error during the authentication process. This might be due to:
</p>
<ul style="text-align: left; color: #666; margin: 1rem 0;">
<li>Invalid or expired authorization code</li>
<li>Mismatched redirect URI configuration</li>
<li>Reddit OAuth app not properly configured</li>
</ul>
<div class="error-details">{{ error }}</div>
<a href="/?auth_error=true" class="retry-button">Return to Main Page</a>
</div>
</body>
</html>
+82
View File
@@ -0,0 +1,82 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Authentication Successful - BDFR Web Interface</title>
<style>
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
margin: 0;
padding: 0;
min-height: 100vh;
display: flex;
align-items: center;
justify-content: center;
}
.success-container {
background: white;
padding: 2rem;
border-radius: 10px;
box-shadow: 0 10px 25px rgba(0,0,0,0.1);
text-align: center;
max-width: 400px;
width: 90%;
}
.success-icon {
font-size: 3rem;
color: #4CAF50;
margin-bottom: 1rem;
}
.success-title {
color: #333;
margin-bottom: 1rem;
font-size: 1.5rem;
}
.success-message {
color: #666;
margin-bottom: 1.5rem;
line-height: 1.5;
}
.redirect-message {
color: #888;
font-size: 0.9rem;
margin-top: 1rem;
}
.spinner {
border: 3px solid #f3f3f3;
border-top: 3px solid #667eea;
border-radius: 50%;
width: 30px;
height: 30px;
animation: spin 1s linear infinite;
margin: 0 auto 1rem;
}
@keyframes spin {
0% { transform: rotate(0deg); }
100% { transform: rotate(360deg); }
}
</style>
</head>
<body>
<div class="success-container">
<div class="success-icon"></div>
<h1 class="success-title">Authentication Successful!</h1>
<p class="success-message">
You have successfully authenticated with Reddit. You can now use all features of the BDFR Web Interface.
</p>
<div class="spinner"></div>
<p class="redirect-message">
Redirecting you back to the main interface...
</p>
</div>
<script>
// Redirect to main page after 3 seconds
setTimeout(function() {
window.location.href = '/?authenticated=true';
}, 3000);
</script>
</body>
</html>
+209
View File
@@ -0,0 +1,209 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>BDFR Web Interface</title>
<link rel="stylesheet" href="/static/css/style.css">
</head>
<body>
<div class="container">
<header>
<h1>Bulk Downloader for Reddit</h1>
<p class="subtitle">Web Interface</p>
<!-- Authentication Status -->
<div id="authSection" class="auth-section" style="display: none;">
<div class="auth-status">
<div class="auth-info">
<span class="auth-label">Reddit Account:</span>
<span id="authUser" class="auth-user">-</span>
<span id="authStatus" class="auth-status-indicator">🔴 Not Connected</span>
</div>
<div class="auth-actions">
<button id="loginBtn" class="btn btn-small btn-outline">🔐 Login with Reddit</button>
<button id="logoutBtn" class="btn btn-small btn-outline" style="display: none;">🚪 Logout</button>
</div>
</div>
</div>
</header>
<main>
<!-- Unified Download Form Section -->
<section class="download-section">
<div class="form-container-unified">
<div class="form-card-unified">
<h2>📥 Download Reddit Content</h2>
<form id="unifiedForm" class="download-form">
<!-- Mode Selection -->
<div class="form-section mode-section">
<h4>🎯 Download Mode</h4>
<div class="radio-group mode-radio-group">
<label class="radio-label mode-option" data-tooltip="Download media files (images, videos, gifs) from posts">
<input type="radio" name="download_mode" value="download" checked>
<span class="radio-custom"></span>
<span class="mode-label">
<strong>Download</strong>
<small>Media files only</small>
</span>
</label>
<label class="radio-label mode-option" data-tooltip="Save post metadata (title, author, comments) as JSON/XML without downloading media">
<input type="radio" name="download_mode" value="archive">
<span class="radio-custom"></span>
<span class="mode-label">
<strong>Archive</strong>
<small>Metadata only</small>
</span>
</label>
<label class="radio-label mode-option" data-tooltip="Download media files AND save metadata - complete backup of posts">
<input type="radio" name="download_mode" value="clone">
<span class="radio-custom"></span>
<span class="mode-label">
<strong>Clone</strong>
<small>Media + Metadata</small>
</span>
</label>
</div>
</div>
<!-- Source Type Selection -->
<div class="form-section">
<h4>📍 Source Type</h4>
<div class="radio-group">
<label class="radio-label">
<input type="radio" name="source_type" value="subreddit" checked>
<span class="radio-custom"></span>
Subreddit
</label>
<label class="radio-label">
<input type="radio" name="source_type" value="user">
<span class="radio-custom"></span>
User
</label>
</div>
</div>
<!-- Source Name Input -->
<div class="form-group">
<label for="sourceName" id="sourceNameLabel">Subreddit Name:</label>
<input type="text" id="sourceName" name="source_name" placeholder="e.g., python, machinelearning" required>
<small class="form-help" id="sourceNameHelp">Enter subreddit name without 'r/'</small>
</div>
<!-- Filter Options -->
<div class="form-row">
<div class="form-group">
<label for="limit">Limit:</label>
<input type="number" id="limit" name="limit" value="25" min="1" max="1000">
<small class="form-help">Max posts to process (1-1000)</small>
</div>
<div class="form-group">
<label for="sort">Sort by:</label>
<select id="sort" name="sort">
<option value="hot">Hot</option>
<option value="top" selected>Top</option>
<option value="new">New</option>
<option value="rising">Rising</option>
<option value="controversial">Controversial</option>
</select>
</div>
</div>
<div class="form-row">
<div class="form-group">
<label for="timeFilter">Time Filter:</label>
<select id="timeFilter" name="time_filter">
<option value="">All Time</option>
<option value="hour">Past Hour</option>
<option value="day">Past Day</option>
<option value="week">Past Week</option>
<option value="month">Past Month</option>
<option value="year">Past Year</option>
</select>
</div>
<div class="form-group">
<label for="minScore">Min Score:</label>
<input type="number" id="minScore" name="min_score" value="" placeholder="0">
<small class="form-help">Minimum upvotes</small>
</div>
</div>
<!-- Advanced Options -->
<div class="form-section">
<h4>⚙️ Advanced Options</h4>
<div class="checkbox-group">
<label class="checkbox-label">
<input type="checkbox" id="noDupes" name="no_dupes">
<span class="checkmark"></span>
Avoid Duplicates
</label>
<label class="checkbox-label" style="margin-left: 30px; font-size: 0.9em;">
<input type="checkbox" id="simpleCheck" name="simple_check">
<span class="checkmark"></span>
Use Simple Check (faster URL-based detection)
</label>
<label class="checkbox-label">
<input type="checkbox" id="makeHardLinks" name="make_hard_links">
<span class="checkmark"></span>
Create Hard Links
</label>
</div>
</div>
<input type="hidden" id="authState" name="auth_state" value="">
<button type="submit" class="btn btn-primary">🚀 Start Download</button>
</form>
</div>
</div>
</section>
<!-- Progress Section -->
<section class="progress-section">
<h2>📊 Download Progress</h2>
<div id="progressContainer" class="progress-container">
<div class="no-downloads">
<div class="empty-state">
<div class="empty-icon">📥</div>
<p>No active downloads</p>
<p>Start a download above to see progress here.</p>
</div>
</div>
</div>
<!-- Active Downloads List -->
<div id="downloadsList" class="downloads-list" style="display: none;">
<div class="downloads-header">
<h3>Active Downloads</h3>
</div>
<div id="downloadsItems" class="downloads-items"></div>
</div>
</section>
<!-- Status Section -->
<section class="status-section">
<div class="status-card">
<h3>System Status</h3>
<div class="status-item">
<span class="status-label">BDFR Status:</span>
<span id="bdfrStatus" class="status-value">Checking...</span>
</div>
<div class="status-item">
<span class="status-label">WebSocket:</span>
<span id="wsStatus" class="status-value">Disconnected</span>
</div>
</div>
</section>
</main>
<footer>
<p>&copy; 2024 BDFR Web Interface. Powered by FastAPI.</p>
</footer>
</div>
<script src="/static/js/app.js"></script>
</body>
</html>