157 lines
5.2 KiB
Markdown
157 lines
5.2 KiB
Markdown
# BDFR Web Interface
|
|
|
|
A complete web-based interface for the Bulk Downloader for Reddit (BDFR) with real-time progress tracking and scheduled downloads.
|
|
|
|
This tool is a frontend for the BDFR tool found here: https://github.com/Serene-Arc/bulk-downloader-for-reddit
|
|
Although I've made a few changes to allow for better repeat download persistence to avoid redownloading previously downloaded content.
|
|
|
|
## Features
|
|
|
|
### 🎯 **Three Download Modes**
|
|
- **Download**: Media files only (images, videos, gifs) from Reddit posts
|
|
- **Archive**: Post metadata (title, author, comments) in JSON/XML format
|
|
- **Clone**: Complete backup combining media files and metadata
|
|
|
|
### 📍 **Multiple Source Types**
|
|
- **Subreddit Downloads**: Download content from any subreddit with customizable filters
|
|
- **User Downloads**: Download posts from specific users (submitted, upvoted, or saved content)
|
|
|
|
### ⚡ **Real-Time Progress Tracking**
|
|
- WebSocket-based live progress updates
|
|
- Visual progress bars and status indicators
|
|
- Download queue management with cancel functionality
|
|
- Phase tracking (downloading, processing, completing)
|
|
|
|
### 🔐 **Reddit OAuth2 Authentication**
|
|
- Secure authentication with Reddit API
|
|
- Access to private content (saved/upvoted posts)
|
|
- Session management and token refresh
|
|
|
|
### ⚙️ **Advanced Options**
|
|
- **Duplicate Detection**: Avoid re-downloading existing files
|
|
- **Hard Link Creation**: Save disk space with file links
|
|
- **Scheduled Downloads**: Set up daily automatic downloads
|
|
- **Content Filtering**: Filter by score, time range, and sort order
|
|
- **Simple Check**: Fast URL-based duplicate detection
|
|
|
|
### 📊 **Management Dashboard**
|
|
- Active download monitoring
|
|
- Scheduled task management
|
|
- System status indicators
|
|
- Download history and statistics
|
|
|
|
## Quick Start with Docker Compose
|
|
|
|
```yaml
|
|
version: '3.8'
|
|
services:
|
|
bdfr-web:
|
|
image: moderatewinguy/bdfr-web:latest
|
|
# Uncomment and set user if needed (optional for most setups)
|
|
# user: "1000:1000" # Set to your user's UID:GID (run 'id' to find yours)
|
|
ports:
|
|
- "8000:8000"
|
|
volumes:
|
|
- ./downloads:/app/downloads # Reddit content storage
|
|
- ./data:/app/data # Database persistence
|
|
environment:
|
|
- BDFR_CLIENT_ID=your_reddit_client_id
|
|
- BDFR_CLIENT_SECRET=your_reddit_client_secret
|
|
- BDFR_REDIRECT_URI=http://localhost:8000/auth/callback
|
|
```
|
|
|
|
Run with:
|
|
```bash
|
|
docker-compose up -d
|
|
```
|
|
|
|
Access at: http://localhost:8000
|
|
|
|
## Configuration
|
|
|
|
### User Permissions
|
|
|
|
The container is designed to run as any user, allowing it to match your system's permissions.
|
|
|
|
**For TrueNAS/NAS Systems:**
|
|
- Use the "Custom User" setting in your container configuration
|
|
- Set it to the UID of the user that owns your mounted shares
|
|
- The container will automatically run as that user
|
|
|
|
**For Docker Compose:**
|
|
If needed, uncomment the `user:` line in docker-compose.yml:
|
|
```yaml
|
|
user: "1000:1000" # Set to your user's UID:GID
|
|
```
|
|
|
|
Find your UID/GID with: `id` (Linux/Mac) or `wsl id` (Windows WSL)
|
|
|
|
**The container will work with any UID/GID** - just ensure the mounted volumes are owned by the same user.
|
|
|
|
### Environment Variables
|
|
|
|
| Variable | Description | Required |
|
|
|----------|-------------|----------|
|
|
| `BDFR_CLIENT_ID` | Reddit OAuth2 client ID | Yes |
|
|
| `BDFR_CLIENT_SECRET` | Reddit OAuth2 client secret | Yes |
|
|
| `BDFR_REDIRECT_URI` | OAuth2 redirect URI | Yes |
|
|
| `HOST` | Server host (default: 0.0.0.0) | No |
|
|
| `PORT` | Server port (default: 8000) | No |
|
|
| `DEBUG` | Enable debug mode | No |
|
|
|
|
## Volume Mounts
|
|
|
|
- **`/app/downloads`**: All Reddit content is stored here
|
|
- **`/app/data`**: SQLite databases for scheduled tasks and application data
|
|
|
|
## API Endpoints
|
|
|
|
- `GET /` - Web interface
|
|
- `POST /api/download/subreddit` - Start subreddit download
|
|
- `POST /api/download/user` - Start user download
|
|
- `GET /api/downloads` - List active downloads
|
|
- `DELETE /api/downloads/{id}` - Cancel download
|
|
- `GET /health` - Health check
|
|
- `WS /ws/progress` - Real-time progress updates
|
|
|
|
## Getting Reddit OAuth Credentials
|
|
|
|
1. Go to https://www.reddit.com/prefs/apps
|
|
2. Click "Create App" or "Create Another App"
|
|
3. Select "web app" and fill in the details
|
|
4. Set redirect URI to: `http://localhost:8000/auth/callback`
|
|
5. Copy the client ID and client secret to your `.env` file
|
|
|
|
## Use Cases
|
|
|
|
- **Content Archiving**: Save Reddit posts and media for offline access
|
|
- **Research Data Collection**: Gather post metadata for analysis
|
|
- **Media Backup**: Download images/videos from favorite subreddits
|
|
- **Automated Downloads**: Schedule daily content collection
|
|
- **Personal Archive**: Backup your own posts and saved content
|
|
|
|
## Supported Content Sources
|
|
|
|
- Direct image/video links
|
|
- Imgur albums and images
|
|
- Reddit native media (images, videos, gifs)
|
|
- YouTube videos
|
|
- Gfycat animations
|
|
- And many more via yt-dlp fallback
|
|
|
|
## Technical Details
|
|
|
|
- **Base Image**: Python 3.11-slim
|
|
- **Web Framework**: FastAPI with WebSocket support
|
|
- **Frontend**: Vanilla JavaScript with modern CSS
|
|
- **Database**: SQLite for task scheduling
|
|
- **Authentication**: OAuth2 with Reddit API
|
|
- **File Processing**: Multi-threaded downloads with progress tracking
|
|
|
|
|
|
---
|
|
|
|
**Note**: This image includes the complete BDFR tool, so you can also run CLI commands directly:
|
|
```bash
|
|
docker exec bdfr-web bdfr download /app/downloads --subreddit python -L 50
|
|
``` |