BDFR_Web/bdfr/API_README.md

# BDFR API Layer

The BDFR API layer provides a direct integration interface for the web interface, eliminating the need for subprocess console parsing. It wraps the existing BDFR core classes and provides a clean API with structured progress callbacks.

## Overview

The API layer consists of several key components:

- **BDFRManager**: Main API interface class
- **ProgressCallback**: Abstract base class for progress notifications
- **ProgressEvent**: Structured progress event data
- **DownloadType**: Enumeration of supported download types
- **DownloadStatus**: Enumeration of download statuses

## Quick Start

### Basic Usage

```python
from bdfr.api import BDFRManager, LoggingCallback

# Create a manager
manager = BDFRManager("./downloads")

# Download from a subreddit
download_id = manager.download_subreddit(
    "python",
    limit=50,
    sort="hot",
    no_dupes=True
)

# Check status
status = manager.get_download_status(download_id)
print(f"Progress: {status['progress']}%")
```

### Advanced Usage with Custom Callbacks

```python
from bdfr.api import BDFRManager, ProgressCallback, ProgressEvent

class WebSocketCallback(ProgressCallback):
    def __init__(self, websocket):
        self.websocket = websocket

    async def on_progress(self, event: ProgressEvent):
        await self.websocket.send_json(event.to_dict())

    async def on_error(self, event: ProgressEvent):
        await self.websocket.send_json(event.to_dict())

    async def on_completed(self, event: ProgressEvent):
        await self.websocket.send_json(event.to_dict())

# Use custom callbacks
callbacks = [LoggingCallback(), WebSocketCallback(ws)]
download_id = manager.download_subreddit("technology", limit=100, progress_callbacks=callbacks)
```

## API Reference

### BDFRManager

The main API class that manages downloads and provides the primary interface.

#### Constructor

```python
BDFRManager(download_directory: Optional[Union[str, Path]] = None)
```

- `download_directory`: Base directory for downloads. Defaults to current directory.

#### Methods

##### `create_download()`

Create a new download operation.

```python
create_download(
    download_type: DownloadType,
    name: str,
    config: Optional[Configuration] = None,
    progress_callbacks: Optional[List[ProgressCallback]] = None
) -> str
```

Returns a download ID for tracking the operation.

##### `start_download()`

Start a download operation.

```python
start_download(download_id: str) -> bool
```

Returns `True` if started successfully.

##### `get_download_status()`

Get the current status of a download.

```python
get_download_status(download_id: str) -> Optional[Dict[str, Any]]
```

Returns download status information or `None` if not found.

##### `cancel_download()`

Cancel a running download.

```python
cancel_download(download_id: str) -> bool
```

Returns `True` if cancelled successfully.

##### `list_downloads()`

List all active downloads.

```python
list_downloads() -> Dict[str, Dict[str, Any]]
```

Returns dictionary of download information keyed by download ID.

##### `cleanup_completed()`

Clean up old completed downloads.

```python
cleanup_completed(max_age_seconds: int = 3600) -> int
```

Returns number of downloads cleaned up.

#### Convenience Methods

##### `download_subreddit()`

Download content from a subreddit.

```python
download_subreddit(
    subreddit_name: str,
    limit: Optional[int] = None,
    sort: str = "hot",
    time_filter: str = "all",
    no_dupes: bool = False,
    progress_callbacks: Optional[List[ProgressCallback]] = None
) -> str
```

##### `download_user()`

Download content from a user.

```python
download_user(
    username: str,
    limit: Optional[int] = None,
    submitted: bool = True,
    upvoted: bool = False,
    saved: bool = False,
    no_dupes: bool = False,
    progress_callbacks: Optional[List[ProgressCallback]] = None
) -> str
```

##### `archive_subreddit()`

Archive subreddit data (metadata only).

```python
archive_subreddit(
    subreddit_name: str,
    format_type: str = "json",
    limit: Optional[int] = None,
    progress_callbacks: Optional[List[ProgressCallback]] = None
) -> str
```

##### `clone_subreddit()`

Clone subreddit (both download and archive).

```python
clone_subreddit(
    subreddit_name: str,
    limit: Optional[int] = None,
    format_type: str = "json",
    no_dupes: bool = False,
    progress_callbacks: Optional[List[ProgressCallback]] = None
) -> str
```

### ProgressCallback

Abstract base class for implementing progress callbacks.

#### Methods

##### `on_progress(event: ProgressEvent)`

Called when progress is made.

##### `on_error(event: ProgressEvent)`

Called when an error occurs.

##### `on_completed(event: ProgressEvent)`

Called when download is completed.

### ProgressEvent

Represents a progress event with structured data.

#### Attributes

- `event_type`: "progress", "status", "error", or "completed"
- `download_id`: Unique identifier for the download
- `message`: Human-readable message
- `progress`: Progress percentage (0-100)
- `data`: Additional structured data
- `timestamp`: When the event occurred

#### Methods

##### `to_dict() -> Dict[str, Any]`

Convert to dictionary for JSON serialization.

### DownloadType

Enumeration of supported download types:

- `SUBREDDIT`: Download from subreddit
- `USER`: Download from user
- `MULTIREDDIT`: Download from multireddit
- `SUBMISSIONS`: Download specific submissions
- `ARCHIVE`: Archive only (no downloads)
- `CLONE`: Both download and archive

### DownloadStatus

Enumeration of download statuses:

- `QUEUED`: Download is queued but not started
- `RUNNING`: Download is in progress
- `COMPLETED`: Download completed successfully
- `FAILED`: Download failed with an error
- `CANCELLED`: Download was cancelled
- `PAUSED`: Download is paused

## Configuration

The API uses the existing BDFR `Configuration` class. You can pass a custom configuration to `create_download()` or use the convenience methods with their specific parameters.

### Common Configuration Options

- `limit`: Maximum number of posts to process
- `sort`: Sort method (hot, top, new, controversial, rising)
- `time`: Time filter (all, hour, day, week, month, year)
- `no_dupes`: Avoid duplicate downloads
- `make_hard_links`: Create hard links for duplicates
- `format`: Archive format (json, xml, yaml)

## Web Interface Integration

### FastAPI Integration Example

```python
from fastapi import FastAPI, WebSocket
from bdfr.api import get_bdfr_manager, WebSocketCallback

app = FastAPI()
manager = get_bdfr_manager("./downloads")

@app.post("/api/download/subreddit")
async def download_subreddit(subreddit: str, limit: int = 10):
    download_id = manager.download_subreddit(subreddit, limit=limit)
    return {"download_id": download_id}

@app.websocket("/ws/progress/{download_id}")
async def progress_websocket(websocket: WebSocket, download_id: str):
    await websocket.accept()

    class FastAPICallback(WebSocketCallback):
        def __init__(self):
            super().__init__(None)

        async def on_progress(self, event: ProgressEvent):
            if event.download_id == download_id:
                await websocket.send_json(event.to_dict())

    # Add callback to existing download or create new one
    # (Implementation depends on your specific needs)
```

### Real-time Progress Updates

The API provides structured progress events that can be easily serialized to JSON for web clients:

```python
# Example progress event
{
    "event_type": "progress",
    "download_id": "123e4567-e89b-12d3-a456-426614174000",
    "message": "Downloaded submission abc123 from r/python",
    "progress": 45.2,
    "data": {
        "items_processed": 12,
        "items_found": 25,
        "current_item": "abc123",
        "phase": "downloading_submission"
    },
    "timestamp": "2023-12-07T10:30:45.123456"
}
```

## Error Handling

The API provides comprehensive error handling:

- **Download errors**: Network issues, authentication problems, etc.
- **Configuration errors**: Invalid parameters, missing files, etc.
- **System errors**: Disk space, permissions, etc.

All errors are captured and reported through the progress callback system with detailed error information.

## Threading and Concurrency

The API is designed to be thread-safe and supports concurrent downloads:

- Each download runs in its own thread
- Progress callbacks are async-safe
- Multiple downloads can run simultaneously
- Thread-safe status tracking

## Logging

The API integrates with Python's logging system:

```python
import logging

# Configure logging
logging.basicConfig(level=logging.INFO)

# Use LoggingCallback for automatic log output
callbacks = [LoggingCallback("my_app")]
```

## Examples

See `bdfr/examples/api_usage.py` for comprehensive examples including:

- Basic usage
- Custom callbacks
- Web integration
- Error handling
- User downloads
- Archive operations

## Migration from Subprocess

### Before (Subprocess)

```python
import subprocess
import json

# Start BDFR via subprocess
proc = subprocess.Popen([
    "python", "-m", "bdfr", "download",
    "--subreddit", "python",
    "--limit", "50",
    "./downloads"
], stdout=subprocess.PIPE, stderr=subprocess.PIPE)

# Parse console output for progress
while True:
    line = proc.stdout.readline().decode().strip()
    if not line:
        break
    # Parse progress from console output...
```

### After (API)

```python
from bdfr.api import BDFRManager, LoggingCallback

# Use API directly
manager = BDFRManager("./downloads")
download_id = manager.download_subreddit("python", limit=50)

# Get structured progress updates
status = manager.get_download_status(download_id)
print(f"Progress: {status['progress']}%")
```

## Benefits

1. **No subprocess overhead**: Direct integration with BDFR core
2. **Structured progress**: Rich progress events instead of console parsing
3. **Better error handling**: Detailed error information and stack traces
4. **Thread-safe**: Concurrent downloads with proper synchronization
5. **Web-friendly**: JSON-serializable progress events
6. **Extensible**: Custom progress callbacks for different integrations
7. **Maintainable**: Clean separation of concerns

## Requirements

- Python 3.7+
- Existing BDFR installation
- Dependencies: `praw`, `requests`, and other BDFR dependencies

## Installation

The API layer is included with BDFR and requires no additional installation. Simply import and use:

```python
from bdfr.api import BDFRManager