ModerateWinGuy/BDFR_Web
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #704 from OMEGARAZER/pipx-support

Browse Source 
closes https://github.com/aliparlakci/bulk-downloader-for-reddit/issues/702
This commit is contained in:
Serene
2022-12-01 11:55:52 +10:00
committed by GitHub
parent ad12fc1b7a 7b7167643f
commit e18014cc8a
3 changed files with 79 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files

57
README.md
View File

@@ -1,4 +1,5 @@
# Bulk Downloader for Reddit # Bulk Downloader for Reddit
[![PyPI version](https://img.shields.io/pypi/v/bdfr.svg)](https://pypi.python.org/pypi/bdfr) [![PyPI version](https://img.shields.io/pypi/v/bdfr.svg)](https://pypi.python.org/pypi/bdfr)
[![PyPI downloads](https://img.shields.io/pypi/dm/bdfr)](https://pypi.python.org/pypi/bdfr) [![PyPI downloads](https://img.shields.io/pypi/dm/bdfr)](https://pypi.python.org/pypi/bdfr)
[![Python Test](https://github.com/aliparlakci/bulk-downloader-for-reddit/actions/workflows/test.yml/badge.svg?branch=master)](https://github.com/aliparlakci/bulk-downloader-for-reddit/actions/workflows/test.yml) [![Python Test](https://github.com/aliparlakci/bulk-downloader-for-reddit/actions/workflows/test.yml/badge.svg?branch=master)](https://github.com/aliparlakci/bulk-downloader-for-reddit/actions/workflows/test.yml)
@@ -10,19 +11,30 @@ If you wish to open an issue, please read [the guide on opening issues](docs/CON
Included in this README are a few example Bash tricks to get certain behaviour. For that, see [Common Command Tricks](#common-command-tricks). Included in this README are a few example Bash tricks to get certain behaviour. For that, see [Common Command Tricks](#common-command-tricks).
## Installation ## Installation
*Bulk Downloader for Reddit* needs Python version 3.9 or above. Please update Python before installation to meet the requirement. Then, you can install it as such:
*Bulk Downloader for Reddit* needs Python version 3.9 or above. Please update Python before installation to meet the requirement. Then, you can install it via pip with:
```bash ```bash
python3 -m pip install bdfr --upgrade python3 -m pip install bdfr --upgrade
``` ```
or via [pipx](https://pypa.github.io/pipx) with:
```bash
python3 -m pipx install bdfr --upgrade
```
**To update BDFR**, run the above command again after the installation. **To update BDFR**, run the above command again after the installation.
### AUR Package ### AUR Package
If on Arch Linux or derivative operating systems such as Manjaro, the BDFR can be installed through the AUR. If on Arch Linux or derivative operating systems such as Manjaro, the BDFR can be installed through the AUR.
- Latest Release: https://aur.archlinux.org/packages/python-bdfr/ - Latest Release: <https://aur.archlinux.org/packages/python-bdfr>
- Latest Development Build: https://aur.archlinux.org/packages/python-bdfr-git/ - Latest Development Build: <https://aur.archlinux.org/packages/python-bdfr-git>
### Source code ### Source code
If you want to use the source code or make contributions, refer to [CONTRIBUTING](docs/CONTRIBUTING.md#preparing-the-environment-for-development) If you want to use the source code or make contributions, refer to [CONTRIBUTING](docs/CONTRIBUTING.md#preparing-the-environment-for-development)
## Usage ## Usage
@@ -36,42 +48,47 @@ Note that the `clone` command is not a true, failthful clone of Reddit. It simpl
After installation, run the program from any directory as shown below: After installation, run the program from any directory as shown below:
```bash ```bash
python3 -m bdfr download bdfr download
``` ```
```bash ```bash
python3 -m bdfr archive bdfr archive
``` ```
```bash ```bash
python3 -m bdfr clone bdfr clone
``` ```
However, these commands are not enough. You should chain parameters in [Options](#options) according to your use case. Don't forget that some parameters can be provided multiple times. Some quick reference commands are: However, these commands are not enough. You should chain parameters in [Options](#options) according to your use case. Don't forget that some parameters can be provided multiple times. Some quick reference commands are:
```bash ```bash
python3 -m bdfr download ./path/to/output --subreddit Python -L 10 bdfr download ./path/to/output --subreddit Python -L 10
``` ```
```bash ```bash
python3 -m bdfr download ./path/to/output --user reddituser --submitted -L 100 bdfr download ./path/to/output --user reddituser --submitted -L 100
``` ```
```bash ```bash
python3 -m bdfr download ./path/to/output --user reddituser --submitted --all-comments --comment-context bdfr download ./path/to/output --user me --saved --authenticate -L 25 --file-scheme '{POSTID}'
``` ```
```bash ```bash
python3 -m bdfr download ./path/to/output --user me --saved --authenticate -L 25 --file-scheme '{POSTID}' bdfr download ./path/to/output --subreddit 'Python, all, mindustry' -L 10 --make-hard-links
``` ```
```bash ```bash
python3 -m bdfr download ./path/to/output --subreddit 'Python, all, mindustry' -L 10 --make-hard-links bdfr archive ./path/to/output --user reddituser --submitted --all-comments --comment-context
``` ```
```bash ```bash
python3 -m bdfr archive ./path/to/output --subreddit all --format yaml -L 500 --folder-scheme '' bdfr archive ./path/to/output --subreddit all --format yaml -L 500 --folder-scheme ''
``` ```
Alternatively, you can pass options through a YAML file. Alternatively, you can pass options through a YAML file.
```bash ```bash
python3 -m bdfr download ./path/to/output --opts my_opts.yaml bdfr download ./path/to/output --opts my_opts.yaml
``` ```
For example, running it with the following file For example, running it with the following file
@@ -87,8 +104,9 @@ subreddit:
``` ```
would be equilavent to (take note that in YAML there is `file_scheme` instead of `file-scheme`): would be equilavent to (take note that in YAML there is `file_scheme` instead of `file-scheme`):
```bash ```bash
python3 -m bdfr download ./path/to/output --skip mp4 --skip avi --file-scheme "{UPVOTES}_{REDDITOR}_{POSTID}_{DATE}" -L 10 -S top --subreddit EarthPorn --subreddit CityPorn bdfr download ./path/to/output --skip mp4 --skip avi --file-scheme "{UPVOTES}_{REDDITOR}_{POSTID}_{DATE}" -L 10 -S top --subreddit EarthPorn --subreddit CityPorn
``` ```
In case when the same option is specified both in the YAML file and in as a command line argument, the command line argument takes prs In case when the same option is specified both in the YAML file and in as a command line argument, the command line argument takes prs
@@ -237,7 +255,6 @@ The following options apply only to the `download` command. This command downloa
- `--max-score-ratio` - `--max-score-ratio`
- This skips all submissions which have higher than specified upvote ratio - This skips all submissions which have higher than specified upvote ratio
### Archiver Options ### Archiver Options
The following options are for the `archive` command specifically. The following options are for the `archive` command specifically.
@@ -266,7 +283,7 @@ Alternatively, you can use the command-line [xargs](https://en.wikipedia.org/wik
For a list of users `users.txt` (one user per line), type: For a list of users `users.txt` (one user per line), type:
```bash ```bash
cat users.txt | xargs -L 1 echo --user | xargs -L 50 python3 -m bdfr download <ARGS> cat users.txt | xargs -L 1 echo --user | xargs -L 50 bdfr download <ARGS>
``` ```
The part `-L 50` is to make sure that the character limit for a single line isn't exceeded, but may not be necessary. This can also be used to load subreddits from a file, simply exchange `--user` with `--subreddit` and so on. The part `-L 50` is to make sure that the character limit for a single line isn't exceeded, but may not be necessary. This can also be used to load subreddits from a file, simply exchange `--user` with `--subreddit` and so on.
@@ -360,12 +377,16 @@ The individual modules of the BDFR, used to download submissions from websites,
Modules can be disabled through the command line interface for the BDFR or more permanently in the configuration file via the `disabled_modules` option. The list of downloaders that can be disabled are the following. Note that they are case-insensitive. Modules can be disabled through the command line interface for the BDFR or more permanently in the configuration file via the `disabled_modules` option. The list of downloaders that can be disabled are the following. Note that they are case-insensitive.
- `Direct` - `Direct`
- `DelayForReddit`
- `Erome` - `Erome`
- `Gallery` (Reddit Image Galleries) - `Gallery` (Reddit Image Galleries)
- `Gfycat` - `Gfycat`
- `Imgur` - `Imgur`
- `PornHub`
- `Redgifs` - `Redgifs`
- `SelfPost` (Reddit Text Post) - `SelfPost` (Reddit Text Post)
- `Vidble`
- `VReddit` (Reddit Video Post)
- `Youtube` - `Youtube`
- `YoutubeDlFallback` - `YoutubeDlFallback`
@@ -394,6 +415,7 @@ The logfiles that the BDFR outputs are consistent and quite detailed and in a fo
## List of currently supported sources ## List of currently supported sources
- Direct links (links leading to a file) - Direct links (links leading to a file)
- Delay for Reddit
- Erome - Erome
- Gfycat - Gfycat
- Gif Delivery Network - Gif Delivery Network
@@ -402,8 +424,9 @@ The logfiles that the BDFR outputs are consistent and quite detailed and in a fo
- Reddit Text Posts - Reddit Text Posts
- Reddit Videos - Reddit Videos
- Redgifs - Redgifs
- Vidble
- YouTube - YouTube
- Streamable - Any source supported by [YT-DLP](https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites.md) should be compatable
## Contributing ## Contributing

2
docs/CONTRIBUTING.md
View File

@@ -51,7 +51,7 @@ $ python3 -m pip install -e .
Then, you can run the program from anywhere in your disk as such: Then, you can run the program from anywhere in your disk as such:
```bash ```bash
$ python3 -m bdfr bdfr
``` ```
## Style Guide ## Style Guide

6
setup.cfg
View File

@@ -4,7 +4,7 @@ description_file = README.md
description_content_type = text/markdown description_content_type = text/markdown
home_page = https://github.com/aliparlakci/bulk-downloader-for-reddit home_page = https://github.com/aliparlakci/bulk-downloader-for-reddit
keywords = reddit, download, archive keywords = reddit, download, archive
version = 2.6.0 version = 2.6.2
author = Ali Parlakci author = Ali Parlakci
author_email = parlakciali@gmail.com author_email = parlakciali@gmail.com
maintainer = Serene Arc maintainer = Serene Arc
@@ -20,3 +20,7 @@ platforms = any
[files] [files]
packages = bdfr packages = bdfr
[entry_points]
console_scripts =
bdfr = bdfr.__main__:cli