OMEGARAZER
83f45e7f60
Standardize shebang and coding declaration
...
Standardizes shebang and coding declarations.
Coding matches what's used by install tools such as pip(x).
Removes a few init files that were not needed.
2022-12-19 18:32:37 -05:00
OMEGARAZER
3aa740e979
Add soft fail on 5xx Prawcore errors.
2022-12-10 12:36:54 -05:00
Serene-Arc
0873a4a2b2
Format according to the black standard
2022-12-03 15:11:17 +10:00
OMEGARAZER
831f49daa6
Refurb linting
...
Lint with [refurb](https://github.com/dosisod/refurb ) using `--disable 126 --python-version 3.9`
2022-11-30 17:19:02 -05:00
Serene-Arc
5341d6f12c
Add catch for per-submission praw errors
2022-11-20 18:54:56 +10:00
OMEGARAZER
3c7f85725e
Narrow except
...
Narrow except to AttributeError
2022-11-08 12:06:20 -05:00
OMEGARAZER
dc5a9ef497
link_to depreciation coverage
...
Futureproof for link_to depreciation.
https://bugs.python.org/issue39950
2022-10-14 18:15:49 -04:00
Serene-Arc
2bbf1b644e
Change logging message
2022-07-23 17:24:05 +10:00
Serene-Arc
f22a8aec4d
Fix line length
2022-07-23 17:24:05 +10:00
Jacob Chapman
5d76fcd5aa
Update downloader.py
2022-07-23 17:24:05 +10:00
Jacob Chapman
4fc0d5dc1d
Add score filtering
2022-07-23 17:24:05 +10:00
dbanon87
1530456cf7
Update downloader.py
2021-11-29 09:23:04 -05:00
Serene-Arc
d0d72c8229
Add integration test for downloader option
2021-11-24 12:49:11 +10:00
Jay R. Wren
2b50ee0724
add test. fix typos.
2021-11-24 12:49:11 +10:00
Serene-Arc
7bca303b1b
Add in downloader parameters
2021-07-29 19:10:10 +10:00
Serene-Arc
3cdae99490
Implement callbacks for downloading
2021-07-27 13:39:49 +10:00
Serene-Arc
e5be624f1e
Check submission URL against filter before factory
2021-06-23 14:30:39 +10:00
Serene-Arc
7c27b7bf12
Update logging message
2021-06-13 09:49:42 +10:00
Serene-Arc
8ba2d0bb55
Add missing return statement
2021-06-10 18:59:22 +10:00
Serene
6dcef83666
Add ability to disable modules ( #434 )
...
* Fix test name to match standard
* Rename file
* Add ability to disable modules
* Update README
* Fix missing comma
* Fix more missing commas. sigh...
Co-authored-by: Ali Parlakçı <parlakciali@gmail.com >
2021-06-06 13:47:56 +03:00
Serene
434aeb8feb
Add a combined command for the archiver and downloader: clone ( #433 )
...
* Simplify downloader function
* Add basic scraper class
* Add "scrape" command
* Rename "scrape" command to "clone"
* Add integration tests for clone command
* Update README
* Fix failing test
2021-06-06 13:29:09 +03:00
Serene-Arc
79fba4ac4a
Fix indent
2021-05-31 13:42:41 +10:00
Serene-Arc
e2582ecb3e
Catch error with MacOS writing per issue #407
2021-05-23 12:17:14 +10:00
Serene-Arc
47a4951279
Rename variable
2021-05-23 12:13:44 +10:00
Serene-Arc
4395dd4646
Update logging messages to include submission IDs
2021-05-22 11:53:44 +10:00
Serene-Arc
a104a154fc
Simplify method structure
2021-05-22 11:53:44 +10:00
Ali Parlakci
da8c64ec51
Read files in chunks instead when hashing ( #416 )
2021-05-22 08:46:39 +10:00
Ali Parlakci
7c401b1461
Merge branch 'reddit_connector_refactor' of https://github.com/Serene-Arc/bulk-downloader-for-reddit into Serene-Arc-reddit_connector_refactor
2021-05-17 13:53:48 +03:00
Serene
c581bef790
Set file creation times to the post creation time ( #391 )
2021-05-17 13:49:35 +03:00
Serene-Arc
7016603763
Refactor out super class RedditConnector
2021-05-17 11:50:17 +10:00
Ali Parlakci
200916a150
Rename --exclude-id(-file) to --skip-id(-file)
2021-05-17 10:30:55 +10:00
Ali Parlakci
f768a7d61c
Rename --skip to --skip-format
2021-05-17 10:30:55 +10:00
Serene-Arc
7e70175e4c
Change logging message to include submission ID
2021-05-10 19:03:20 +10:00
Serene-Arc
afa3e2548f
Add customisable time formatting
2021-05-03 14:05:05 +10:00
Serene-Arc
eda12e5274
Make downloadfilter apply itself to Resources
2021-05-03 14:02:03 +10:00
Serene-Arc
711f8b0c76
Add exception for r/all in subreddit check
2021-05-02 14:00:23 +10:00
Serene-Arc
14195157de
Catch errors for banned or private subreddits
2021-05-01 13:36:38 +10:00
Serene-Arc
17499baf61
Add informative error when testing user existence
2021-04-28 12:03:28 +10:00
Serene-Arc
e6551bb797
Return banned users as not existing
2021-04-28 12:03:28 +10:00
Serene-Arc
db46676dec
Catch error when logfile accessed concurrently
2021-04-28 12:03:28 +10:00
Serene-Arc
cb41d4749a
Add option to specify logfile location
2021-04-28 12:03:28 +10:00
Serene-Arc
7438543f49
Remove unused variable
2021-04-28 12:03:28 +10:00
Serene-Arc
ca495a6677
Add missing typing declaration
2021-04-28 12:03:28 +10:00
Serene-Arc
214c883a10
Simplify regex string slightly
2021-04-28 12:03:28 +10:00
Serene
b37ff0714f
Fix time filters ( #279 )
2021-04-18 16:44:52 +03:00
Serene
d8752b15fa
Add option to skip specified subreddits ( #268 )
...
* Rename variables
* Add option to skip specific subreddits
* Update README
2021-04-18 16:44:52 +03:00
Serene-Arc
bd9f276acc
Rename module
2021-04-18 16:44:52 +03:00
