Commit Graph

74 Commits

Author SHA1 Message Date
Anna-Maria Meriniemi e545f8ae19
Introduce baked queries (#592)
SQA's baked queries prepares the queries in advance, caching yada yada.
Makes thing a bit faster.
Also bigger speedup included is a shoddy cache for the total torrent
count (only applied to baked queries currently). Caching the value for a
few dozen seconds shaves off some wasted time, as it's mostly just used
for pagination.
2020-07-12 00:14:06 -07:00
queue 4fcef92b94
elasticsearch 7.x compatability (#576)
* es_mapping: update turning off dynamic mappings

they changed it in 6.x

https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic.html
https://github.com/elastic/elasticsearch/pull/25734

* es_mapping: remove _all field

deprecated in 6.0 anyway

* es_mapping.yml: fix deprecated mapping type

https://www.elastic.co/guide/en/elasticsearch/reference/6.7/removal-of-types.html#_schedule_for_removal_of_mapping_types

it gives a really unhelpful error otherwise, oof.

* es: fix remaining 7.xisms

the enabled: false apparently only applies to
"object" fields now, need index: false

and the _type got removed everywhere. Seems to work now.

* Fix weird offset error with word_delimiter_graph

yet another es7-ism i guess

* Fix warning and some app stuff for ES 7.x

Co-authored-by: Arylide <Arylide@users.noreply.github.com>
2020-07-12 00:10:47 -07:00
Nicolas F 5c943f35e3 Add login endpoint rate limiting
This doesn't discriminate between failed logins and successful
logins, but only counts POST requests. The limit is set to 6 per
hour.
2020-06-23 00:49:26 +02:00
Nicolas F 5da7635164 account: add IP rate limiting for account creation
Add a configurable cooldown in seconds for which no further accounts
can be created from the same IP.
2019-08-26 10:10:11 +02:00
A nyaa developer 532439356f nuke comments (#577)
* nuke comments

* improve update_comment_count_db
2019-08-13 13:15:16 -07:00
Nicolas F d8e796f3e0 Move to Python 3.7 and update dependencies (#551)
* forms: replace re._pattern_type with re.Pattern

Python 3.7 removed re._pattern_type and replaced it with
re.Pattern.

* readme: update for Python 3.7

* Update requirements

Also remove some unused ones which were neither a direct dependency
nor a dependency of our dependencies.

* account: force ASCII usernames on login form

Our database doesn't like it when we check for unicode data in
a column that stores ASCII data, so let's stop it before it
gets that far.

* Move travis CI to Python 3.7

* travis: use xenial dist

* fix newer linter warnings

Apparently bare excepts are literally Hitler, and we have some
new import sorting rules. Hooray!

* requirements: remove six

This is a dependency for sqlalchemy-utils, but we ourselves don't
depend on it directly because we've never been on Python 2 ever.

* Update requirements.txt
2019-08-10 18:39:53 -07:00
Nicolas F 16814d6eb7 Add trusted application functionality (#533)
* Add trusted application functionality

This lets users apply for trusted status, given certain minimum
requirements. Moderators can then review the applications, giving
a recommendation, and administrators can accept or reject them.

If an application is accepted or rejected, the user receives an
e-mail about it.

Markdown images are not rendered in applications to prevent browsers
from sending automatic requests to untrusted webservers.

Users who have had their application rejected cannot re-apply for a set
amount of days.

* minor fixes
2019-08-10 18:18:44 -07:00
TheAMM a7558823d0 Further cleanup 2019-04-08 19:18:11 +02:00
Alex Ingram 75e7e942fb Fix some JOIN insanity with SQLAlchemy. 2019-04-08 19:18:11 +02:00
TheAMM d7f9618fbf upload/edit: sanitize some string fields
This commit introduces a regex to replace illegal (expectedly unused)
characters from torrent display name, information link and description
upon upload or edit.

Fixes #541
2019-02-25 12:27:09 +01:00
Anna-Maria Meriniemi 8365894268 config: add anonymous upload/registration limiting (#537) 2019-01-02 03:40:56 -08:00
doge 3190394eea Comment Hiding
This commit adds the ability hide comments on torrent pages, and a user
preferences tab to allow registered users to make this the default
behavior.
2018-11-20 13:45:31 +01:00
TheAMM b2ddba994c Fix open redirect (#519)
The funny thing is that we don't even use this anywhere, and the
referrer is useless on forms. But hey, maybe someday.
2018-09-16 21:16:35 +03:00
Nicolas F e892f358e8 users: activation button improvements (#506)
Now uses flask.flash to give the person who clicks it feedback
when it's clicked, because caching may make things confusing.

Also only activate a user if they're inactive, as again, caching
can lead to staff pressing the button multiple times in a row,
leading to unnecessary log messages.
2018-07-09 22:15:19 -07:00
Nicolas F 8644472533 Add registration IP (#507)
This will keep track of which IP a user has registered from, to
prevent evading rangebans. It will only be shown to admins.
2018-07-09 22:14:45 -07:00
Nicolas F a38e5d5b53 Implement range bans (#478)
* Implement range bans

People connecting from banned IP ranges are unable to upload
torrents anonymously, and need to manually have their accounts
activated.

This adds a new table "rangebans", and a command line utility,
"rangeban.py", which can be used to add, list and remove rangebans
from the command line.

As an example:

./rangeban.py ban 192.168.0.0/24

This would rangeban anything in this /24.

The temporary_tor column allows automated scripts to clean out and
re-add ever-changing sets of ranges to be banned without affecting
the other ranges.

This has only been tested for IPv4.

* Revise Rangebans

Add an id column, and change "temporary_tor" to "temp". Also
index masked_cidr and mask.

* rangebans: fix enabled and the binary op

kill me

* Add enabling/disabling bans to rangeban.py

* rangebans: fail earlier on garbage arguments

* rangebans: fix linter errors

* rangeban.py: don't shadow builtin keyword 'id'

* rangebans: change temporary ban logic, column

The 'temp' column is now a nullable time column. If the field is
null, the ban is understood to be permanent. If there is a time
in there, it's understood to be the creation time of the ban.

This allows scripts to e.g. delete all temporary bans older than
a certain amount of time.

Also, rename the '_cidr_string' column to 'cidr_string', because
reasons.

* rangeban.py: use ip_address to parse CIDR subnet

* rangebans: fixes to the mask calculation and query

Both were not bugs per-se, but just technically not needed/correct.

* De-meme apparently
2018-06-29 20:15:04 -07:00
Edward Betts d407f09cab Correct spelling mistakes. (#495) 2018-05-28 04:54:54 -07:00
Nicolas F bb9a62f71b user page: add manual activation button for mods (#472)
* user page: add manual activation button for mods

Moderators can press this button on inactive users to manually
activate their accounts.

Furthermore, the admin form code has been refactored a bit, reducing
some code duplication.
2018-05-10 18:57:59 -07:00
Arylide 03094b6d36 Commit editing time 2018-04-02 13:18:39 -07:00
nyaadev f1bab93a94 fix two bugs and a minor issue 2018-04-02 22:06:41 +02:00
Nicolas F 60ce4ec3f1 Implement comment locking (#439)
* Implement comment locking

This adds a new flags to torrents, which is only editable by
moderators and admins. If checked, it does not allow unprivileged
users to post, edit or delete comments on that torrent.

* Rename "locked" to "comment_locked".

* Shorter button and additional words on alt text

* Admin log: Change comment locking message

dude I love bikeshedding xd

* Bikeshedding over admin log messages

* >&
Also some bikeshedding
2018-03-25 17:03:49 -07:00
Nicolas F c405f49eb6 Redo nuke functionality (#459)
This started out as a simple rebase, but then I rebased the wrong
branches and it all got confusing, so here it is as a new dank
commit.

We now have an @admin_only decorator, and we ask for confirmation
before we nuke. We can also see the nuke button when users are
banned, and nuking is a separate endpoint with a separate form.

Additionally, it now uses the new tracker API.
2018-03-25 16:24:44 -07:00
Arylide 0b98b2454a New help section for IRC and some prod changes I never put in the repo because lazy. 2018-02-22 23:23:53 -08:00
nyaadev 8de2663fc2 Remove deprecated torrent delete code. 2018-02-16 19:58:31 +01:00
A nyaa developer d7b413e4d7 site-specific changes for new tracker (#453) 2018-02-12 15:52:35 -08:00
Nicolas F 7bef642f4e Don't submit reports for already banned torrents (#448)
If users kept their page open for a while before reporting a
torrent, and mods got it in the meantime, users could still
submit reports for that torrent. This is silly and really doesn't
need to happen.
2018-02-08 12:12:54 -08:00
nyaadev 658eefe42a fix uncommon exception in report system
fix html style issue in admin box on user page
2018-02-06 23:05:37 +01:00
nyaadev f2411db485 fix migration 2018-02-02 20:53:46 +01:00
nyaadev d151cca4ef fix last commit 2018-02-02 20:39:02 +01:00
TheAMM a92d886b5c Name fixes, DRY 2018-02-02 20:39:02 +01:00
nyaadev fd0a02b95c Move bencoded info dicts from mysql torrent_info table to info_dict directory. DB change!
IMPORTANT!!! Make sure to run utils/infodict_mysql2file.py before upgrading the database.
2018-02-02 20:39:02 +01:00
TheAMM d4fcd36b1b Fix mods' comment list filter 2018-01-04 00:34:17 +02:00
Anna-Maria Meriniemi 3941a0b9b3
Quick and dirty comment list for moderators to look at (#421) 2017-12-04 15:51:31 +02:00
TheAMM 4cdf7f4ab3 Support searching for base32 info hash (BTIH)
"BitTorrent info hashes" are generally found in magnet uris.
An info hash is 40 characters in hex and 32 in base32 so the searches won't clash.
2017-11-14 21:27:15 +02:00
Shane 72c997173c [Schema+config change] Comment editing (#396)
* Comment editing
* Optional time limit for comment editing
2017-11-05 16:26:30 +02:00
TheAMM b4c0ad9e84 torrents.py: Add link to torrent page into .torrent comment metadata 2017-11-04 23:46:50 +02:00
Nicolas F 96468a5a11 torrents: don't allow download of deleted torrents (#399)
Only allow moderators to download torrent files that are deleted.
2017-11-03 18:54:08 +02:00
Nicolas F 7095567b29 Comment deletion: fix small lack of validation (#395)
People could delete their own comments on torrents other than
the one the URL would indicate, which meant they could mess with
the total comment count on a torrent by having it be higher than
it actually is through repeatedly posting and deleting comments
that way.

However, they could only ever delete their own comments, so this
isn't a huge issue in the first place.
2017-10-29 20:01:19 +02:00
Nicolas F 4019343d50 Implement torrent nuking ability for mods (#377)
* Implement torrent nuking ability for mods

This deletes all torrents of a specific user.
A current caveat is that it will delete both sukebei and nyaa torrents,
but will only leave a log entry in the current flavour's log.

Also did some bootstrap untangling on the user view page.

* Per-flavour logging

Hopefully this works. Maybe.

* Tracker API: chunk into 100-element sublists

* isort

* Restrict nuking to superadmins

Also do a lint.sh.
2017-10-16 18:17:12 -07:00
Anna-Maria Meriniemi de1fd2f1bc [Config change] Upload ratelimit for non-trusted uploaders (#384)
* Implement upload ratelimit for non-trusted uploaders

Users may upload X torrents in Y minutes after which they
will have to wait Z minutes between uploads.

* Show torrent period count when ratelimited

* Only ratelimit new accounts
2017-10-09 18:41:18 -07:00
Nicolas F 37546354a7 Explicitly tell users they were banned (#379)
This tells users who are banned the reason that they are banned,
and doesn't show the same message for inactive users.

IP banned users are still just shown the boring 403 page.
2017-10-07 19:16:24 -07:00
Anna-Maria Meriniemi 9e87e810af [Config change] Password reset by email (#381)
* Password reset by email

Adds endpoint, templates, email templates, forms

* Timeout password reset request in six hours
2017-10-08 04:34:40 +03:00
Anna-Maria Meriniemi 6d09920abd Update email verification, add Mailgun backend (#380)
Changes config.example.py!
2017-10-07 17:31:32 -07:00
Nicolas F 214952e7b6 Disable report button on empty reason (#374)
Previously, people couldn't quite tell you needed to give a report
reason. Now we disable the submit button until there is a reason,
and flask.flash() if someone manages to submit an empty reason
anyway.
2017-10-03 17:03:22 -07:00
Anna-Maria Meriniemi c5d705210d Read-only maintenance mode setting for config.py (#356)
Disables all POSTs, optionally allowing users to log in (without updating last login date)
Blocked POSTs will redirect to the GET endpoint if possible, otherwise to referrer or in last case, home page.
API requests will get a plaintext message with 405 status code.
2017-09-04 18:16:52 -04:00
nyaadev 57a71be7b8 minor changes 2017-08-31 14:20:34 +02:00
Anna-Maria Meriniemi 4eed4971a3 Fix banning torrents without uploader ip
Pretty much just for archived torrents
2017-08-26 02:52:59 +03:00
Anna-Maria Meriniemi aa3c2ba77b Fix banning anonymous torrents
We miss ye, testing
2017-08-26 02:36:38 +03:00
A nyaa developer f8a314df4f Better bans (#341)
* better bans

* put jinja2 template into correct file
2017-08-25 18:53:35 -04:00
Anna-Maria Meriniemi 39fcfc0058 Add (optional) validation for minimum anonymous torrent size (#342)
MINIMUM_ANONYMOUS_TORRENT_SIZE can be used to require a minimum total
size of torrents uploaded by anonymous users (ie. without accounts).

Sets up a "framework" for post-WTForm torrent validation as well;
this can easily be extended into filename blacklists and such.
2017-08-19 20:48:08 -04:00