Commit Graph

521 Commits

Author SHA1 Message Date
Anna-Maria Meriniemi bc1901baa5 ES: implement real substring matching (#500)
...by splitting input into characters, instead of whitespace delimited
words. This means you can now match partial words, real substrings from
anywhere: "foo ba" will match "Foo Bar Baz", while previously you had to
have full words ("foo bar") to match anything.

My dev setup incurred an 8% increase in storage usage, from ~13MB to
~14MB (for ~40k torrents).
Small change, big improvement. Wonder why I didn't do this at first.
2018-06-08 00:59:19 -07:00
Edward Betts d407f09cab Correct spelling mistakes. (#495) 2018-05-28 04:54:54 -07:00
Kcchouette b999f8d39f Fix availiable → available (#491) 2018-05-14 02:36:26 -07:00
Nicolas F bb9a62f71b user page: add manual activation button for mods (#472)
* user page: add manual activation button for mods

Moderators can press this button on inactive users to manually
activate their accounts.

Furthermore, the admin form code has been refactored a bit, reducing
some code duplication.
2018-05-10 18:57:59 -07:00
Alex Ingram 8f4202c098 Improve mobile user experience. 2018-04-24 23:19:14 -05:00
Alex Ingram 8498ff8e62 Fix torrent index on narrow viewports. (#484)
people should get bigger monitors
2018-04-23 15:47:18 -07:00
Anna-Maria Meriniemi 0cc25c3569
[ES] Improve search term preprocessing to include literal groups (#477)
* Extend ES term preprocessing for OR groups

Implements handling "foo"|"bar" literal OR groups in the Elasticsearch
term preprocessor. Groups can be negated with -, but don't mesh with
precedence (like plain literals).

This is a partial hack, the real solution would be to parse the entire
search terms ourselves, with AND and OR groups, negations etc. But
having that work neatly with the simple_query_string would be bit of a
hassle.

* Update help.html search tips

since search (quoting strings) has changed a bit.
2018-04-15 09:53:36 +03:00
Anna-Maria Meriniemi 0b78428abc [ES Change] Improve Elasticsearch term quoting (#473)
* Optimize Elasticsearch fullword field

Since the main display_name field ngrams words up to 15 characters,
anything to and under that will already be indexed - the fullword field
(which we have for words longer than 15 characters) needs to index only
words longer than that.

* Preprocess ES terms for better literal matching

This commit adds a new .exact subfield to display_name, which holds a
barely-filtered version of the original title we can do "literal"
matching against. This is not real substring matching, but quoting
terms now actually does something!

Implements a simple preprocessor for the search terms to extract quoted
parts from the search terms, optionally prefixed with - to negate them.
The preprocessor will create a query that'll join all three query-types:
the simple_query_string, must-phrases and must-not-phrases.
2018-04-13 17:06:25 -07:00
nyaadev 8f9400bb5f Revert "[Schema change] Torrents flags bitflag column to indexed columns (#471)"
This reverts commit 41a2a32f66.

Performs worse in some cases than what we had before.
2018-04-08 08:36:42 +02:00
A nyaa developer 41a2a32f66 [Schema change] Torrents flags bitflag column to indexed columns (#471)
* convert torrent table flags column from bitflag to independent indexed columns

* elasticsearch integration (untested)

* improve performance
2018-04-07 22:44:53 -07:00
Nicolas F c786bd20f8 Fix nuke button prompt (#469)
Hitting the cancel button does not return "", but null. Therefore
the toLowerCase() fails, and throwing an exception means "sure go
ahead submitting this" to JS for some godforsaken reason.

Just remove the toLowerCase for now, have people type the names
properly.
2018-04-04 23:32:04 +02:00
Nicolas F 291f859a4f Use Flask-Assets to minify self-hosted JS files (#468)
* Use Flask-Assets to minify self-hosted JS files

By having Flask-Assets minify the two JS files we ship, namely
main.js and bootstrap-select.js, we can shave off 28406 bytes.

The minified files are generated on startup. If one wishes to
manually clean them up or build them, they can use the
"flask assets" management command, e.g. "flask assets clean".

* Workaround to fix tests

State carries over in tests, which is the dumbest shit ever. Fix it
by clearing the bundles before setting them.
2018-04-04 16:02:05 +02:00
Arylide 03094b6d36 Commit editing time 2018-04-02 13:18:39 -07:00
nyaadev f1bab93a94 fix two bugs and a minor issue 2018-04-02 22:06:41 +02:00
Nicolas F 60ce4ec3f1 Implement comment locking (#439)
* Implement comment locking

This adds a new flags to torrents, which is only editable by
moderators and admins. If checked, it does not allow unprivileged
users to post, edit or delete comments on that torrent.

* Rename "locked" to "comment_locked".

* Shorter button and additional words on alt text

* Admin log: Change comment locking message

dude I love bikeshedding xd

* Bikeshedding over admin log messages

* >&
Also some bikeshedding
2018-03-25 17:03:49 -07:00
Nicolas F 2b5f9922e9 Add rel="prev"/"next" attribs on pagination (#462) 2018-03-25 16:32:03 -07:00
Nicolas F ad5ea6d91e Use rel attributes on links in the info field (#463) 2018-03-25 16:30:57 -07:00
Nicolas F e9b1f6a6c4 Add some rel attributes to links inside markdown (#461)
I currently don't differentiate between "trusted" markdown and
untrusted, but this should be good enough. Basically tells the
browser not to send a referrer, and (not sure if relevant here)
not to expose a window opener object. Also tells search engines
that the link is not endorsed with "nofollow".
2018-03-25 16:30:07 -07:00
Nicolas F dd4510f371 Fix user page ban list styling for multiple bans (#460)
Change each ban to be a bullet point in an unordered list.
2018-03-25 16:27:28 -07:00
Nicolas F c405f49eb6 Redo nuke functionality (#459)
This started out as a simple rebase, but then I rebased the wrong
branches and it all got confusing, so here it is as a new dank
commit.

We now have an @admin_only decorator, and we ask for confirmation
before we nuke. We can also see the nuke button when users are
banned, and nuking is a separate endpoint with a separate form.

Additionally, it now uses the new tracker API.
2018-03-25 16:24:44 -07:00
Arylide 0b98b2454a New help section for IRC and some prod changes I never put in the repo because lazy. 2018-02-22 23:23:53 -08:00
nyaadev 8de2663fc2 Remove deprecated torrent delete code. 2018-02-16 19:58:31 +01:00
A nyaa developer d7b413e4d7 site-specific changes for new tracker (#453) 2018-02-12 15:52:35 -08:00
Nicolas F 7bef642f4e Don't submit reports for already banned torrents (#448)
If users kept their page open for a while before reporting a
torrent, and mods got it in the meantime, users could still
submit reports for that torrent. This is silly and really doesn't
need to happen.
2018-02-08 12:12:54 -08:00
nyaadev 658eefe42a fix uncommon exception in report system
fix html style issue in admin box on user page
2018-02-06 23:05:37 +01:00
Anna-Maria Meriniemi e5fe63156d Fix flat PR (#446)
* Clean up PR #349 

- Rely on os.makedirs(..., exist_ok=True) for "thread"-safety

- Remove the previous info_dict when we know the transaction went through.

- bytes.hex() will always be lowercase (unless we go off CPython):
c3d9508ff2/Python/pystrhex.c (L5-L49)
c3d9508ff2/Python/codecs.c (L16)

- Reintroduce comments and meaningful creation dates in generated torrents:
Also make create_default_metadata_base set the correct metadata now
2018-02-04 13:56:29 +01:00
nyaadev f2411db485 fix migration 2018-02-02 20:53:46 +01:00
nyaadev d151cca4ef fix last commit 2018-02-02 20:39:02 +01:00
TheAMM a92d886b5c Name fixes, DRY 2018-02-02 20:39:02 +01:00
nyaadev fd0a02b95c Move bencoded info dicts from mysql torrent_info table to info_dict directory. DB change!
IMPORTANT!!! Make sure to run utils/infodict_mysql2file.py before upgrading the database.
2018-02-02 20:39:02 +01:00
sfan5 418856a4bf Undo responsive table for reports (#444)
It was removed on purpose in fdb041c23b.
Instead just add a CSS rule to fix the table header.
2018-02-01 11:35:04 -08:00
sfan5 0fac1c820d Update dark theme (#441)
* Update dark theme CSS

* Use reponsive table on Admin > Reports page

Fixes dark theme styling of the table header.
2018-02-01 10:50:31 -08:00
Nicolas F 0285c12264 commenting: show CAPTCHA to new accounts (#443)
Basically re-use the upload CAPTCHA code to also do this for
comments.
2018-02-01 10:50:00 -08:00
Anna-Maria Meriniemi f8a287caa0
Improve and tidy up email blacklist regexes (Hotmail) (#438)
Because reading warnings is overrated.
This does not fix people using custom domains, but it's more likely
they'll know what's up when their email is thrown into the void.

Fixes #437.
2018-01-27 01:55:35 +02:00
Leo Izen f0bd96fe8d static: losslessly optimise PNG images even more (#432)
Used TruePNG and zopflipng to optimise the images even more,
saving a whopping 4073 bytes.

The optimisation is lossless, i.e. the decoded pixel values do not
change at all.
2018-01-16 21:34:43 -08:00
TheAMM d4fcd36b1b Fix mods' comment list filter 2018-01-04 00:34:17 +02:00
Nicolas F 362b3e3dfa user: clean up admin_form HTML (#400)
Properly scaffold it, remove a superfluous form-group div, and style
the select properly.
2018-01-03 14:28:39 -08:00
Nicolas F d00f3686f7 static: losslessly optimise PNG images (#427)
Used zopflipng to optimise the images, saving a whopping 8238 bytes.

The optimisation is lossless, i.e. the decoded pixel values do not
change at all.
2017-12-22 06:44:39 -08:00
Anna-Maria Meriniemi 3941a0b9b3
Quick and dirty comment list for moderators to look at (#421) 2017-12-04 15:51:31 +02:00
JodanJodan 052a038763 Add tabindexes to login elements (#420)
Fixes issue with password managers (e.g. KeePass) tabbing to 'Forgot your password?' link instead of password field.
2017-12-02 13:31:51 +02:00
Anna-Maria Meriniemi 7f9dc622b1 Email blacklist (#419) 2017-11-22 17:19:47 -08:00
Anna-Maria Meriniemi 1f31427e5e Alert hotmail users of the void on register page (#418)
* Alert hotmail users of the void on register page

* Words
2017-11-22 16:40:23 -08:00
Anna-Maria Meriniemi 5990cf2f50 Remove tracker limit and always add our trackers (#417)
With all trackers.txt trackers being included in generated .torrents,
we can now be certain the magnet (which use trackers.txt) and the .torrent
uses will not be split up in different swarms in case the main announce dies.
(That is, if uploaders add enough of their own trackers and additional trackers
were deemed unnecessary (at least 5 already), the magnet and .torrent would only
share the main site announce)
2017-11-22 00:02:22 -08:00
TheAMM 4cdf7f4ab3 Support searching for base32 info hash (BTIH)
"BitTorrent info hashes" are generally found in magnet uris.
An info hash is 40 characters in hex and 32 in base32 so the searches won't clash.
2017-11-14 21:27:15 +02:00
TheAMM 630c69727d Torrent validation: explicitly mention missing announce-key 2017-11-13 23:01:58 +02:00
TheAMM e183d420ea Fix CF-stripped newlines for comments 2017-11-10 05:50:57 +02:00
TheAMM faf7548cd0 Convert edited timestamp to local time 2017-11-10 05:50:50 +02:00
TheAMM 0b094a7229 Decode HTML entities in comments, make blockquotes smaller
Flask escapes < > etc, and the markdown parser does not consider
&gt; as > aka blockquote. Fixed by decoding HTML source before
rendering markdown.
2017-11-10 02:12:08 +02:00
Andrew Zhao f6735219f0 Fix keyError from pop (#407)
Remind me not to ever merge anything ever again
2017-11-06 23:52:25 +02:00
Andrew Zhao c0be2571e1 Reset page offset when updating search parameters (#406)
Updating the search query parameters should reset to the first page
2017-11-06 10:13:55 +02:00
Shane 72c997173c [Schema+config change] Comment editing (#396)
* Comment editing
* Optional time limit for comment editing
2017-11-05 16:26:30 +02:00
TheAMM b4c0ad9e84 torrents.py: Add link to torrent page into .torrent comment metadata 2017-11-04 23:46:50 +02:00
Nicolas F 96468a5a11 torrents: don't allow download of deleted torrents (#399)
Only allow moderators to download torrent files that are deleted.
2017-11-03 18:54:08 +02:00
Nicolas F 7095567b29 Comment deletion: fix small lack of validation (#395)
People could delete their own comments on torrents other than
the one the URL would indicate, which meant they could mess with
the total comment count on a torrent by having it be higher than
it actually is through repeatedly posting and deleting comments
that way.

However, they could only ever delete their own comments, so this
isn't a huge issue in the first place.
2017-10-29 20:01:19 +02:00
Arylide ca7dc276e2 Fix some text and revert a js thing 2017-10-20 22:24:36 -07:00
Nicolas F b4c9d7cd1b Fix filter dropdowns if JS is disabled (#389)
Before bootstrap-select is loaded, a small JS piece replaces the
class for the pickers with the appropriate bootstrap-select classes.

If there is no JS, the dropdowns will stay as form-control.
2017-10-20 21:01:36 -07:00
Nicolas F 4019343d50 Implement torrent nuking ability for mods (#377)
* Implement torrent nuking ability for mods

This deletes all torrents of a specific user.
A current caveat is that it will delete both sukebei and nyaa torrents,
but will only leave a log entry in the current flavour's log.

Also did some bootstrap untangling on the user view page.

* Per-flavour logging

Hopefully this works. Maybe.

* Tracker API: chunk into 100-element sublists

* isort

* Restrict nuking to superadmins

Also do a lint.sh.
2017-10-16 18:17:12 -07:00
Anna-Maria Meriniemi de1fd2f1bc [Config change] Upload ratelimit for non-trusted uploaders (#384)
* Implement upload ratelimit for non-trusted uploaders

Users may upload X torrents in Y minutes after which they
will have to wait Z minutes between uploads.

* Show torrent period count when ratelimited

* Only ratelimit new accounts
2017-10-09 18:41:18 -07:00
Nicolas F 37546354a7 Explicitly tell users they were banned (#379)
This tells users who are banned the reason that they are banned,
and doesn't show the same message for inactive users.

IP banned users are still just shown the boring 403 page.
2017-10-07 19:16:24 -07:00
Anna-Maria Meriniemi 9e87e810af [Config change] Password reset by email (#381)
* Password reset by email

Adds endpoint, templates, email templates, forms

* Timeout password reset request in six hours
2017-10-08 04:34:40 +03:00
Anna-Maria Meriniemi 6d09920abd Update email verification, add Mailgun backend (#380)
Changes config.example.py!
2017-10-07 17:31:32 -07:00
Nicolas F 19eaa2c532 Fix timezone awareness for account creation time (#378) 2017-10-06 22:36:13 +03:00
Anna-Maria Meriniemi 1e5f61ddf7 [Config change] Require recaptcha of new uploaders (#376)
* Reorganize config.example.py, add ACCOUNT_RECAPTCHA_AGE

* Require new accounts to pass recaptcha on upload

Based on ACCOUNT_RECAPTCHA_AGE in config.
2017-10-04 16:05:35 -07:00
Nicolas F 214952e7b6 Disable report button on empty reason (#374)
Previously, people couldn't quite tell you needed to give a report
reason. Now we disable the submit button until there is a reason,
and flask.flash() if someone manages to submit an empty reason
anyway.
2017-10-03 17:03:22 -07:00
Arylide 0eea181020 Fix search form typo and some HTML 2017-10-03 16:57:40 -07:00
nyaadev 4091cba837 fix mysql error and bad colors in css 2017-09-19 01:32:45 +02:00
nyaadev 2e6e924247 fix pep8 error message 2017-09-19 01:16:34 +02:00
nyaadev 89590c91bb Add ascii check in User.by_username to fix mysql errors 2017-09-18 13:28:15 +02:00
Anna-Maria Meriniemi c5d705210d Read-only maintenance mode setting for config.py (#356)
Disables all POSTs, optionally allowing users to log in (without updating last login date)
Blocked POSTs will redirect to the GET endpoint if possible, otherwise to referrer or in last case, home page.
API requests will get a plaintext message with 405 status code.
2017-09-04 18:16:52 -04:00
Arylide 5e93d7ec6d probably fix a css thing and change out a tracker 2017-09-04 15:14:26 -07:00
sfan5 45e1a650fb Improve dark theme (2) (#358)
* Further improve dark theme

* do something about .alert-info, idk
2017-09-04 18:01:40 -04:00
sfan5 4995f95ebd Improve dark theme (#357)
* Add 'dark' class to <body> if dark theme is activated

* Make dark theme suck considerably less
2017-09-03 18:18:39 -04:00
TheAMM 6b49af4fb7 Exception handling: include a random string to identify stacks from log 2017-09-02 22:18:32 +03:00
Anna-Maria Meriniemi 36d3f8aed0 Early filename validation (#354)
Currently just rejects torrents with RTL overrides in filenames
2017-09-01 18:27:14 -04:00
Anna-Maria Meriniemi 04a8ff95d2 Words (#355)
* Add words

* Add more words

Words are nice
2017-09-01 18:14:11 -04:00
TheAMM 07b45622f4 Get rid of index name hardcoding, read from database (and cache)
Index names are not necessarily named 'ix_table_column', so inspect the real schema for index names. Results are cached in memory.
(I have no clue how, but mine are prefixed idx_ instead of ix_!)
Clears out the "sort" variable naming as well.
2017-08-31 19:12:50 +03:00
nyaadev 57a71be7b8 minor changes 2017-08-31 14:20:34 +02:00
Alex Ingram df288bfadd Add Firefox search provider. (#353)
* Add Firefox search provider.

* Add search provider to layout.

* Add search thing for Sukebei, too.

* Fix layout.
2017-08-30 23:52:50 -04:00
nyaadev 294a12700c fix mysql being shit 2017-08-31 03:36:29 +02:00
Anna-Maria Meriniemi 4eed4971a3 Fix banning torrents without uploader ip
Pretty much just for archived torrents
2017-08-26 02:52:59 +03:00
Anna-Maria Meriniemi aa3c2ba77b Fix banning anonymous torrents
We miss ye, testing
2017-08-26 02:36:38 +03:00
A nyaa developer f8a314df4f Better bans (#341)
* better bans

* put jinja2 template into correct file
2017-08-25 18:53:35 -04:00
TheAMM 6aab5557d6 ...remove debug print
Alas, hasty mergers
I'm not force-pushing master on a 1030-star repo because of an accidental debug line, no sirrre.
2017-08-20 04:06:04 +03:00
Anna-Maria Meriniemi 39fcfc0058 Add (optional) validation for minimum anonymous torrent size (#342)
MINIMUM_ANONYMOUS_TORRENT_SIZE can be used to require a minimum total
size of torrents uploaded by anonymous users (ie. without accounts).

Sets up a "framework" for post-WTForm torrent validation as well;
this can easily be extended into filename blacklists and such.
2017-08-19 20:48:08 -04:00
Nicolas F 48d4217f02 help page: Add help for deletion (#340)
Some people may not realise that they can do this themselves,
and maybe they'll consult the help page before abusing the report
button.
2017-08-19 06:38:32 -04:00
Kfir Hadas a758f5f078 Optimize routes (define url_prefix in Blueprints)
* /admin/*
* /api/*
2017-08-15 17:12:54 +03:00
nyaadev 0bcd782695 Minor improvements 2017-08-15 01:41:31 +02:00
Arylide 2cd2dbf66d Replaced dead tracker and give mods a hammer 2017-08-14 16:02:06 -07:00
mreweilk 2e50f14fb7 Change admin routes from /adminlog and /reports to /admin/log and /admin/reports 2017-08-14 13:30:17 -04:00
nyaadev d5b8a3a2ae Increase maximum comment size from 255 to 1024.
DB change: Change comment text field from VARCHAR(255) to mysql.TEXT
2017-08-14 19:08:36 +02:00
nyaadev 1c3724cae1 Delete cached torrent file when replacing torrent. 2017-08-07 00:20:02 +02:00
nyaadev 602d35bab7 Integrate tracker API for torrent ban/unban 2017-08-06 23:55:45 +02:00
Arylide 5fb178f3a8 idk 2017-08-05 13:03:43 -07:00
TheAMM 7a04d64f9c Weed out last few hardcoded urls 2017-08-05 22:43:13 +03:00
A nyaa developer e728ca1818 Expose soft delete to users and allow reuploading of deleted torrents. (#331)
Add banning torrents for moderators which disallows reuploading.
New delete UI.
2017-08-05 15:41:59 -04:00
Anna-Maria Meriniemi 95e35a10c9 Change torrent Content-Disposition back to inline
As per comments in #288, Firefox forces a "what do you want to do with this"-dialog for 'attachment' even when it's set to open them immediately in another program.
2017-08-03 11:28:21 +03:00
Kfir Hadas 87dd95f1e0 Refactor into an app factory [2 of 2] (#322)
* Replace all `from nyaa import app` imports with `app = flask.current_app` (or `from flask import current_app as app` where possible)
* Add a separate config object for top-level and class statements as `nyaa.extensions.config`
Required because those codes don't have app context at the time of evaluation/execution.
* Remove `routes.py` file and register all blueprints in `nyaa/__init__.py`
* Refactor `nyaa/__init__.py` into an app factory
* Update tools
* Update tests (temporary, will be replaced)
2017-08-01 21:02:08 +03:00
Kfir Hadas 0181d6cb33 Prepare for app factory [1 of 2] (#315)
* Move db, assets, debug toolbar and fix_paginate into nyaa.extensions
* Change all `from nyaa import db` imports to `from nyaa.extensions import db`
* Move `nyaa.torrents.create_magnet_from_es_info` context processor into template-utils blueprint
* Fix tools (wrap in `with app.app_context():` where needed)
2017-07-30 20:35:16 +03:00
TheAMM 1dae331156 Completely handle qBittorrent webseeds
Fixes #319

Zero webseeds: empty bytestring
One webseed: bytestring
More than one: a list of bytestrings (as the spec asks for!)
2017-07-30 11:49:13 +03:00
TheAMM 987103b213 Handle qButtorrent empty url-list/webseeds
url-list is supposed to be omitted in case of no webseeds,
but qB will instead use an empty bytestring '' as the value.
This commit makes the validation more lenient regarding url-list.
2017-07-30 11:36:38 +03:00