Apparently, when running things with Python 3.7, these three
dependencies ran into build issues with a renamed struct field.
Upgrading them seems to fix the issue, and hopefully keeps them
working with Python 3.6 as well.
Windows has a few special filenames that it does not allow the
explorer.exe and command line to see, but can still be created by
applications. This is due to some jank DOS compatibility.
These filenames can be abused to troll Windows users, so we should
probably blacklist them.
Adds a new config entry (EMAIL_SERVER_BLACKLIST, tuple of IPv4 addresses
as strings) and an email validator for registering, which will query all
the MX records for the domain and reject the registration if any of the
A records for any of the MX records are found in the blacklist.
If the query fails, the blacklist is ignored; the email is accepted.
Infobubble text is now in a separate file, along with a timestamp,
so that changes to it don't result in merge conflicts too often.
We also add some JS to make the bubble dismissible, keeping track
of the last timestamp that was dismissed in localstorage.
Removes the specialized template ES magnet creator, since create_magnet()
can use both Torrents and ES objects. Search results will get the
properly escaped magnets, now.
Slightly optimizes the tracker adding and string joins.
RIP base32, wonder how many bad clients will break with sha1.
A cosmetic change.
Swapping quote_via to quote, we don't convert spaces to pluses, keeping
the name intact (which only mattered until peers send the metadata).
Using Flask-Caching, we can memoize the magnet_uri method. Here, a
timeout of 1 hour is chosen, though that value can be fiddled with.
The cache is defined in extensions.py, but gets initialised in
__init__.py.
* Add config option to enable/disable gravatar
This is useful when running a development instance behind a firewall
or NAT, where gravatar cannot reach you to serve up the default user
avatar.
* Pregenerate Gravatar default image URLs
If possible (i.e. SERVER_NAME is set), we can pregenerate the constant
gravatar default URL once at application startup, and re-use that,
as url_for calls are surprisingly expensive.
Especially on torrent view pages with lots of comments, this cuts down
on url_for calls massively, saving on my system about 0.3 ms per call.
Now uses flask.flash to give the person who clicks it feedback
when it's clicked, because caching may make things confusing.
Also only activate a user if they're inactive, as again, caching
can lead to staff pressing the button multiple times in a row,
leading to unnecessary log messages.
* Implement range bans
People connecting from banned IP ranges are unable to upload
torrents anonymously, and need to manually have their accounts
activated.
This adds a new table "rangebans", and a command line utility,
"rangeban.py", which can be used to add, list and remove rangebans
from the command line.
As an example:
./rangeban.py ban 192.168.0.0/24
This would rangeban anything in this /24.
The temporary_tor column allows automated scripts to clean out and
re-add ever-changing sets of ranges to be banned without affecting
the other ranges.
This has only been tested for IPv4.
* Revise Rangebans
Add an id column, and change "temporary_tor" to "temp". Also
index masked_cidr and mask.
* rangebans: fix enabled and the binary op
kill me
* Add enabling/disabling bans to rangeban.py
* rangebans: fail earlier on garbage arguments
* rangebans: fix linter errors
* rangeban.py: don't shadow builtin keyword 'id'
* rangebans: change temporary ban logic, column
The 'temp' column is now a nullable time column. If the field is
null, the ban is understood to be permanent. If there is a time
in there, it's understood to be the creation time of the ban.
This allows scripts to e.g. delete all temporary bans older than
a certain amount of time.
Also, rename the '_cidr_string' column to 'cidr_string', because
reasons.
* rangeban.py: use ip_address to parse CIDR subnet
* rangebans: fixes to the mask calculation and query
Both were not bugs per-se, but just technically not needed/correct.
* De-meme apparently
...by splitting input into characters, instead of whitespace delimited
words. This means you can now match partial words, real substrings from
anywhere: "foo ba" will match "Foo Bar Baz", while previously you had to
have full words ("foo bar") to match anything.
My dev setup incurred an 8% increase in storage usage, from ~13MB to
~14MB (for ~40k torrents).
Small change, big improvement. Wonder why I didn't do this at first.
* user page: add manual activation button for mods
Moderators can press this button on inactive users to manually
activate their accounts.
Furthermore, the admin form code has been refactored a bit, reducing
some code duplication.
Before, long.tokens.with.dots.or.dashes would get edgengrammed up to the
ngram limit, so we'd get to long.tokens.wit which would then be split -
discarding "with.dots.or.dashes" completely. The fullword index would
keep the complete large token, but without any ngramming, so incomplete
searches (like "tokens") would not match it, only the full token.
Now, we split words before ngramming them, so the main index will
properly handle words up to the ngram limit. The fullword index will
still handle the longer words for non-ngram matching.
Also optimized away duplicate tokens from the indices (since we rely on
boolean matching, not scoring) to save a couple megabytes of space.
* added es_sync_config.json and minified js to .gitignore
* reordered README.md to reflect that MySQL Binlogging must be enabled before running import_to_es.py
* Extend ES term preprocessing for OR groups
Implements handling "foo"|"bar" literal OR groups in the Elasticsearch
term preprocessor. Groups can be negated with -, but don't mesh with
precedence (like plain literals).
This is a partial hack, the real solution would be to parse the entire
search terms ourselves, with AND and OR groups, negations etc. But
having that work neatly with the simple_query_string would be bit of a
hassle.
* Update help.html search tips
since search (quoting strings) has changed a bit.
* Optimize Elasticsearch fullword field
Since the main display_name field ngrams words up to 15 characters,
anything to and under that will already be indexed - the fullword field
(which we have for words longer than 15 characters) needs to index only
words longer than that.
* Preprocess ES terms for better literal matching
This commit adds a new .exact subfield to display_name, which holds a
barely-filtered version of the original title we can do "literal"
matching against. This is not real substring matching, but quoting
terms now actually does something!
Implements a simple preprocessor for the search terms to extract quoted
parts from the search terms, optionally prefixed with - to negate them.
The preprocessor will create a query that'll join all three query-types:
the simple_query_string, must-phrases and must-not-phrases.
Hitting the cancel button does not return "", but null. Therefore
the toLowerCase() fails, and throwing an exception means "sure go
ahead submitting this" to JS for some godforsaken reason.
Just remove the toLowerCase for now, have people type the names
properly.
* Use Flask-Assets to minify self-hosted JS files
By having Flask-Assets minify the two JS files we ship, namely
main.js and bootstrap-select.js, we can shave off 28406 bytes.
The minified files are generated on startup. If one wishes to
manually clean them up or build them, they can use the
"flask assets" management command, e.g. "flask assets clean".
* Workaround to fix tests
State carries over in tests, which is the dumbest shit ever. Fix it
by clearing the bundles before setting them.
* Implement comment locking
This adds a new flags to torrents, which is only editable by
moderators and admins. If checked, it does not allow unprivileged
users to post, edit or delete comments on that torrent.
* Rename "locked" to "comment_locked".
* Shorter button and additional words on alt text
* Admin log: Change comment locking message
dude I love bikeshedding xd
* Bikeshedding over admin log messages
* >&
Also some bikeshedding
I currently don't differentiate between "trusted" markdown and
untrusted, but this should be good enough. Basically tells the
browser not to send a referrer, and (not sure if relevant here)
not to expose a window opener object. Also tells search engines
that the link is not endorsed with "nofollow".
This started out as a simple rebase, but then I rebased the wrong
branches and it all got confusing, so here it is as a new dank
commit.
We now have an @admin_only decorator, and we ask for confirmation
before we nuke. We can also see the nuke button when users are
banned, and nuking is a separate endpoint with a separate form.
Additionally, it now uses the new tracker API.
python-mysql-replication (or PyMySQL) would return less than 20 bytes
for info-hashes that had null bytes near the end, leaving incomplete
hashes in the ES index. Without delving too deep into the real issue
(be it lack of understanding MySQL storing binary data or a bug in
the libraries), thankfully we can just pad the fixed-size info-hashes
to be 20 bytes.
Padding in import_to_es.py may be erring on the side of caution, but
safe is established to be better than sorry.
(SQLAlchemy is unaffected by this bug)
Fixes#456
If users kept their page open for a while before reporting a
torrent, and mods got it in the meantime, users could still
submit reports for that torrent. This is silly and really doesn't
need to happen.
