SQA's baked queries prepares the queries in advance, caching yada yada.
Makes thing a bit faster.
Also bigger speedup included is a shoddy cache for the total torrent
count (only applied to baked queries currently). Caching the value for a
few dozen seconds shaves off some wasted time, as it's mostly just used
for pagination.
Users could double their ratelimit by uploading some torrents as
anonymous submissions, then log into their account and post more.
We can stop this by making the filter_uploader helper function use
an sqlalchemy.or_ query to check for uploads from either that user
or that user's IP.
The Flask-Caching cache can now properly be configured from the
config.py, and redis caching has experimentally been tested and
confirmed to be working in theory.
We also document that one may want to use CACHE_THRESHOLD to limit
the maximum number of items in the simple cache.
* forms: replace re._pattern_type with re.Pattern
Python 3.7 removed re._pattern_type and replaced it with
re.Pattern.
* readme: update for Python 3.7
* Update requirements
Also remove some unused ones which were neither a direct dependency
nor a dependency of our dependencies.
* account: force ASCII usernames on login form
Our database doesn't like it when we check for unicode data in
a column that stores ASCII data, so let's stop it before it
gets that far.
* Move travis CI to Python 3.7
* travis: use xenial dist
* fix newer linter warnings
Apparently bare excepts are literally Hitler, and we have some
new import sorting rules. Hooray!
* requirements: remove six
This is a dependency for sqlalchemy-utils, but we ourselves don't
depend on it directly because we've never been on Python 2 ever.
* Update requirements.txt
* Add trusted application functionality
This lets users apply for trusted status, given certain minimum
requirements. Moderators can then review the applications, giving
a recommendation, and administrators can accept or reject them.
If an application is accepted or rejected, the user receives an
e-mail about it.
Markdown images are not rendered in applications to prevent browsers
from sending automatic requests to untrusted webservers.
Users who have had their application rejected cannot re-apply for a set
amount of days.
* minor fixes
Firefox would insert a lot of weird whitespace, because its
user-select would default to "all" and not "text", so it copypastes
the whitespace of otherwise completely invisible HTML elements.
I don't know why Firefox does this. It's stupid, and not what the
user expects. They're copying what they see, not what the DOM is.
Avoid repeated style attributes, instead use nth-child CSS selectors
and classes where appropriate.
Also, get rid of the useless <div> around column headings. They don't
seem to do anything useful and are just more stuff for the browser's
layout engine to juggle around.
We can also use Jinja2 to remove some of the unnecessary whitespace
around table headings, which makes things like copy-pasting in Firefox
act less weird. This includes some other whitespace fixes too.
Infobubble text is now in a separate file, along with a timestamp
in the main file, so that changes to it don't result in merge
conflicts too often.
We also add some JS to make the bubble dismissible, keeping track
of the last timestamp that was dismissed in localstorage.
A timestamp of 0 disables the infobubble altogether.
* Fix total torrent count for user listings
The total count would previously be bound by the maximum number
of pages we allow. Since we run the count query anyway, we can
just save this result and use it in the template.
* search: allow users to view all their uploads
Maximum page limitations shouldn't take effect when users are looking
at a listing of their own torrents.
This commit adds a caching_url_for using functools.lru_cache (currently
with maxsize at 4096 entries) and replaces flask.url_for with it, as
there is no harm in doing so.
This greatly improves template generation speed, from ~115ms to ~75ms on
the front page (using the simple benchmark introduced in the previous
commit).
* templates: cache torrent view filelist
Using flask-caching, we can add a 1 hour cache to the template
output of a filelist, varying it by the key "filelist" + the
hex infohash of a torrent.
Using a very big filelist as a test, I get a difference in page
load speeds of about a magnitude. (400ms -> 37 ms)
* templates: increase filelist cache to 24 hours
This commit introduces a regex to replace illegal (expectedly unused)
characters from torrent display name, information link and description
upon upload or edit.
Fixes#541
If a user has a comment under the edit time limit in a comment
locked torrent, but also are still affected by the new account CAPTCHA
cooldown, the template would throw an error as we tried to getattr on
a None object (namely, the comment_form).
To fix this, we also need to check around the edit form whether the
comment_form exists.
Someone put this inside the loop despite it essentially being
constant. Probably makes immeasurably little difference perf-wise,
but why not fix it anyway.
Windows has a few special filenames that it does not allow the
explorer.exe and command line to see, but can still be created by
applications. This is due to some jank DOS compatibility.
These filenames can be abused to troll Windows users, so we should
probably blacklist them.
Adds a new config entry (EMAIL_SERVER_BLACKLIST, tuple of IPv4 addresses
as strings) and an email validator for registering, which will query all
the MX records for the domain and reject the registration if any of the
A records for any of the MX records are found in the blacklist.
If the query fails, the blacklist is ignored; the email is accepted.
Infobubble text is now in a separate file, along with a timestamp,
so that changes to it don't result in merge conflicts too often.
We also add some JS to make the bubble dismissible, keeping track
of the last timestamp that was dismissed in localstorage.
Removes the specialized template ES magnet creator, since create_magnet()
can use both Torrents and ES objects. Search results will get the
properly escaped magnets, now.
Slightly optimizes the tracker adding and string joins.
RIP base32, wonder how many bad clients will break with sha1.
A cosmetic change.
Swapping quote_via to quote, we don't convert spaces to pluses, keeping
the name intact (which only mattered until peers send the metadata).
Using Flask-Caching, we can memoize the magnet_uri method. Here, a
timeout of 1 hour is chosen, though that value can be fiddled with.
The cache is defined in extensions.py, but gets initialised in
__init__.py.
* Add config option to enable/disable gravatar
This is useful when running a development instance behind a firewall
or NAT, where gravatar cannot reach you to serve up the default user
avatar.
* Pregenerate Gravatar default image URLs
If possible (i.e. SERVER_NAME is set), we can pregenerate the constant
gravatar default URL once at application startup, and re-use that,
as url_for calls are surprisingly expensive.
Especially on torrent view pages with lots of comments, this cuts down
on url_for calls massively, saving on my system about 0.3 ms per call.
Now uses flask.flash to give the person who clicks it feedback
when it's clicked, because caching may make things confusing.
Also only activate a user if they're inactive, as again, caching
can lead to staff pressing the button multiple times in a row,
leading to unnecessary log messages.
* Implement range bans
People connecting from banned IP ranges are unable to upload
torrents anonymously, and need to manually have their accounts
activated.
This adds a new table "rangebans", and a command line utility,
"rangeban.py", which can be used to add, list and remove rangebans
from the command line.
As an example:
./rangeban.py ban 192.168.0.0/24
This would rangeban anything in this /24.
The temporary_tor column allows automated scripts to clean out and
re-add ever-changing sets of ranges to be banned without affecting
the other ranges.
This has only been tested for IPv4.
* Revise Rangebans
Add an id column, and change "temporary_tor" to "temp". Also
index masked_cidr and mask.
* rangebans: fix enabled and the binary op
kill me
* Add enabling/disabling bans to rangeban.py
* rangebans: fail earlier on garbage arguments
* rangebans: fix linter errors
* rangeban.py: don't shadow builtin keyword 'id'
* rangebans: change temporary ban logic, column
The 'temp' column is now a nullable time column. If the field is
null, the ban is understood to be permanent. If there is a time
in there, it's understood to be the creation time of the ban.
This allows scripts to e.g. delete all temporary bans older than
a certain amount of time.
Also, rename the '_cidr_string' column to 'cidr_string', because
reasons.
* rangeban.py: use ip_address to parse CIDR subnet
* rangebans: fixes to the mask calculation and query
Both were not bugs per-se, but just technically not needed/correct.
* De-meme apparently
