1
0
Fork 0
mirror of https://gitlab.com/SIGBUS/nyaa.git synced 2024-10-31 23:25:58 +00:00
Commit graph

627 commits

Author SHA1 Message Date
nyaadev 8b809de8a2 improve update_comment_count_db 2019-08-13 14:17:25 +02:00
nyaadev 93dffcd663 nuke comments 2019-08-13 14:13:37 +02:00
Nicolas F 80c9d580eb Improve cache configuration (#564)
The Flask-Caching cache can now properly be configured from the
config.py, and redis caching has experimentally been tested and
confirmed to be working in theory.

We also document that one may want to use CACHE_THRESHOLD to limit
the maximum number of items in the simple cache.
2019-08-10 18:53:57 -07:00
Nicolas F d8e796f3e0 Move to Python 3.7 and update dependencies (#551)
* forms: replace re._pattern_type with re.Pattern

Python 3.7 removed re._pattern_type and replaced it with
re.Pattern.

* readme: update for Python 3.7

* Update requirements

Also remove some unused ones which were neither a direct dependency
nor a dependency of our dependencies.

* account: force ASCII usernames on login form

Our database doesn't like it when we check for unicode data in
a column that stores ASCII data, so let's stop it before it
gets that far.

* Move travis CI to Python 3.7

* travis: use xenial dist

* fix newer linter warnings

Apparently bare excepts are literally Hitler, and we have some
new import sorting rules. Hooray!

* requirements: remove six

This is a dependency for sqlalchemy-utils, but we ourselves don't
depend on it directly because we've never been on Python 2 ever.

* Update requirements.txt
2019-08-10 18:39:53 -07:00
Nicolas F 16814d6eb7 Add trusted application functionality (#533)
* Add trusted application functionality

This lets users apply for trusted status, given certain minimum
requirements. Moderators can then review the applications, giving
a recommendation, and administrators can accept or reject them.

If an application is accepted or rejected, the user receives an
e-mail about it.

Markdown images are not rendered in applications to prevent browsers
from sending automatic requests to untrusted webservers.

Users who have had their application rejected cannot re-apply for a set
amount of days.

* minor fixes
2019-08-10 18:18:44 -07:00
nallown ff44d7a51c merge #481 add trusted and remake information to RSS feed 2019-08-10 23:32:38 +02:00
Nicolas F 90607d6993 main.css: fix stat styling on dark mode 2019-05-20 16:02:04 +02:00
Nicolas F 8614da6322 view.html: fix comment text copypasting in Firefox
Firefox would insert a lot of weird whitespace, because its
user-select would default to "all" and not "text", so it copypastes
the whitespace of otherwise completely invisible HTML elements.

I don't know why Firefox does this. It's stupid, and not what the
user expects. They're copying what they see, not what the DOM is.
2019-04-30 23:12:11 +02:00
Nicolas F 8f1925ce05 search_results: various fixes to the DOM
Avoid repeated style attributes, instead use nth-child CSS selectors
and classes where appropriate.

Also, get rid of the useless <div> around column headings. They don't
seem to do anything useful and are just more stuff for the browser's
layout engine to juggle around.

We can also use Jinja2 to remove some of the unnecessary whitespace
around table headings, which makes things like copy-pasting in Firefox
act less weird. This includes some other whitespace fixes too.
2019-04-30 23:11:40 +02:00
Nicolas F 38f8880966 Make blue info bubble dismissible, configurable (#526)
Infobubble text is now in a separate file, along with a timestamp
in the main file, so that changes to it don't result in merge
conflicts too often.

We also add some JS to make the bubble dismissible, keeping track
of the last timestamp that was dismissed in localstorage.

A timestamp of 0 disables the infobubble altogether.
2019-04-08 19:50:30 -07:00
Nicolas F 7de7147743 Query limit fixes (#555)
* Fix total torrent count for user listings

The total count would previously be bound by the maximum number
of pages we allow. Since we run the count query anyway, we can
just save this result and use it in the template.

* search: allow users to view all their uploads

Maximum page limitations shouldn't take effect when users are looking
at a listing of their own torrents.
2019-04-08 16:59:30 -07:00
TheAMM 885cccca40 Cache url_for calls with lru_cache
This commit adds a caching_url_for using functools.lru_cache (currently
with maxsize at 4096 entries) and replaces flask.url_for with it, as
there is no harm in doing so.

This greatly improves template generation speed, from ~115ms to ~75ms on
the front page (using the simple benchmark introduced in the previous
commit).
2019-04-08 19:28:26 +02:00
TheAMM a7558823d0 Further cleanup 2019-04-08 19:18:11 +02:00
Alex Ingram bd4eba9ab7 Change options in models.py based on AMM feedback. 2019-04-08 19:18:11 +02:00
Alex Ingram 75e7e942fb Fix some JOIN insanity with SQLAlchemy. 2019-04-08 19:18:11 +02:00
TheAMM b0c51e9fa0 Add very simple benchmarking support when debugging 2019-04-08 19:31:34 +03:00
Nicolas F db83989d5d extensions: limit pagination to max num of pages 2019-04-02 18:57:56 +02:00
Nicolas F 6608c99b90 search: allow limiting the maximum number of pages 2019-04-02 17:56:43 +02:00
Nicolas F 2782589cda config: add live.no to email blacklist 2019-03-30 16:33:08 +01:00
Nicolas F 6a63ee241a templates: cache torrent view filelist (#542)
* templates: cache torrent view filelist

Using flask-caching, we can add a 1 hour cache to the template
output of a filelist, varying it by the key "filelist" + the
hex infohash of a torrent.

Using a very big filelist as a test, I get a difference in page
load speeds of about a magnitude. (400ms -> 37 ms)

* templates: increase filelist cache to 24 hours
2019-03-08 06:29:51 -08:00
Nicolas F 585e1aeae7 tests: add tests for XML char replacement 2019-02-25 12:27:09 +01:00
TheAMM d7f9618fbf upload/edit: sanitize some string fields
This commit introduces a regex to replace illegal (expectedly unused)
characters from torrent display name, information link and description
upon upload or edit.

Fixes #541
2019-02-25 12:27:09 +01:00
Alex Ingram 1374375a16 Fix category icon display issue. (#547) 2019-02-24 15:07:49 +01:00
Nicolas F 51192342a6
view: fix template throwing exception on edge case (#549)
If a user has a comment under the edit time limit in a comment
locked torrent, but also are still affected by the new account CAPTCHA
cooldown, the template would throw an error as we tried to getattr on
a None object (namely, the comment_form).

To fix this, we also need to check around the edit form whether the
comment_form exists.
2019-02-24 15:03:38 +01:00
nyaadev 2e2e741d0f fix migration, add server_default for boolean 2019-01-24 12:31:48 +01:00
Alex Headley 18ebf134d5 feature: docker development environment (#544)
Add docker support

It's self-contained so should not interfere with anything else and can easily be thrown out if nobody wants it anymore.
2019-01-15 18:02:59 +01:00
Nicolas F a6d800b4ca templates: move icon_dir set outside of the loop
Someone put this inside the loop despite it essentially being
constant. Probably makes immeasurably little difference perf-wise,
but why not fix it anyway.
2019-01-12 20:15:15 +01:00
Anna-Maria Meriniemi 8365894268 config: add anonymous upload/registration limiting (#537) 2019-01-02 03:40:56 -08:00
AC e4780aa47d Replace leechers-paradise with desync (#536) 2018-12-13 18:05:26 -08:00
A nyaa developer 4fea839f51
fix user preferences from #515 (#532) 2018-12-06 15:47:08 +01:00
doge e04bbdfbf9 Replaced spaces with tabs 2018-11-20 13:45:31 +01:00
doge 92ca3a577f Generate database migration script 2018-11-20 13:45:31 +01:00
doge 3190394eea Comment Hiding
This commit adds the ability hide comments on torrent pages, and a user
preferences tab to allow registered users to make this the default
behavior.
2018-11-20 13:45:31 +01:00
Nicolas F 18bdf465f7 requirements: update dependencies for newer python
Apparently, when running things with Python 3.7, these three
dependencies ran into build issues with a renamed struct field.
Upgrading them seems to fix the issue, and hopefully keeps them
working with Python 3.6 as well.
2018-11-11 02:52:33 +01:00
Nalapl3 eabe13e20a Add amount of comments to RSS (#528)
Closes #496.
2018-10-27 17:17:13 +03:00
Nicolas F c967b831c6 backend: blacklist Windows reserved filenames (#527)
Windows has a few special filenames that it does not allow the
explorer.exe and command line to see, but can still be created by
applications. This is due to some jank DOS compatibility.

These filenames can be abused to troll Windows users, so we should
probably blacklist them.
2018-10-27 17:13:44 +03:00
TheAMM b2ddba994c Fix open redirect (#519)
The funny thing is that we don't even use this anywhere, and the
referrer is useless on forms. But hey, maybe someday.
2018-09-16 21:16:35 +03:00
Anna-Maria Meriniemi 8c892f09cc config/forms: allow blacklisting email servers by IP (#518)
Adds a new config entry (EMAIL_SERVER_BLACKLIST, tuple of IPv4 addresses
as strings) and an email validator for registering, which will query all
the MX records for the domain and reject the registration if any of the
A records for any of the MX records are found in the blacklist.
If the query fails, the blacklist is ignored; the email is accepted.
2018-08-20 11:29:30 -07:00
Nicolas F 86b7eb7ccd Revert "Make blue info bubble dismissible, configurable (#508)"
This reverts commit 62ec8a5236.
2018-08-03 04:34:19 +02:00
Nicolas F 62ec8a5236 Make blue info bubble dismissible, configurable (#508)
Infobubble text is now in a separate file, along with a timestamp,
so that changes to it don't result in merge conflicts too often.

We also add some JS to make the bubble dismissible, keeping track
of the last timestamp that was dismissed in localstorage.
2018-08-02 19:16:28 -07:00
TheAMM bf090c0fab torrents: clean up & optimize magnets further
Removes the specialized template ES magnet creator, since create_magnet()
can use both Torrents and ES objects. Search results will get the
properly escaped magnets, now.
Slightly optimizes the tracker adding and string joins.
RIP base32, wonder how many bad clients will break with sha1.
2018-07-21 22:06:58 +03:00
Tadeo Kondrak ef7ff1b9ce Add torrent counter on profile page (#409) (#513) 2018-07-17 14:52:54 +03:00
TheAMM 2bf96196cb torrents: Clean up magnet URI quotation
A cosmetic change.
Swapping quote_via to quote, we don't convert spaces to pluses, keeping
the name intact (which only mattered until peers send the metadata).
2018-07-15 04:40:07 +03:00
Nicolas F bd419c5d39 Add 1 hour cache to magnet URIs (#503)
Using Flask-Caching, we can memoize the magnet_uri method. Here, a
timeout of 1 hour is chosen, though that value can be fiddled with.

The cache is defined in extensions.py, but gets initialised in
__init__.py.
2018-07-10 01:22:49 -07:00
Anna-Maria Meriniemi 5c8b119611 config: Add Elasticsearch hosts (#492) 2018-07-09 22:26:23 -07:00
Nicolas F 56a670977e [Config Change] Various Gravatar-related changes (#467)
* Add config option to enable/disable gravatar

This is useful when running a development instance behind a firewall
or NAT, where gravatar cannot reach you to serve up the default user
avatar.

* Pregenerate Gravatar default image URLs

If possible (i.e. SERVER_NAME is set), we can pregenerate the constant
gravatar default URL once at application startup, and re-use that,
as url_for calls are surprisingly expensive.

Especially on torrent view pages with lots of comments, this cuts down
on url_for calls massively, saving on my system about 0.3 ms per call.
2018-07-09 22:20:26 -07:00
Nicolas F e892f358e8 users: activation button improvements (#506)
Now uses flask.flash to give the person who clicks it feedback
when it's clicked, because caching may make things confusing.

Also only activate a user if they're inactive, as again, caching
can lead to staff pressing the button multiple times in a row,
leading to unnecessary log messages.
2018-07-09 22:15:19 -07:00
Nicolas F 8644472533 Add registration IP (#507)
This will keep track of which IP a user has registered from, to
prevent evading rangebans. It will only be shown to admins.
2018-07-09 22:14:45 -07:00
Nicolas F a38e5d5b53 Implement range bans (#478)
* Implement range bans

People connecting from banned IP ranges are unable to upload
torrents anonymously, and need to manually have their accounts
activated.

This adds a new table "rangebans", and a command line utility,
"rangeban.py", which can be used to add, list and remove rangebans
from the command line.

As an example:

./rangeban.py ban 192.168.0.0/24

This would rangeban anything in this /24.

The temporary_tor column allows automated scripts to clean out and
re-add ever-changing sets of ranges to be banned without affecting
the other ranges.

This has only been tested for IPv4.

* Revise Rangebans

Add an id column, and change "temporary_tor" to "temp". Also
index masked_cidr and mask.

* rangebans: fix enabled and the binary op

kill me

* Add enabling/disabling bans to rangeban.py

* rangebans: fail earlier on garbage arguments

* rangebans: fix linter errors

* rangeban.py: don't shadow builtin keyword 'id'

* rangebans: change temporary ban logic, column

The 'temp' column is now a nullable time column. If the field is
null, the ban is understood to be permanent. If there is a time
in there, it's understood to be the creation time of the ban.

This allows scripts to e.g. delete all temporary bans older than
a certain amount of time.

Also, rename the '_cidr_string' column to 'cidr_string', because
reasons.

* rangeban.py: use ip_address to parse CIDR subnet

* rangebans: fixes to the mask calculation and query

Both were not bugs per-se, but just technically not needed/correct.

* De-meme apparently
2018-06-29 20:15:04 -07:00
Nicolas F f04e0fd2ae comments: fix editing for users with a CAPTCHA (#502)
* comments: fix editing for users with a CAPTCHA

My bad!

* Update help.html
2018-06-16 16:57:14 -07:00