mirror of
https://gitlab.com/SIGBUS/nyaa.git
synced 2024-06-18 13:23:12 +00:00
|
||
---|---|---|
.github | ||
configs | ||
migrations | ||
nyaa | ||
torrent_cache | ||
utils | ||
.gitignore | ||
.travis.yml | ||
config.example.py | ||
config_es_sync.json | ||
create_es.sh | ||
db_create.py | ||
db_migrate.py | ||
es_mapping.yml | ||
import_to_es.py | ||
LICENSE | ||
lint.sh | ||
README.md | ||
requirements.txt | ||
run.py | ||
sync_es.py | ||
trackers.txt | ||
uwsgi.ini | ||
WSGI.py |
NyaaV2
Setup
- Install dependencies https://github.com/pyenv/pyenv/wiki/Common-build-problems
- Install
pyenv
https://github.com/pyenv/pyenv/blob/master/README.md#installation - Install
pyenv-virtualenv
https://github.com/pyenv/pyenv-virtualenv/blob/master/README.md pyenv install 3.6.1
pyenv virtualenv 3.6.1 nyaa
pyenv activate nyaa
- Install dependencies with
pip install -r requirements.txt
- Copy
config.example.py
intoconfig.py
- Change TABLE_PREFIX to
nyaa_
orsukebei_
depending on the site
Setting up MySQL/MariaDB database for advanced functionality
- Enable
USE_MYSQL
flag in config.py - Install latest mariadb by following instructions here https://downloads.mariadb.org/mariadb/repositories/
- Tested versions:
mysql Ver 15.1 Distrib 10.0.30-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2
- Tested versions:
- Run the following commands logged in as your root db user:
CREATE USER 'test'@'localhost' IDENTIFIED BY 'test123';
GRANT ALL PRIVILEGES ON * . * TO 'test'@'localhost';
FLUSH PRIVILEGES;
CREATE DATABASE nyaav2 DEFAULT CHARACTER SET utf8 COLLATE utf8_bin;
- To setup and import nyaa_maria_vx.sql:
mysql -u <user> -p nyaav2
DROP DATABASE nyaav2;
CREATE DATABASE nyaav2 DEFAULT CHARACTER SET utf8 COLLATE utf8_bin;
SOURCE ~/path/to/database/nyaa_maria_vx.sql
Finishing up
- Run
python db_create.py
to create the database - Load the .sql file
mysql -u user -p nyaav2
SOURCE cocks.sql
- Remember to change the default user password to an empty string to disable logging in
- Start the dev server with
python run.py
- When you are finished developing, deactivate your virtualenv with
source deactivate
Enabling ElasticSearch
Basics
- Install jdk
sudo apt-get install openjdk-8-jdk
- Install elasticsearch https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html
sudo systemctl enable elasticsearch.service
sudo systemctl start elasticsearch.service
- Run
curl -XGET 'localhost:9200'
and make sure ES is running - Optional: install Kabana as a search frontend for ES
Enable MySQL Binlogging
- Add the
[mariadb]
bin-log section to my.cnf and reload mysql server - Connect to mysql
SHOW VARIABLES LIKE 'binlog_format';
- Make sure it shows ROW
- Connect to root user
GRANT REPLICATION SLAVE ON *.* TO 'test'@'localhost';
where test is the user you will be runningsync_es.py
with
Setting up ES
- Run
./create_es.sh
and this creates two indicies:nyaa
andsukebei
- The output should show
acknowledged: true
twice - The safest bet is to disable the webapp here to ensure there's no database writes
- Run
python import_to_es.py
withSITE_FLAVOR
set tonyaa
- Run
python import_to_es.py
withSITE_FLAVOR
set tosukebei
- These will take some time to run as it's indexing
Setting up sync_es.py
- Sync_es.py keeps the ElasticSearch index updated by reading the BinLog
- Configure the MySQL options with the user where you granted the REPLICATION permissions
- Connect to MySQL, run
SHOW MASTER STATUS;
. - Copy the output to
/var/lib/sync_es_position.json
with the contents{"log_file": "FILE", "log_pos": POSITION}
and replace FILENAME with File (something like master1-bin.000002) in the SQL output and POSITION (something like 892528513) with Position - Set up
sync_es.py
as a service and run it, preferably as the system/root - Make sure
sync_es.py
runs within venv with the right dependencies
Enable the USE_ELASTIC_SEARCH
flag in config.py
, restart the application, and you're good to go.
Database migrations
- Uses flask-Migrate
- Run
./db_migrate.py db migrate
to generate the migration script after database model changes. - Take a look at the result in
migrations/versions/...
to make sure nothing went wrong. - Run
./db_migrate.py db upgrade
to upgrade your database.
Code Quality:
- Remember to follow PEP8 style guidelines and run
./lint.sh
before committing.