NyaaV2

Setup

Install dependencies https://github.com/pyenv/pyenv/wiki/Common-build-problems
Install pyenv https://github.com/pyenv/pyenv/blob/master/README.md#installation
Install pyenv-virtualenv https://github.com/pyenv/pyenv-virtualenv/blob/master/README.md
pyenv install 3.6.1
pyenv virtualenv 3.6.1 nyaa
pyenv activate nyaa
Install dependencies with pip install -r requirements.txt
Copy config.example.py into config.py
Change TABLE_PREFIX to nyaa_ or sukebei_ depending on the site

Enable USE_MYSQL flag in config.py
Install latest mariadb by following instructions here https://downloads.mariadb.org/mariadb/repositories/
- Tested versions: mysql Ver 15.1 Distrib 10.0.30-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2
Run the following commands logged in as your root db user:
- CREATE USER 'test'@'localhost' IDENTIFIED BY 'test123';
- GRANT ALL PRIVILEGES ON * . * TO 'test'@'localhost';
- FLUSH PRIVILEGES;
- CREATE DATABASE nyaav2 DEFAULT CHARACTER SET utf8 COLLATE utf8_bin;
To setup and import nyaa_maria_vx.sql:
- mysql -u <user> -p nyaav2
- DROP DATABASE nyaav2;
- CREATE DATABASE nyaav2 DEFAULT CHARACTER SET utf8 COLLATE utf8_bin;
- SOURCE ~/path/to/database/nyaa_maria_vx.sql

Run python db_create.py to create the database
Load the .sql file
- mysql -u user -p nyaav2
- SOURCE cocks.sql
- Remember to change the default user password to an empty string to disable logging in
Start the dev server with python run.py
When you are finished developing, deactivate your virtualenv with source deactivate

Install jdk sudo apt-get install openjdk-8-jdk
Install elasticsearch https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html
sudo systemctl enable elasticsearch.service
sudo systemctl start elasticsearch.service
Run curl -XGET 'localhost:9200' and make sure ES is running
Optional: install Kabana as a search frontend for ES

Add the [mariadb] bin-log section to my.cnf and reload mysql server
Connect to mysql
SHOW VARIABLES LIKE 'binlog_format';
- Make sure it shows ROW
Connect to root user
GRANT REPLICATION SLAVE ON *.* TO 'test'@'localhost'; where test is the user you will be running sync_es.py with

Run ./create_es.sh and this creates two indicies: nyaa and sukebei
The output should show acknowledged: true twice
The safest bet is to disable the webapp here to ensure there's no database writes
Run python import_to_es.py with SITE_FLAVOR set to nyaa
Run python import_to_es.py with SITE_FLAVOR set to sukebei
These will take some time to run as it's indexing

Sync_es.py keeps the ElasticSearch index updated by reading the BinLog
Configure the MySQL options with the user where you granted the REPLICATION permissions
Connect to MySQL, run SHOW MASTER STATUS;.
Copy the output to /var/lib/sync_es_position.json with the contents {"log_file": "FILE", "log_pos": POSITION} and replace FILENAME with File (something like master1-bin.000002) in the SQL output and POSITION (something like 892528513) with Position
Set up sync_es.py as a service and run it, preferably as the system/root
Make sure sync_es.py runs within venv with the right dependencies

Enable the USE_ELASTIC_SEARCH flag in config.py, restart the application, and you're good to go.

Uses flask-Migrate
Run ./db_migrate.py db migrate to generate the migration script after database model changes.
Take a look at the result in migrations/versions/... to make sure nothing went wrong.
Run ./db_migrate.py db upgrade to upgrade your database.