You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
aldacron f5551dd1e5 moved some files 6 years ago
.github Add issue template (and remind people we know about the search thankyouverymuch) 6 years ago
configs moved some files 6 years ago
nyaa updated MOTD for new search awareness 6 years ago
torrent_cache Initial commit. 6 years ago
utils moved some files 6 years ago
.gitignore Make sure torrent backup directory exists before writing torrent 6 years ago
LICENSE Add license (GPLv3) 6 years ago
README.md hooked up ES... 90% done, need to figure out how to generate magnet URIs 6 years ago
WSGI.py Initial commit. 6 years ago
config.example.py hooked up ES... 90% done, need to figure out how to generate magnet URIs 6 years ago
create_es.sh hooked up ES... 90% done, need to figure out how to generate magnet URIs 6 years ago
db_create.py Ghetto migrate solution, in case you re-run db_create, it won't add categories again 6 years ago
es_mapping.yml Reverted previous commit for mapping 6 years ago
import_to_es.py added timeout to import and sync es 6 years ago
lint.sh Initial commit. 6 years ago
requirements.txt hooked up ES... 90% done, need to figure out how to generate magnet URIs 6 years ago
run.py Initial commit. 6 years ago
sync_es.py added timeout to import and sync es 6 years ago
trackers.txt removed explodie as suggested 6 years ago
uwsgi.ini Initial commit. 6 years ago

README.md

NyaaV2

Setup:

  • Create your virtualenv, for example with pyvenv venv
  • Enter your virtualenv with source venv/bin/activate
  • Install dependencies with pip install -r requirements.txt
  • Run python db_create.py to create the database
  • Start the dev server with python run.py

Updated Setup (python 3.6.1):

Setting up MySQL/MariaDB database for advanced functionality

  • Enable USE_MYSQL flag in config.py
  • Install latest mariadb by following instructions here https://downloads.mariadb.org/mariadb/repositories/
    • Tested versions: mysql Ver 15.1 Distrib 10.0.30-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2
  • Run the following commands logged in as your root db user:
    • CREATE USER 'test'@'localhost' IDENTIFIED BY 'test123';
    • GRANT ALL PRIVILEGES ON * . * TO 'test'@'localhost';
    • FLUSH PRIVILEGES;
    • CREATE DATABASE nyaav2 DEFAULT CHARACTER SET utf8 COLLATE utf8_bin;
  • To setup and import nyaa_maria_vx.sql:
    • mysql -u <user> -p nyaav2
    • DROP DATABASE nyaav2;
    • CREATE DATABASE nyaav2 DEFAULT CHARACTER SET utf8 COLLATE utf8_bin;
    • SOURCE ~/path/to/database/nyaa_maria_vx.sql

Finishing up

  • Run python db_create.py to create the database
  • Load the .sql file
    • mysql -u user -p nyaav2
    • SOURCE cocks.sql
    • Remember to change the default user password to an empty string to disable logging in
  • Start the dev server with python run.py
  • Deactivate source deactivate

Enabling ElasticSearch

Basics

Enable MySQL Binlogging

  • Add the [mariadb] bin-log section to my.cnf and reload mysql server
  • Connect to mysql
  • SHOW VARIABLES LIKE 'binlog_format';
    • Make sure it shows ROW
  • Connect to root user
  • GRANT REPLICATION SLAVE ON *.* TO 'test'@'localhost'; where test is the user you will be running sync_es.py with

Setting up ES

  • Run ./create_es.sh and this creates two indicies: nyaa and sukebei
  • The output should show akncolwedged: true twice
  • The safest bet is to disable the webapp here to ensure there's no database writes
  • Run python import_to_es.py with SITE_FLAVOR set to nyaa
  • Run python import_to_es.py with SITE_FLAVOR set to sukebei
  • These will take some time to run as it's indexing

Setting up sync_es.py

  • Sync_es.py keeps the ElasticSearch index updated by reading the BinLog
  • Configure the MySQL options with the user where you granted the REPLICATION permissions
  • Connect to MySQL, run SHOW MASTER STATUS;.
  • Copy the output to /var/lib/sync_es_position.json with the contents {"log_file": "FILE", "log_pos": POSITION} and replace FILENAME with File (something like master1-bin.000002) in the SQL output and POSITION (something like 892528513) with Position
  • Set up sync_es.py as a service and run it, preferably as the system/root
  • Make sure sync_es.py runs within venv with the right dependencies

Good to go!

  • After that, enable the USE_ELASTIC_SEARCH flag and restart the webapp and you're good to go

Code Quality:

  • Remember to follow PEP8 style guidelines and run ./lint.sh before committing.