Wednesday, 11 November 2015

Newznab

Apologies if I've gone through this before, but I really wanted to start having a tidy up of my instructions and also wanted to to follow them to put things back from scratch. This time, I'm going to build a newsgroup indexer with NewzNab.

Now that I've offloaded my cloud server to a Raspberry Pi, I'm going to go and re-install Newznab onto my old broken Toshiba laptop. The laptop works well enough until you give it a graphical display, looks like something is up with the display chip but it's not needed here. The OS I'm going to use is Ubuntu server as it just seems to work.

All this work is done via SSH from a Linux desktop, you can just copy and paste the commands into a terminal window. First thing we need to do is run updates:

sudo apt-get update
sudo apt-get upgrade

Now let's make the directory structure:

sudo mkdir /var/www
sudo mkdir /var/www/newznab
sudo chmod 0777 /var/www/newznab

Now it's time to install php:

sudo apt-get install php5 php5-dev php-pear php5-gd php5-mysql php5-curl


Now we make some changes to the php configuration file:

sudo nano /etc/php5/cli/php.ini


Under the resource limits section we need to change the maximum execution time to 120:

max_execution_time = 120

Then under Module Settings we need to change the time zone settings, in my case I'm the UK so I'm going to change it to:

date,timezone = Europe/London

There are other options depending on where you live, you can find these here.

Now it's SQL server time with:

sudo apt-get install mysql-client-5.5 libmysqlclient-dev apache2 mysql-server-5.5

You will be asked to create a password for the SQL server, make sure it's a good strong one, then we edit another file.


sudo nano /etc/php5/apache2/php.ini


Under the resource limits section we need to change the maximum execution time to 120:

max_execution_time = 120

and


memory_limit = -1  


Then under Module Settings we need to change the time zone settings, in my case I'm the UK so I'm going to change it to:

date,timezone = Europe/London


Once you've saved that file create a new one with:

sudo nano /etc/apache2/sites-available/newznab.conf

And paste the following into it:

<VirtualHost *:80>
ServerAdmin webmaster@localhost
ServerName localhost

DocumentRoot /var/www/newznab/www
ErrorLog /var/log/apache2/error.log
LogLevel warn
</VirtualHost>

Change the port if you want it to run on a different port.

One more change to the Apache configuration file before we go on:

sudo nano /etc/apache2/apache2.conf

Find this bit

<Directory /var/www/>
        Options Indexes FollowSymLinks
        AllowOverride None
        Require all granted
</Directory>

And change it to this:

<Directory /var/www/>
        Options Indexes FollowSymLinks
        AllowOverride All
        Require all granted
</Directory>


We now need to get hold of Newznab itself, you can get it from here. Newznab classic works but there are some limitations, if you are serious just pay the money for a better product and you get free upgrades too.

Get the file downloaded and then we need to extract it and copy it to the right place, 

As I have a paid for version I'm going to use a slightly different approach, for this you need to install subversion with the following command:

sudo apt-get install subversion

Then to get Newznab you enter the following command:

sudo svn co svn://svn.newznab.com/nn/branches/nnplus /var/www/newznab

At this point you will be asked for the root password for your system and the SVN login details that have been emailed to you by the Newznab team.

Now it's time to activate the changes:

sudo a2ensite newznab
sudo a2dissite 000-default
sudo a2enmod rewrite
sudo service apache2 restart

And now it's time to set permissions on the folders:


sudo chmod 777 /var/www/newznab/www/lib/smarty/templates_c
sudo chmod 777 /var/www/newznab/www/covers/movies
sudo chmod 777 /var/www/newznab/www/covers/music
sudo chmod 777 /var/www/newznab/www/covers/tv
sudo chmod 777 /var/www/newznab/www
sudo chmod 777 /var/www/newznab/www/install
sudo chmod 777 /var/www/newznab/nzbfiles/
sudo chmod 777 /var/www/newznab/www/covers/anime


We need to install a few more applications:

sudo apt-add-repository ppa:kirillshkrogalev/ffmpeg-next
sudo apt-get update
sudo apt-get install unrar mediainfo lame ffmpeg


When you run the Newznab config and go in to edit the site you will need to put the paths in for these files, they normally are:

/usr/bin/unrar
/usr/bin/mediainfo
/usr/bin/ffmpeg
/usr/bin/lame


And now point your browser to 

http://servername/install 

And just follow the instructions and prompts to configure your very own Newznab server.

Once you have finished the comfiguration, run this following command to change permissions on one of the folders.

sudo chmod 777 /var/www/newznab/nzbfiles/tmpunrar

Big thanks to the howtogeek website for help with these instructions, I've added a few bits and pieces to get to this stage.

Now that we have made it to this stage it's time to finish off the setting up and then we can add some newsgroups and get the server scanning these for updated files.

Go to the edit site option from your browser and fill in the information required, including the paths for the files we installed earlier such as unrar etc.
You also need to enter the newznab ID you got when you bought the software.
I want it to check and delete password protected releases.
And finally, I want to enable spotnab, just fill in the gaps with your information, it doesn't seem to want you to create any accounts.

Once you've done all that, you can save the settings and click on bulk add groups, in the group list box type *binaries* and in the active box select no. This will populate Newznab with the available binary groups but won't download anything from them initially.
Now click on "View Groups" and type in "Television" into the search box, click on go and it will display the name of all groups with the word "Television" in them. Click on "Activate" to activate the ones you want to scan and then we go back to the command line.

First we change to root with:

sudo -i

Then we change to the script directory with:

cd /var/www/newznab/misc/update_scripts

Then we run:

php update_binaries_threaded.php

This downloads all the updates from the active groups, followed by:

php update_binaries_releases.php

We should now have some files in newznab and we can think about adding some new groups, but we really need to get this process running on startup. So we can create a script in /usr/bin with the following command:

sudo nano /usr/bin/newznab

Paste the following text into it:


cd /var/www/newznab/misc/update_scripts

php update_binaries_threaded.php

php update_binaries_releases.php

Save it and run:

sudo chmod +x /usr/bin/newznab


So far the script looks like this:

cd /var/www/newznab/misc/update_scripts
php update_binaries_threaded.php
php update_binaries_releases.php

This means it's going to run once and then quit, so we need to make it loop back to the start once it's finished.



while :

do

cd /var/www/newznab/misc/update_scripts/

echo "Optimising Database"
php optimise_db.php

echo "Updating TV Schedule"
php update_tvschedule.php

echo "Updating Binaries"
php update_binaries_threaded.php

echo "Updating Releases"
php update_releases.php


echo "Resting"

        echo "Press [CTRL+C] to stop.."
        sleep 30
done

This will now continue until you press Control + C, the only thing is that you need to manually start it from the command line when you want to run it.


If you want it to run on startup, add it to the /etc/rc.local file as follows

sudo nano /etc/rc.local

Add a line before exit at the bottom that says

/usr/bin/newznab &

Save the file and then make it executable with:

sudo chmod +x /etc/rc.local

Finally, test it by running sudo /etc/rc.local

The only problem with this method is that there's no easy way to break the cycle.

I'm open to suggestions for better options, in the meantime, have fun with it.

By the way, you will need to have an account with a usenet provider, I tend to stick with Newsdemon as they are quite reliable,