[HOW-TO] Headless Rapidshare Downloading for XBMC LIVE/Linux
#1
As a result to the conversation between Danz0l and I on this thread;
Jdownloader guide for xbmclive beta ?

I decided to put together a "How-To" to achieve Headless Downloading from Rapidshare and other filesharing sites on a XBMC Live/Linux Box. I will show you how to install the DownloadDaemon and also a WebUI for controlling the Downloads.

The Project homepage is at http://downloaddaemon.sourceforge.net/, I have no part in the development of this software but I am a user and a big fan.

Before we start you will need a webserver installed on your XBMC Box if you want the WebUI.

Skip this section if you already have a WebServer like Apache installed, just make sure you also have PHP installed+setup.
Google will help you.


Lighttpd WebServer + PHP Guide : Enter the following commands
Code:
sudo aptitude update
Code:
sudo aptitude install lighttpd php5-cgi
Code:
sudo lighty-enable-mod fastcgi
Code:
sudo /etc/init.d/lighttpd restart
To test the Server with PHP enter the following in the terminal
Code:
sudo nano /var/www/test.php
Paste in the following
Code:
<?php
phpinfo();
?>
Then Hit Ctrl+x and then y and then Enter.

Navigate in a browser to http://XBMC.IP.ADDRESS/test.php and you should see a sample webpage with your PHP version Number. This Means success, if not working then Check Google for Guide for Ubuntu Webserver with PHP.


Now let's begin installing DownloadDaemon: SSH into your XBMC Box, or get a Terminal by Ctrl+Alt+F1 and log in as main user (xbmc probably)


1. Add the proper PPA repo to our sources
Code:
sudo add-apt-repository ppa:agib/ppa

2. Update our Package List
Code:
sudo aptitude update

3. Install DownloadDaemon
Code:
sudo aptitude install downloaddaemon

4. Make Temp Directory for WebUI install and Move to Temp Directory
Code:
mkdir ~/temp && cd ~/temp

5. Download WebUI
Code:
wget http://sourceforge.net/projects/downloaddaemon/files/Version%201.0/ddclient-php/ddclient-php-1.0.1.tar.gz

6. untar WebUI
Code:
tar xvfz ddclient-php-1.0.1.tar.gz

7. Copy WebUI to webserver directory
Code:
sudo cp -R ddclient-php-1.0.1/ddclient-php /var/www/downloaddaemon/

8. Tidy up after ourselves!!
Code:
cd ~ && rm -R temp

9. Reboot
Code:
sudo reboot

After reboot use a browser to go to http://XBMC.IP.ADDRESS/downloaddaemon

The default password is blank, which can then be changed through the WebUI.
Login info for Premium Rapidshare etc. can be configured through the WebUI.
The WebUI is fairly self explanitory.


Another option is to also download a client for another Computer (Windows) that will allow you add/remove/monitor downloads by connecting to the Daemon. This can be done from the Project Homepage (version 1.0rc2 is the newest windows client version and works perfectly)

All other info from the Project Homepage linked above, The Project forums are a great help.

Please consider donation if you like/use the software. (again I genuinely am not involved in the project!!)

PS: I am a regular Linux user not overly experienced so I may not be able to help with all issues, but post here and I will try and help.

The Capt.
Reply
#2
running xbmc-live i am getting this fail message


Code:
Setting up lighttpd (1.4.26-1.1ubuntu3) ...
Syntax OK
* Starting web server lighttpd                                                                                                                              2010-10-06 11:59:42: (network.c.345) can't bind to port:  80 Address already in use
                                                                                                                                                      [fail]
invoke-rc.d: initscript lighttpd, action "start" failed.

-=Jason=-
Reply
#3
Flomaster Wrote:running xbmc-live i am getting this fail message


Code:
Setting up lighttpd (1.4.26-1.1ubuntu3) ...
Syntax OK
* Starting web server lighttpd                                                                                                                              2010-10-06 11:59:42: (network.c.345) can't bind to port:  80 Address already in use
                                                                                                                                                      [fail]
invoke-rc.d: initscript lighttpd, action "start" failed.

-=Jason=-

Try Command
Code:
sudo lsof -i :80
to list all processes running on port 80.

I am running this successfully on a xbmc-standalone set-up, does anyone know if LIVE has a server already using port 80?
Reply
#4
Capt. looks like Dropbox is running on port 80, Drop box as also been making my hard drive click like crazy from Read/Write for some odd reason so I have been killing its process.

this dropbox is more trouble on linux than its worth.
Code:
xbmc@XBMCLive:~$ sudo lsof -i :80
COMMAND  PID     USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
dropbox 1153     xbmc   17u  IPv4   6742      0t0  TCP XBMCLive.local:42138->174.36.30.43-static.reverse.softlayer.com:www (ESTABLISHED)
apache2 1538     root    3u  IPv4   6159      0t0  TCP *:www (LISTEN)
apache2 1545 www-data    3u  IPv4   6159      0t0  TCP *:www (LISTEN)
apache2 1546 www-data    3u  IPv4   6159      0t0  TCP *:www (LISTEN)
xbmc@XBMCLive:~$


-=Jason=-
Reply
#5
Flomaster Wrote:Capt. looks like Dropbox is running on port 80, Drop box as also been making my hard drive click like crazy from Read/Write for some odd reason so I have been killing its process.

this dropbox is more trouble on linux than its worth.
Code:
xbmc@XBMCLive:~$ sudo lsof -i :80
COMMAND  PID     USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
dropbox 1153     xbmc   17u  IPv4   6742      0t0  TCP XBMCLive.local:42138->174.36.30.43-static.reverse.softlayer.com:www (ESTABLISHED)
apache2 1538     root    3u  IPv4   6159      0t0  TCP *:www (LISTEN)
apache2 1545 www-data    3u  IPv4   6159      0t0  TCP *:www (LISTEN)
apache2 1546 www-data    3u  IPv4   6159      0t0  TCP *:www (LISTEN)
xbmc@XBMCLive:~$


-=Jason=-

You also already have Apache installed, so you have a WebServer already. lighttpd is just another WebServer, so use Apache instead if you have it installed.
Reply
#6
I use rapidleech on Archlinux i use it to download from megaupload on headless sever.
It comes with this plugins.

Code:
2shared.com
4shared.com
adrive.com
anime-eden.com
badongo.com
bigandfree.com
bitroad.net
btaccel.com
depositfiles.com
dl.free.fr
downtown.vc
easy-share.com
egoshare.com
esnips.com
evilshare.com
fileadded.com
filefactory.com
fileflyer.com
filego.net
files.to
filesend.net
freakshare.net
furk.net
gigasize.com
hotfile.com
ifile.it
ifolder.ru
imageshack.us
kewlshare.com
letitbit.net
mediafire.com
megaporn.com
megashare.com
megashares.com
megaupload.com
megavideo.com
midupload.com
netload.in
odsiebie.com
oron.com
rapidshare.com
rapidshare.de
savefile.com
saveqube.com
sendspace.com
share-now.net
share-online.biz
sharebase.to
sharebomb.com
sharedzilla.com
sharingmatrix.com
simpleupload.net
sourceforge.net
speedyshare.com
stagevu.com
storage.to
supershare.pl
turbobit.net
turboupload.com
ul.to
uploadbox.com
uploaded.to
uploading.com
uploads.bizhat.com
video.google.com
vip-file.com
x7.to
youload.to
youtube.com
ziddu.com
zippyshare.com
zshare.net
Reply
#7
Capt.Insano Wrote:You also already have Apache installed, so you have a WebServer already. lighttpd is just another WebServer, so use Apache instead if you have it installed.

I just did the test.php thing and went to my ipaddress/test.php and google chrome asked me if I wanted to download test.php I said yes and it did. not sure if thats the correct function or not.

-=Jason=-
Reply
#8
Flomaster Wrote:I just did the test.php thing and went to my ipaddress/test.php and google chrome asked me if I wanted to download test.php I said yes and it did. not sure if thats the correct function or not.

-=Jason=-

Google Ubuntu Apache + PHP setup, it sounds like you have Apache installed but not PHP installed + Setup.
Reply
#9
sweet I have it working..

I did notice on step 5 I had to remove the / at the end of the command or it downloaded a index.htm file and step 7 I had to add "sudo" to the front of the command.

-=Jason=-
Reply
#10
Flomaster Wrote:sweet I have it working..

I did notice on step 5 I had to remove the / at the end of the command or it downloaded a index.htm file and step 7 I had to add "sudo" to the front of the command.

-=Jason=-

OP updated!!

Thanks,
The Capt.
Reply
#11
Great guide capt.

I'll add to this on how to compile from source using the SVN (as the PPA repository is very behind).

Downloading, Compiling and Installing Downloaddaemon from source.

Ok to download and compile Downloaddaemon from the SVN branch you will likely need a bunch of packages (in XBMC Live anyways). So lets first install subversion and some compiler related programs.

Quote:sudo apt-get install build-essential g++ libcurl4-gnutls-dev cmake subversion

Ok we should be good to go. Now we need to check out the trunk sourcecode of Downloaddaemon. First create a directory to store the source.

Quote:mkdir svn && mkdir svn/downloaddaemon

Ok brilliant, lets get subversion to download the latest trunk code to our computer, ready to compile, into that directory.

Quote:svn co https://downloaddaemon.svn.sourceforge.n...emon/trunk

That will download the latest downloaddaemon revision to our harddrive ready to compile.

Now lets change into that directory and make a build directory

Quote:cd svn/downloaddaemon && mkdir build && cd build

Ok if all goes well we should now be happily sitting in our build directory ready to go.

Lets configure the source for your machine:

Quote:cmake ../src/daemon

If errors come up, then its likely your missing some packages. Check the errors and see what package you will need to install.

Ok, so we have managed to download the latest downloaddaemon trunk from SVN using subversion, created all the necessary directories and managed to configure it for our machine. All that is left for us to do now is compile it and install. So lets build the daemon

Quote:make

If all goes well (and it should), the only thing left to do is to install our nice newly built program.

Quote:sudo make install


NOTE : This will overwrite your configuration files if you have had downloaddaemon running before (likely from the PPA repository). so if you need them, please back them up before running the above command.

To do that just enter

Quote:mkdir /etc/downloaddaemon/backup && cp /etc/downloaddaemon/* /etc/downloaddaemon/backup

All your backed up conf files are now stored in a sub directory called backup of the main downloaddaemon config directory.

To restore them just type

Quote: cp /etc/downloaddaemon/backup1/* /etc/downloaddaemon/


Also note that ownership of the etc/downloaddaemon directory with revert to the downloadaemon user and you will need to use chmod/chown commands so you can run DD as your user again. Check out running downloaddaemon as another user on how to do this.

One further thing to note when compiling from source is that downloaddaemon will now be located in /usr/local. If you originally installed from the repository, downloaddaemon will be installed to /usr, which isn't likely the best place for it. Ironically this means we can keep the stable version installed whilst checking out our latest compiled version, in the knowledge that, if things don't work as we had expected, we can always revert to the older version from the repository.

I renamed my compiled version so i knew which version i was running to DownloadDaemon-trunk (remember to update your init.d scripts to reflect autostarting the newer version).

***** This guide was posted by the downloaddaemon author Adrian for me, on the sourceforge forums, so i take little credit for it ******
-= My Media Centre =-
ACER Revo R3610 | x2 1.5TB WD Caviar Green SATA Hard Drives (WD15EADS) | x2 2.0TB WD Caviar Green SATA Hard Drive ( WD15EARS ) | 3.5in Black eSATA USB Trayless SATA External Hard Drive Enclosure | Emprex 3009URF Wireless MCE Remote Control | XBMCLive Dharma Beta1
Reply
#12
Running DownloadDaemon as another User
Downloaddaemon runs, as default, using a downloadd user account. This is all very well and good but i have my files ownerships all set up for my xbmc user so when i try moving files via samba i don't have ownership rights to alter their locations, delete, rename etc. So very annoying. Thankfully Downloaddaemon can be made to run using any user you like. Heres how to get it running with our default xbmc user (change xbmc to the user you want to use).

Ok first and foremost lets stop the downloaddaemon daemon so we can alter who it runs under.

Quote:sudo /etc/init.d/downloadd stop

excellent, now lets edit the init.d scripts to reflect we want to run under a different user

Quote:sudo nano /etc/init.d/downloadd

What we are wanting to do is find the instances of when Downloaddaemon is invoked (run) and tell it to run using our user instead. This is achieved by appending the -u switch when we invoke DownloadDaemon with DownloadDaemon -d

-u myuser (where myuser is the user name you want downloaddaemon to run under. in my case this was xbmc )

So go through the file and find all instances of DownloadDaemon -d and append -u xbmc to it. so it looks like

DownloadDaemon -d -u xbmc

I've marked the areas in bold where you need to add this switch (or replace the ones that are there).

Code:
#! /bin/sh
### BEGIN INIT INFO
# Provides:          DownloadDaemon
# Required-Start:    $network
# Required-Stop:     $network
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: DownloadDaemon download manager service
### END INIT INFO

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=DownloadDaemon
NAME=DownloadDaemon
DESC="DownloadDaemon download manager service"

case "$1" in
  start)
    if [ "$(id -u)" != "0" ]; then
           echo "This script must be run as root" 1>&2
           exit 1
    fi
    DownloadDaemon -d [b][color=Red]-u xbmc[/color][/b] 2> /dev/null
    ;;
  stop)
    if [ "$(id -u)" != "0" ]; then
           echo "This script must be run as root" 1>&2
           exit 1
    fi
    killall DownloadDaemon 2> /dev/null
    ;;
  reload|force-reload)
       echo "Unsupported option"
       ;;
  restart)
    if [ "$(id -u)" != "0" ]; then
           echo "This script must be run as root" 1>&2
           exit 1
    fi
    killall DownloadDaemon 2> /dev/null
    DownloadDaemon -d [b][color=Red]-u xbmc[/color][/b] 2> /dev/null
    ;;
  status)
    echo "Status of DownloadDaemon:"
    if [ -z `pidof DownloadDaemon` ]; then
        echo "$NAME is not running."
        exit 3
    else
        echo "$NAME is running."
        exit 0
    fi
    ;;
  *)
    N=/etc/init.d/${0##*/}
    echo "Usage: $N {start|stop|restart|status}" >&2
    exit 1
    ;;
esac

exit 0

Once you have done that save the file and exit, CTRL O, CTRL X

Great, we are almost done.

The only other thing we need do is change the ownership of the downloaddaemon config directory before we run the porgram under our new user.

Quote:chown -R xbmc:downloadd /etc/downloaddaemon

(again replce xbmc with the user you wish to have ownership).

Thats it, thats all thats needed.

Now just restart the downloaddaemon under our new user

Quote:sudo /etc/init.d/downloadd start

Any new files downloaded now will have the rights of our new user, which means i can now copy, replce, rename, move them Smile
-= My Media Centre =-
ACER Revo R3610 | x2 1.5TB WD Caviar Green SATA Hard Drives (WD15EADS) | x2 2.0TB WD Caviar Green SATA Hard Drive ( WD15EARS ) | 3.5in Black eSATA USB Trayless SATA External Hard Drive Enclosure | Emprex 3009URF Wireless MCE Remote Control | XBMCLive Dharma Beta1
Reply
#13
danz0l Wrote:Great guide capt.

I'll add to this on how to compile from source using the SVN (as the PPA repository is very behind).

PPA is now up to date, since 5th Oct.

No need for SVN!
Reply
#14
Capt.Insano Wrote:PPA is now up to date, since 5th Oct.

No need for SVN!

Thats good to know anyways Smile
-= My Media Centre =-
ACER Revo R3610 | x2 1.5TB WD Caviar Green SATA Hard Drives (WD15EADS) | x2 2.0TB WD Caviar Green SATA Hard Drive ( WD15EARS ) | 3.5in Black eSATA USB Trayless SATA External Hard Drive Enclosure | Emprex 3009URF Wireless MCE Remote Control | XBMCLive Dharma Beta1
Reply
#15
some thing happened Sad

I can access via http://myiphere/downloaddaemon but when I try http://myDynDns.net/downloaddaemon it doesn't work Sad what gives

-=Jason=-
Reply

Logout Mark Read Team Forum Stats Members Help
[HOW-TO] Headless Rapidshare Downloading for XBMC LIVE/Linux0