Win Any way to use XBMC as automated MySql scaper? (Headless?)
#1
I've been trying to find an appropriate solution for days and haven't found anything interesting.

There are software that will permit to alter XBMC's content database, I have very mixed results when it comes to use XBMC's MySql database. Plus, no software that I know of use XBMC's scrapers. XBMC itself with its plugins does an amazing job managing the media database, but when you start spreading XBMC boxes here and there and use a centralized database, new problematics emerges.

What I'd like to do is automate the update/scraping process on the server side.

I would like the server to detect new content and launch an XBMC update automatically.
We could use EventGhost to detect folder change, and it could launch XBMC commands to update, but I haven't found a way to run XBMC on the server.

For starters XBMC won't run as a service. you can trick it into running into the tray...
http://forum.xbmc.org/showthread.php?tid=43646

but then again, XBMC won't launch within RemoteDesktop, and none of the VNC alternatives I tried worked.
http://forum.xbmc.org/showthread.php?tid=124431

Ideally, we should be able to configure and run XBMC as a standalone non-video application that remains there for remote access, this way it could be used with XWMM to serve as a library editor
http://forum.xbmc.org/showthread.php?tid=60643

plus, as mentioned, the detection and update process could be automated server-side. So when I rip my DVD on my local computer and push it to my SBM drive, the server would automatically add it and the living room htpc could play it with very little interference.

However, I still haven't a way to run my XBMC on my no-screen media server to achieve that.
It there a solution for that?
Perhaps an effective mySql scrapping software?
I've tried XBNE and discarded it, too much crap with my movie sets, too much bad data from its scraping methodology. I've tried Ember Media Manager but it sucks when it comes to be paired with mysql. I'm out of ideas.
Reply
#2
(2013-01-20, 18:32)Marsupial Wrote: - What I'd like to do is automate the update/scraping process on the server side.

- plus, as mentioned, the detection and update process could be automated server-side. So when I rip my DVD on my local computer and push it to my SBM drive, the server would automatically add it and the living room htpc could play it with very little interference.

-Ideally, we should be able to configure and run XBMC as a standalone non-video application that remains there for remote access, this way it could be used with XWMM to serve as a library editor
There is a auto updater plugin for XBMC that allows you to set interval that XBMC runs a full db update. XMBC updater i think its called and there is also watchdog. Then you just need to have XWMM on one machine...no need to have a a xbmc standalone just for that.

(2013-01-20, 18:32)Marsupial Wrote: - For starters XBMC won't run as a service. you can trick it into running into the tray...

- However, I still haven't a way to run my XBMC on my no-screen media server to achieve that.
It there a solution for that?
why would you need this?

(2013-01-20, 18:32)Marsupial Wrote: but then again, XBMC won't launch within RemoteDesktop, and none of the VNC alternatives I tried worked.
Have you tried Teamviewer? it works for me and i even use it to connected to my sisters PC at her house to help her with XBMC problems. There is a popup at the end using the free version but if you serach the forum there is a post on how to get rid of it.

(2013-01-20, 18:32)Marsupial Wrote: I've tried XBNE and discarded it, too much crap with my movie sets, too much bad data from its scraping methodology. I've tried Ember Media Manager but it sucks when it comes to be paired with mysql. I'm out of ideas.

There are many managers out there http://wiki.xbmc.org/index.php?title=Supplemental_tools
Image

If my replies help you, please click on my reputation Image below :) thanks :)
Reply
#3
Thanks for offering potential solutions, but putting a band-aid on a problem isn't solving it.
The best solution would really be to have a display-less XBMC machine running all its scraping and remote services. I know XBMC isn't a server software, but what better than XBMC to handle XBMC files and scrapers?

None of the media managers works correctly once you have a mySql database. Even MediaElch which actually promises it. Plus, I always wonder... why should we "have" to rely on third party software when the main software does such a great job to begin with?
The media managers don't have appropriate mySQL support (already stated) and don't use XBMC's scraper (already stated as well) Some are very interesting, but their featureset isn't right. Some even damaged my media library and required me to do further work on it. I'd really like to just use XWMM for my minute-alterations, and a server side solution for the biggest part.

I have already tried: XBNE, EMM, MediaElch, SickBeard and MediaCompanion. I discarded all of them.


Further thoughts... using a scheduled update has drawback. Lots of drawback and no actual benefit from my point of view. First, it is NOT the same as event-driven updates. Let say I rip a DVD to be played right away, the "next update" is kind of too far.
Plus, either the update "disturb" someone watching, or you have to schedule it by-night and keep the box 24/7. Hey, the server DOES run 24/7... why not there?
Plus, what happens to someone watching something on the machine you use for updates when you need to do a cleanup because you're moving things around? Things like this happens. Using the front-end machine for back-end management is NOT an appropriate solution in my book.

as far as Teamviewer... I just finished uninstalling it. Today I also tried ultraVNC and TightVNC. TightVNC was the most promising, but XBMC wouldn't start at all, it crashed with the same failure you get when you don't have your videocard drivers (can't instanciate the graphics application)

Teamviewer I could see, but couldn't control the PC at all so I gave up on it.

Seriously, why isn't XBMC capable of running a diet version for management features? Would make a ton of sense... Using a workhorse machine (server) to do the work is a "normal" philosophy, and using lighter "viewer" machines that has no other interaction than reading the database generally makes sense.


something like that
http://forum.xbmc.org/showthread.php?tid=133434

but for windows.
Reply
#4
This might be a solution... a non-official patched version of XBMC eden intended to expost the headless mode
http://forum.xbmc.org/showthread.php?tid=132919

anywhere to get a windows executable of such patched XBMC Eden?
Reply
#5
Well i use NZBDrone (program like Sickbeard) to manage all my TVshows and Couchpotato to mange my movies and NZBdrone finds my shows , sends them to Sabnzb, show get downloaded, NZBdrone, then renames it, moves it, and sends an update to XBMC and the show appears...im using MySQL. Couchpotato is the same...all this running on my server. No human intervention, no worries if people are watching TV. As for Teamviewer, how are you not able to control? You can alt-tab or send show desktop command if need be. I think DVD ripping and updating XBMC could also work with what i use if files are named properly for the applications to recognize them and there are place in the monitored folders to be picked up by the app but since i dont rip stuff ever , i cant really confirm.

Image

If my replies help you, please click on my reputation Image below :) thanks :)
Reply
#6
Well, I am glad your setup works for you and fulfill your needs.

The question remains unanswered. Is there any way to use XBMC without its GUI as part of automated scraper facility?
Reply
#7
do you absolutely need it without GUI, will it not run on your current server hardware? (edit, it seems its more of a configuration/use issue as your running headless, correct?)

Im assuming you have a normal use pc, for work or porn or something as you have a headless server and media center in at least 1 room. why dont you have 3 (or more) xbmc installs. 1 on the server, 1 on your work/porn box and 1 on the media center... you can use the work machine to configure everything... push the settings to the server and your media box, and if you have to do any library mods, do it on the workstation, and with mysql backend, it will sync to the server and the media box. you can also run manual updates from any of the boxes using the JSON interface, and you can setup a 15 minute auto scan on the server..

also, if your having issues with it installed on your server, you may want to create a virtual machine to run xbmc (or that headless mod you have posted above), and give it just enough resources to get through scanning. i have done this with virtual box, and although it will not play videos without chop and pauses, i can use vnc viewer to remote in and configure xbmc and let it do its thing. granted, i have a basic video card on my server, so i dont have the "can't instanciate the graphics application" issue.

what os is your server, assuming windows server 2003?
Reply
#8
No there is no "XBMC Server Mode" or "Headless Mode" - there has been plenty of talk of perhaps creating one.

However the server doesn't have to be the one doing the scraping, any XBMC machine connected to the mysql server can initiate it.


If I have helped you or increased your knowledge, click the 'thumbs up' button to give thanks :) (People with less than 20 posts won't see the "thumbs up" button.)
Reply
#9
The json-rpc api has a method to initiate a scan, see http://wiki.xbmc.org/index.php?title=JSO...brary.Scan - so you could:

1. set your video ripping (downloading?) script to initiate a scan via any of the json-rpc transports (python, http etc); or

2. write a script for your server to periodically scan your media folders for new content, when new content is seen, initiate a scan. It looks like nzbdrone may already implement this.

So your XBMC running in the tray never needs to be accessed over remote access, it can be automatically told to run a scan without you accessing the gui at all.

Alternatively do away with XBMC on the server and designate one of your XBMC machines as always on, so you can initiate the scan on that machine. Maybe this machine could be a low power machine like RPi?

The other approach is to manipulate the mysql database outside xbmc. This is how mythtv scrapers work - you can run them from the commandline and they insert the appropriate info into mythtv's mysql database, and download the pretty stuff. I am not sure how any of those scripts could be adapted to xbmc, and the approach of course runs the risk that changes ot the database structure render your scraping script redundant. (A well thought out and abstracted api may solve this).
If I have helped you or increased your knowledge, click the 'thumbs up' button to give thanks :) (People with less than 20 posts won't see the "thumbs up" button.)
Reply
#10
I agree with the OP a small library management version to run on my media server and scan the library on any changes would be awesome and with the development of advanced upnp a included advanced upnp server would top it off nicely. If I had any coding ability at all I would be all over this unfortunately I couldn't code my way out of a paper bag. It would have to run as a service as I never log on to my server.
If I have been of help, please add to my reputation as a way of saying thanks, it's free.
Reply
#11
Thanks a lot for the support, Dilligaf. I too never log directly on my server. In fact, there's no monitor, and only the onboard video card. Which sucks big time.
So, to answer el_cabong: yes, I need it absolutely without GUI, or at least a version that will instanciate without crashing because of the GUI. As it is right now, I simply cannot run it on the server.

The server is currently running XP pro. I own no "server" licences so I am stuck with Win7 or WinXP. WinXP does the trick nicely once you configure properly.

Q: Can we trick a virtual machine into thinking of a GPU that would let XBMC run? That would be an awesome solution!!!
Nickr, I've been trying to find out applications that would alter the SQL database directly. That would be half-way-there.
BTW I am still on Eden. Should I jump to Frodo?

I would really like to avoid having to patch out by using "another PC" to do the work that should be done by the server. Because, let's face it, the only reason it would make sense to use "another machine" to process the database, is if we had no other way around. Its a plaster on a bleeding wound, not a proper solution.

Reply
#12
@Marsupial I can't work out exactly why you can't get xbmc to run on your server. It surely has a graphics acard, and it shouldn't matter that there is nothing connected to the graphics card.

XBMC will run in vmware - well it will on my laptop, xbmcbuntu running on linux host.

But you can trigger any xbmc machine on your lan to do the scan, just leave your lowest power consuming client on 24/7.

If I have helped you or increased your knowledge, click the 'thumbs up' button to give thanks :) (People with less than 20 posts won't see the "thumbs up" button.)
Reply
#13
As stated before I'm no coder so I could be totally off base but it would seem to me that one could take the existing code and rip out all the bits that aren't needed leaving just the core scraper and database management, so rather than writing code it would be removing code. I would be more than willing to make a "donation" to anyone willing to do it..
If I have been of help, please add to my reputation as a way of saying thanks, it's free.
Reply
#14
http://forum.xbmc.org/showthread.php?tid=129402 also relevant to triggering a scan.
If I have helped you or increased your knowledge, click the 'thumbs up' button to give thanks :) (People with less than 20 posts won't see the "thumbs up" button.)
Reply
#15
@Marsupial: See my post 326 and 328 in THIS thread for running xbmc in an ESXI guest w/o hardware accelerated graphics.


Reply

Logout Mark Read Team Forum Stats Members Help
Any way to use XBMC as automated MySql scaper? (Headless?)0