There is a
new version available with first experimental scraping functionality (not tested on all systems, not documented in the Wiki). It can download game descriptions and images from online sources. Parsing game descriptions can be done with the known flatfile or xml parserConfig. You can add multiple (online and offline) scrapers to one RomCollection to collect data from different sources. RCB will mix the content of all sources (first come, first go). Note that multiple scrapers are only supported in scenarios that use 1 description file per game.
So, this was the good news, now the bad one: This all has to be configured. I implemented this functionality with the old configuration scheme (config.xml + parserConfig) and plan to rework configuration with one of the next releases. If this all works correct in the future it will ease up collecting data but with this first shot it will again add some extras to RCBs configuration files.
This new release requires configuration changes in config.xml. Additionally the upcoming beta versions are not compatible with the old database structure. So you have to delete your current DB if you want to test the new version (even without scraping). (There will be an automatic DB update with the next stable versions, but it would be too much work to implement this for every test release)
Ok, how it works:
To keep your old config working, replace all:
Code:
<descFilePath>PathToData\Amiga\synopsis\%GAME%\synopsis.txt</descFilePath>
<descriptionParserFile>PathToData\Amiga\parserConfig.xml</descriptionParserFile>
with this:
Code:
<scraper parseInstruction="PathToData\Amiga\parserConfig.xml" source="PathToData\Amiga\synopsis\%GAME%\synopsis.txt"/>
To use
http://thevideogamedb.com as online source, you should use this scraper configuration in config.xml (change the path to your parseInstruction):
Code:
<scraper parseInstruction="F:\Emulatoren\data\Testdata Current\Collection V9\VideoGamesDB - parserConfig.xml" source="http://thevideogamedb.com/API/GameDetail.aspx?apikey=%VGDBAPIKey%&crc=%CRC%"/>
There are two placeholders in the URL: %VGDBAPIKey% is an internal thing and will be replaced with RCBs API key that I had to request at vgdb.com. Just leave it like this. %CRC% will be replaced with the crc value of the current rom file. This should be the most robust way to match your rom files with the available game description. Just to be complete: There are two more placeholders available: %FILENAME% and %FOLDERNAME%. They may be of use in other scenarios.
The parserConfig.xml should look like this (this file is also shipped with RCB, you find it in resources/database):
Code:
<parserConfig>
<GameGrammar type="xml" root="Results/Game">
<Game>Name</Game>
<Platform>Platform/Name</Platform>
<Region>Region/Name</Region>
<Genre>Genre/Name</Genre>
<ReleaseYear>ReleaseDate</ReleaseYear>
<Developer>Developer/Name</Developer>
<Publisher>Publisher/Name</Publisher>
<Filetypecover>Boxart/FullURL</Filetypecover>
</GameGrammar>
</parserConfig>
This config tells RCB how to pick the data from the response message. There is only one "special" thing in it: "Filetypecover" grabs the url of an image. This image will be downloaded and used as filetype "cover". RCB will store this image at the path that you define as mediaPath type="cover" in config.xml. All other media types that are available can be handled the same way (Filetypetitle, Filetypeaction, ...).
(I am sorry to somehow reinvent the wheel with this scraping mechanism, but the XBMC scraper-xml seems not to be available from python and a complete implementation of this mechanism would have been a bit too much for this little project. And with the old flatfile parser I already went down the road half the way.)
Note that jas has announced to change the structure of thevideogamedb's xml response. So it may happen that the above parserConfig will not work anymore in some days. And please: if he has finished his API changes and built up the database, help him to add content to this site!
The complete RomCollection entry looks like this (ignore the emulatorCmd, it is just a placeholder):
Code:
<RomCollection>
<name>Collection V9</name>
<consoleName>Sega 32</consoleName>
<searchGameByCRC>True</searchGameByCRC>
<descFilePerGame>True</descFilePerGame>
<scraper parseInstruction="F:\Emulatoren\data\Testdata Current\Collection V9\VideoGamesDB - parserConfig.xml" source="http://thevideogamedb.com/API/GameDetail.aspx?apikey=%VGDBAPIKey%&crc=%CRC%"/>
<emulatorCmd>%ROM%</emulatorCmd>
<romPath>F:\Emulatoren\data\Testdata Current\Collection V9\Roms\*.zip</romPath>
<mediaPath type="cover">F:\Emulatoren\data\Testdata Current\Collection V9\Artwork\cover\%GAME%.jpg</mediaPath>
<fileTypeForGameList>cover</fileTypeForGameList>
<fileTypeForGameListSelected>cover</fileTypeForGameListSelected>
<fileTypeForMainViewGameInfoBig>cover</fileTypeForMainViewGameInfoBig>
<fileTypeForGameInfoViewBackground>cover</fileTypeForGameInfoViewBackground>
<fileTypeForGameInfoViewGamelist>cover</fileTypeForGameInfoViewGamelist>
</RomCollection>
Please let me know what you think about this solution. I know, it adds some more configuration mess but I hope that I am quite flexible with adding new scrapers or reflect changes of existing ones. If this all is a bit more settled, I will add more documentation about how to add new scrapers and maybe we get some more online sources available in the near future. If you know more sites that allow scraping their content or even better provide an API to do this, please tell me.
With the next version I will also try to add a Google or Bing Search to get better results when searching by game name. Otherwise you have to adjust your filenames for every new online source with different naming.
EDIT:
I forgot to mention: Atm thevideogamedb.com has only content for Sega 32 and Virtualboy. So please don't fire your complete SNES-collection against the server. Just have an eye on jas' progress to see if he finishes more systems in the future.
Regards,
malte