Kodi Community Forum

Full Version: Planning for Game Scraping
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Just wanted to start this thread and propose what game infolabels could / should be supported in Kodi?

I know that Retroplayer and Kodi currently doesn't have the ability to scrape info / metadata / artwork for games yet, but I'd like to get the ball rolling by attempting to write a scraper/database for this info for Kodi (semi-based on all the metadata that already exists in the addon I wrote, but written to provide a database for quick and efficient retro game scraping). To do that, we'd need a list of infolabels that games support.

Here's a list of suggested infolabels that I can think of.
  • title - self explanatory
  • alt title(s) - alternate titles for the same game
  • platform - platform(s) the game file is compatible with. Platforms themselves would have to be standardized
  • plot / synopsis / overview - self explanatory
  • date - release date of game
  • genre - self explanatory. Would be beneficial if the genre types was standardized somehow
  • developer / studio - self explanatory
  • publisher - self explanatory
  • region / country code - Lots of game files are tagged currently with region code (US, De, Br, etc). There is sort of a 'standard' for this already, see here.
  • rom code - Lots of game files are tagged currently with rom codes (h = hacked rom, t = trained rom, ! = perfect dump, etc). There is sort of a 'standard' for this already, see here.
  • region format - NTSC, PAL, others?
  • media format - CD, cartridge, floppy, multi-disc floppy, etc
  • discnumber - self explanatory
  • players - could be simple number for max number of players, of could be defined further for vs. games, co-op games, etc
  • poster(s) - could be defined further to front box, back box, flyer, cartridge / disc image, etc
  • banner(s) - self explanatory
  • fanart(s) - self explanatory
  • clearart - self explanatory
  • clearlogo - self explanatory
  • thumb - self explanatory
  • snapshot(s) - could be defined further to title snapshot, in-game snapshot, game-over screen snapshot, etc (MAME image sets go to this detailed level currently)
  • trailer - self explanatory
  • manual - scan of game manual
  • map(s) - maps for in game
  • perspective - top-down, side scrolling, first person, etc. A standardized list would be beneficial
  • rating - similar to IMDB / user rating ESRB - E, M, etc
  • size - file size of game file(s)
  • set - Shows the name of the set the game is part of (ex: all mario games)
  • tagline - self explanatory
  • cast - cast of characters in game (ex: mario, luigi, etc)
  • playcount - self explanatory
  • lastplayed - self explanatory
  • percentcomplete - not sure if this is possible? Probably not.

Seems like a lot of infotags, but I'm sure others can think of some I haven't. What do you all think?
my .02c;

learn from prior mistakes. use a key-value based solution.
Hey mate, as you know, I'm on-board for some IARL and Retroplayer love with what I'm cooking up. But I gotta say, that list of tags is exceedingly overkill. Ambitious, certainly, but I'd think it'd be largely impractical. Better to zero in on more of the universally applicable attributes for games.

Boxes, Fanarts, Logos, Plot, Rating, Genre, Studio, Year, Platform, Trailer, Banner, Snaps / Screenshots - anything else beyond this and you'd stray into areas where consistency of information would likely fragment (ie: some obscure Famicom game, some obscure Japanese GBA game, etc...). Just with the stuff I am doing, that handful of tags / assets is enough to juggle with when you take into account the varying box shapes for games / systems and screenshots and their multitude of aspect ratios.

Plus just from a skinning standpoint, while I'm ever grateful for the Kodi devs for continually pushing Kodi further and further, it's always a PITA transitioning from version to version because of all the changes. Tongue This is only going to grow with Retro now being brought into the main line.

Makes me yearn for the days of skinning for Winamp 2.x. Could knock that crap out in an afternoon. Wink
TheGamesDB might be useful to look at database fields held for each game

http://wiki.thegamesdb.net/index.php/GetGame


Basically the XML returned when doing a game search is this:
Code:
<?xml version="1.0" encoding="UTF-8" ?>

<Data>
<baseImgUrl>http://thegamesdb.net/banners/</baseImgUrl>
<Game>
<id>2</id>
<GameTitle>Crysis</GameTitle>
<Platform>PC</Platform>
<ReleaseDate>11/13/2007</ReleaseDate>
<Overview>From the makers of Far Cry, Crysis offers FPS fans the best-looking, most highly-evolving gameplay, requiring the player to use adaptive tactics and total customization of weapons and armor to survive in dynamic, hostile environments including Zero-G.

Earth, 2019. A team of US scientists makes a frightening discovery on an island in the South China Sea. All contact with the team is lost when the North Korean Government quickly seals off the area. The United States responds by dispatching an elite team of Delta Force Operators to recon the situation. As tension rises between the two nations, a massive alien ship reveals itself in the middle of the island. The ship generates an immense force sphere that freezes a vast portion of the island and drastically alters the global weather system. Now the US and North Korea must join forces to battle the alien menace. With hope rapidly fading, you must fight epic battles through tropical jungle, frozen landscapes, and finally into the heart of the alien ship itself for the ultimate Zero G showdown.</Overview>
<ESRB>M - Mature</ESRB>
<Genres><genre>Shooter</genre></Genres><Players>4+</Players><Co-op>No</Co-op>
<Youtube>http://www.youtube.com/watch?v=i3vO01xQ-DM</Youtube>
<Publisher>Electronic Arts</Publisher>
<Developer>Crytek</Developer>
<Rating>8.75</Rating><Images>
<Similar>
<SimilarCount>2</SimilarCount>
<Game>
<id>15246</id>
<PlatformId>15</PlatformId>
</Game>
<Game>
<id>15225</id>
<PlatformId>12</PlatformId>
</Game>
</Similar>
<fanart>
<original width="1920" height="1080">fanart/original/2-1.jpg</original>
<thumb>fanart/thumb/2-1.jpg</thumb>
</fanart>
<fanart>
<original width="1920" height="1080">fanart/original/2-2.jpg</original>
<thumb>fanart/thumb/2-2.jpg</thumb>
</fanart>
<fanart>
<original width="1920" height="1080">fanart/original/2-3.jpg</original>
<thumb>fanart/thumb/2-3.jpg</thumb>
</fanart>
<fanart>
<original width="1920" height="1080">fanart/original/2-4.jpg</original>
<thumb>fanart/thumb/2-4.jpg</thumb>
</fanart>
<fanart>
<original width="1920" height="1080">fanart/original/2-5.jpg</original>
<thumb>fanart/thumb/2-5.jpg</thumb>
</fanart>
<fanart>
<original width="1920" height="1080">fanart/original/2-6.jpg</original>
<thumb>fanart/thumb/2-6.jpg</thumb>
</fanart>
<boxart side="back" width="1525" height="2162" thumb="boxart/thumb/original/back/2-1.jpg">boxart/original/back/2-1.jpg</boxart>
<boxart side="front" width="1525" height="2160" thumb="boxart/thumb/original/front/2-1.jpg">boxart/original/front/2-1.jpg</boxart>
<banner width="760" height="140">graphical/2-g2.jpg</banner><banner width="760" height="140">graphical/2-g3.jpg</banner><screenshot>
<original width="1920" height="1080">screenshots/2-1.jpg</original>
<thumb>screenshots/thumb/2-1.jpg</thumb>
</screenshot>
<clearlogo width="400" height="95">clearlogo/2.png</clearlogo></Images>
</Game>
</Data>
Hi everyone. First of all, kudos to everyone involved in Retroplayer. Finally it got merged into master!

I would like to contribute to this thread with the list of tags and assets currently in use by Advanced Emulator Launcher, that can be found here. Advanced MAME Launcher (currently in development and with no released version yet) maybe will add a few more metadata fields. Of course this list is open to discussion and improvements and I think it would be nice if the new Kodi games database use the same tags currently being use by the game addons. This will simplify a lot the live of skinners!

I agree with Rufoo that the amount of tags/assets proposed by Zach is maybe too much. I think is it going to be difficult to code the scrapers for so many metadata/assets, many games will just have N/A for many tags (for example, for most games developer and publisher is just the same company). In addition, having a lot of database fields means bigger databases and slower execution for big collections. Keep in mind that current MAME plus Software Lists we are talking about 100.000 ROMs in total. Finally, good frontends like HyperSpin have much simpler databases than the proposed one and it's already a PITA to get all the artwork required to properly configure it!