[Video Titles]

@Video Titles Thank you for sharing details about your setup. Besides, do you have any contribution or suggestion to do about the games database in Kodi?

I am not a programmer. You have xml files already created by Hyperspin, so possibly some way of using those, linking to graphics on the system or network. It would be simple enough to take the XML and make some compatible file that Kodi can use. The games are not changing anytime.

You would have the graphics already if you are using a front end. I for myself would cherry pick the games that I wanted I would not expect a full load. 

Some collaboration with the games db, emu movies, hyperspin etc. 

The internet archive games launcher has created something, so asking them to borrow their code and importing into Kodi 

A few ideas, but I shall leave the coding to the brains instead of the monkey.

[steve1977]

Not sure why the conversation derailed to duplicates and clones. We all use Kodi differently and get excited about different things. Some of us are more "collectors" and even like to include Japanese games in their database even if they cannot be played without Japanese language capabilities. Similar for movies, I am sure many of us will have movies in their library, which they have never watched and never plan to watch (or cannot even watch as the movies are in languages that they don't even understand).

While I personally don't have and do not plan to collect 100,000 roms, I do sympathize and think it will look truly impressive in Kodi to have so many roms and see the different variants. I'd love to see the artwork of some of the Japanese language roms that never made it to translation. I am sure they are quite cool!

I don't think anyone will argue that Kodi needs a games library. Actually, the core of Kodi is its database and I agree with the earlier analogy that the current state of retroplayer is a "Ferrari without an engine" (or an engine without the chassis?). I find Retroplayer a huge accomplishment and am truly excited by it. It's amazing how well it works now. If you'd asked me whether gaming should be part of Kodi a few years ago, I would have said "no". By now, it may well be the part that gets me most excited about Kodi.

The question is developer interest and capacity to work on a game library. It seems that the developers have either no interest or no capacity. It seems @Wintermute0110  has the passion for retro games. Maybe we could all convince him to shift focus from add-on development to joining the Kodi team developing a proper games librart? ;-)

[Wintermute0110]

I am not a programmer. You have xml files already created by Hyperspin, so possibly some way of using those, linking to graphics on the system or network. It would be simple enough to take the XML and make some compatible file that Kodi can use. The games are not changing anytime.

You would have the graphics already if you are using a front end. I for myself would cherry pick the games that I wanted I would not expect a full load. 

Some collaboration with the games db, emu movies, hyperspin etc. 

The internet archive games launcher has created something, so asking them to borrow their code and importing into Kodi 

A few ideas, but I shall leave the coding to the brains instead of the monkey.

If you want to use your Hyperspin XMLs and assets then user HyperLauncher.

Hyperspin XMLs cannot be used in Kodi because it's a subset of No-Intro with USA region games only. Kodi is used worldwide, and Japanese users want to use Japanese ROMs, and European users want to use Europe ROMs in their language when available.

[Wintermute0110]

Not sure why the conversation derailed to duplicates and clones. We all use Kodi differently and get excited about different things. Some of us are more "collectors" and even like to include Japanese games in their database even if they cannot be played without Japanese language capabilities. Similar for movies, I am sure many of us will have movies in their library, which they have never watched and never plan to watch (or cannot even watch as the movies are in languages that they don't even understand).

While I personally don't have and do not plan to collect 100,000 roms, I do sympathize and think it will look truly impressive in Kodi to have so many roms and see the different variants. I'd love to see the artwork of some of the Japanese language roms that never made it to translation. I am sure they are quite cool!

I don't think anyone will argue that Kodi needs a games library. Actually, the core of Kodi is its database and I agree with the earlier analogy that the current state of retroplayer is a "Ferrari without an engine" (or an engine without the chassis?). I find Retroplayer a huge accomplishment and am truly excited by it. It's amazing how well it works now. If you'd asked me whether gaming should be part of Kodi a few years ago, I would have said "no". By now, it may well be the part that gets me most excited about Kodi.

Yes, the discussion got a little bit off topic. Let's go back on track. The main question here is if the database should keep the artwork next to ROMs like this (compact model):

Code:

/home/kodi/ROMs/sega-genesis/Sonic The Hedgehog (USA, Europe).zip
/home/kodi/ROMs/sega-genesis/Sonic The Hedgehog (USA, Europe)_boxfront.png
/home/kodi/ROMs/sega-genesis/Sonic The Hedgehog (USA, Europe)_fanart.png
...
/home/kodi/ROMs/sega-genesis/Sonic The Hedgehog 3 (Europe).zip
/home/kodi/ROMs/sega-genesis/Sonic The Hedgehog 3 (Europe)_boxfront.png
/home/kodi/ROMs/sega-genesis/Sonic The Hedgehog 3 (Europe)_fanart.png
...

OR we should keep them separated like this (separated model):

Code:

/home/kodi/ROMs/sega-genesis/Sonic The Hedgehog (USA, Europe).zip
/home/kodi/ROMs/sega-genesis/Sonic The Hedgehog 3 (Europe).zip
...
/home/kodi/Artwork/sega-genesis/boxfronts/Sonic The Hedgehog (USA, Europe).png
/home/kodi/Artwork/sega-genesis/boxfronts/Sonic The Hedgehog 3 (Europe).png
/home/kodi/Artwork/sega-genesis/fanarts/Sonic The Hedgehog (USA, Europe).png
/home/kodi/Artwork/sega-genesis/fanarts/Sonic The Hedgehog 3 (Europe).png
...

Note that the main difference is that the separated model does not require suffixes for the artwork filenames, where the compact model requires them.

I am an advocate of the separated model and I have argued many reasons in favor of it. However, I know many people like you prefer the compact model and I want to understand the reasons and motivations. For example, before I assumed you like to have the artwork together so you can easily see it with the Windows explorer. If that's the reason, that can be solved with the two approaches I proposed before. If there is another reasons I would like to know them so we can put the pros and cons of both approaches.

The question is developer interest and capacity to work on a game library. It seems that the developers have either no interest or no capacity. It seems @Wintermute0110  has the passion for retro games. Maybe we could all convince him to shift focus from add-on development to joining the Kodi team developing a proper games librart? ;-)

I'm very happy as a developer of the Advanced Launcher addons. Kodi core developer is too much responsibility for me

[sagrath]

I know many people like you prefer the compact model and I want to understand the reasons and motivations.

Well, since I do not have an immense and complete romset, only some roms and some retro-games, I prefer the compact model because it is easier to work with the arts. Well, I guess it would be just like the music arts. But if someone has a full romset, then it really gets complicated to use the compact model. So I ask: Why not both?

[Wintermute0110]

I know many people like you prefer the compact model and I want to understand the reasons and motivations.

Well, since I do not have an immense and complete romset, only some roms and some retro-games, I prefer the compact model because it is easier to work with the arts. Well, I guess it would be just like the music arts. But if someone has a full romset, then it really gets complicated to use the compact model. So I ask: Why not both?

1) To reduce the complexity of the code and documentation.

2) To keep the standards of the emulation community, like ROM scanning with DAT files. Having extra files next to the ROMs will make the ROM scanners crazy.

3) To make Kodi compatible with existing emulation artwork collections (Progretto SNAPS, Retroarch assets).

4) To make Kodi compatible with existing emulation solutions like Retropie, Lakka, etc.

If tools are needed to help users they can be developed. For example, a non-collector user with a small collection using the separated model:

1) AEL (or the Kodi Games DB) can create composite "Fanarts" with all the artwork and place them in a folder. Then, you can get the Windows image viewer and quickly check the artwork you have.

2) Image placeholders can be created. For example, AEL can create a placeholder Boxfront, Fanart, etc. with a message "ROM bla bla bla missing", and you can use you favourite image viewer to identify the work you miss. When downloading the image with your web browser you overwrite the placeholder and you don't have to type the filename,

3) Artwork reports: AEL already produces produces TXT reports with the missing artwork you can check. You can open the report with a TXT editor, search for the artwork only, and copy/paste the filename so you don't have to type it.

4) etc.

[zachmorris]

The main question here is if the database should keep the artwork next to ROMs like this (compact model):

OR we should keep them separated like this (separated model):

This topic is probably better in the development part of the forum, but I'm not sure Kodi would / should use either. Kodi's library database, scraping functionality, artwork database, etc should just be extended from audio/video files to other now-playable files.
- The artwork is stored in the Kodi database just like it is now (although maybe additional art types might be added specific to games in the future)
- Scrapers are included just as they are now, users define 'sources' and denote the folder contains games, and then even further maybe denote what system the folder contains, and what scraper to use (so instead of choosing metadata.tvdb.com, you would chose metadata.thegamesdb.com or whatever)
- Users can choose to use supplementary tools for game management just as they can for media management, and the supplementary media follows the same exact structure as it does for video or audio (again maybe adding additional games specific artwork types in the future)

[Wintermute0110]

This topic is probably better in the development part of the forum, but I'm not sure Kodi would / should use either.

Agree. But maybe here will have more visibility than in the development subforum.

Kodi's library database, scraping functionality, artwork database, etc should just be extended from audio/video files to other now-playable files.

Agree.

- The artwork is stored in the Kodi database just like it is now (although maybe additional art types might be added specific to games in the future)

Yes, artwork is stored in the Kodi cache. However, the important point here is about choosing a directory/filename layout for the ROMs, the ROMs artwork and the Platform artwork and NFO files. For example, Kodi has strict rules about how the Movies artwork should be organised, how the Music artwork should be organised, etc. And the point here is designing a scheme that works for everyone, users with small curated collections (A users) and collector, completionist users (users B).

- Scrapers are included just as they are now, users define 'sources' and denote the folder contains games, and then even further maybe denote what system the folder contains, and what scraper to use (so instead of choosing metadata.tvdb.com, you would chose metadata.thegamesdb.com or whatever)

I agree about associating the platform to each ROM source folder. This will facilitate to choose the cores to run the ROMs, the scrapers operation, etc. This is particularly important for arcade/MAME ROMs, because it is almost impossible to automatically know or even guess the platform. Note that for console/No-Intro systems it is very easy (different file extensions) and CD-based platforms with intermediate difficulty (CD images must be opened and inspected, and there are many formats, ISO, CHD, BIN/CUE, etc.).

However, main problem here is the artwork. For example, this is currently the size of the artwork/assets I have for AML:

Code:

18.2 GiB [##########] /fanarts_SL
   15.6 GiB [########  ] /fanarts
   14.8 GiB [########  ] /3dboxes
   13.8 GiB [#######   ] /videosnaps
   12.9 GiB [#######   ] /3dboxes_SL
   11.4 GiB [######    ] /manuals
    9.3 GiB [#####     ] /manuals_SL
    7.0 GiB [###       ] /flyers
    4.7 GiB [##        ] /videosnaps_SL
    4.1 GiB [##        ] /covers_SL
    3.2 GiB [#         ] /artwork
    3.0 GiB [#         ] /PCBs
    2.8 GiB [#         ] /cabinets
    1.7 GiB [          ] /marquees
    1.6 GiB [          ] /snaps
    1.6 GiB [          ] /titles
    1.1 GiB [          ] /cpanels
  738.6 MiB [          ] /titles_SL
  533.2 MiB [          ] /snaps_SL
  506.1 MiB [          ] /artpreviews
  380.1 MiB [          ] /clearlogos

Total disk usage: 128.9 GiB

Just the arcade assets are 130 GB! Obviously, rather than scraping that I think is better to directly download the artwork from many collections available, place it in the correct directory, and Kodi scanner can readily pick it up.

Of course this is not a problem for A users.

- Users can choose to use supplementary tools for game management just as they can for media management, and the supplementary media follows the same exact structure as it does for video or audio (again maybe adding additional games specific artwork types in the future)

Supplementary tools are already there (CLRMamePro, Romcenter, etc.) and have been there for around 20 years. Supplementary media already exists (ProgrettoSNAPS, Emumovies) so the thing here is not reinventing the wheel again and again, instead to adapt Kodi to what already exists. When we talk about games these tools assume that the artwork is named like the ROMs and placed in different directories. Even MAME itself requires the artwork to be named like the ROMs and in different directories.

Lastly, what I am trying to avoid will all this is something similar to what happened with the Artist information folder. In the case of the games, there could be a Platform information folder where you can place NFO files and artwork for the platforms (using a special naming scheme) and the artwork for ROMS belonging to each platform can be in a subdirectory. For example:

Code:

/home/kodi/Artwork/Sega Genesis_icon.png
/home/kodi/Artwork/Sega Genesis_fanart.png
/home/kodi/Artwork/Sega Genesis.NFO
/home/kodi/Artwork/Sega Genesis/boxfronts/Sonic The Hedgehog (Europe).png
/home/kodi/Artwork/Sega Genesis/fanarts/Sonic The Hedgehog (Europe).png
...

If the Kodi database designers make a poor decision it means B users will need to rename thousands of files or scrape gigabytes of artwork. Or, like in the Artist information folder, have to modify the filename scheme later to fix mistakes in the designing stage.

[Linden Ryuujin]

I like having the art in the folder with the ROM for two reasons:
1. It's the standard everything else in kodi uses
2. I can move a single folder (to another device etc) and I get everything associated with the ROM.

I think it's clear from this thread there's at least some people who would like to keep the art with the files. Personally my feeling from using other ROM libraries is that they all are pretty rough, that's why I've been waiting for kodi to bring in game support!

I don't feel I have the knowedge to try and add a whole game library to Kodi, but I might be able to scrape togeather a pull request for AEL which would support art in the same folder as the rom.

[Wintermute0110]

I like having the art in the folder with the ROM for two reasons:
1. It's the standard everything else in kodi uses

That can be refuted as; Kodi has never used games before, so what has worked for Movies, Music, etc. does not necessarily mean it will work for games.

2. I can move a single folder (to another device etc) and I get everything associated with the ROM.

True, but if you organise your setup like this

Code:

F:/Games/ROMs/Sega Genesis/...
F:/Games/Assets/Sega Genesis/...

You only need to move the Games directory and you are done. Also, even better if you want to share your ROMs and artwork in several devices, you set a network share in one of the Kodi boxes or a NAS, mount the remote directories, and you are done. You don't have to copy/move any file at all.

I think it's clear from this thread there's at least some people who would like to keep the art with the files. Personally my feeling from using other ROM libraries is that they all are pretty rough, that's why I've been waiting for kodi to bring in game support!

I don't feel I have the knowedge to try and add a whole game library to Kodi, but I might be able to scrape togeather a pull request for AEL which would support art in the same folder as the rom.

I totally understand that you and some people like to have the artwork next to the ROMs. It could be a personal preference, or simply you like it, or you think it's more elegant, or whatever. I respect that because I respect other people's opinions. But liking something is not a good reason we can use when designing a piece of software. When designing a database that will be used by many people we should take into account technical reasons, for example the nature of the objects (games in this case), the previous work already done by others, what similar software does, how can improve that, etc., and not opinions or preferences. To be more exact: technical reasons should be the leading points, and then when a choice arises that cannot be backed up by technical reasons, only then, we can take opinions into account.

I have the feeling I have written many many reasons for the design I propose, and all the arguments in return are like: "It makes sense to have ROMs and artwork together" (this is not a reason, it's an opinion), "I like to be in full control of my artwork" (you will be in full control of your artwork whether is together or separated), "Your ROM collection is too big and that's your fault" (who are you to tell me how many ROMs I have?), "We should keep ROMs and artwork together because it's what Kodi does for Movies and Music" (past returns do not guarantee future returns)

If you have not done so, please read the story behind the Artist information folder I linked before. A poor design caused trouble, which was solved by modifying the database/storage layout design. And such modifications are painful for everyone, because it could mean a lot of work in people's setups.

Can you please elaborate what you mean by "my feeling from using other ROM libraries is that they all are pretty rough, that's why I've been waiting for kodi to bring in game support"? If you have any feature request for AEL please share it and I will assess the benefits and implement it if it's a good idea. Note that the lack of views is not AEL fault, it is the skinners job to do views.

[steve1977]

I can understand the pros and cons for both sorting logics. I am personally indifferent.

The messages imply that there actually is development activity on a native library? Is this the case and any hope for Kodi 19?

[HeresJohnny]

May I just point out that the treatment of artwork is somewhat similar to what happens in the music section. Most of us music lovers have their artwork and music tags in place and don't want Kodi to tamper with that, especially not fill up the folders with more files. The solution there was a dedicated artist information folder where all info and artwork downloaded by Kodi will be sorted in subfolders for the respective artists and albums.

The situation here sounds somewhat similar so a game_information folder could be a possible solution.

[Wintermute0110]

I can understand the pros and cons for both sorting logics. I am personally indifferent.

The messages imply that there actually is development activity on a native library? Is this the case and any hope for Kodi 19?

To the best of my knowledge work has not started at all. Currently, only addons provide database-like services for games:

1) Hyperlauncher uses the XML databases and assets (artwork) from Hyperspin.

2) IAGL uses a custom pre-compiled database that uses remote assets and ROMs.

3) RCB and AEL have custom databases which are pretty similar actually. In other words, the metadata (title, plot, release year, etc.) and the assets (title, snap, boxfront, ...) supported by both addons is almost the same with minimal differences. Perhaps RCB database is more complete than AEL, because AEL design is minimalistic; I prefer to have less metadata fields and complete that having many metadata fields with a lot of them incomplete. For example, RCB supports Developer and Publisher, whereas AEL only supports Developer. The reason is that in most games (80%/90%) Developer and Publisher is the same company so having both fields is kind of redundant. AEL is minimalistic in design because scalability and performance is a must; less metadata fields means smaller databases in MB and smaller DBs means faster loading times.

If we design some database model for the future Kodi games database I can start the implementation in AEL right now, and users can start modifying their setups with knowledge that all the time they spend (organising ROMs and assets, creating NFO files, etc.) will not be wasted and skinners can start coding the game views assured that once the database moves from addons into the Kodi core the infolabels exposed to the skin are the same.

EDIT: regarding if the database will be ready for Kodi Matrix... TTBOMK probably not. Development pace is slow because the main developer (Garbear) is now busy. Also, there are many outstanding issues which I think have more priority than the games DB (for example, the Player Manager (AKA multijoystick support), the save state manager, OpenGL support, etc.).

[MacGyver]

I can't skin for anything until I can get the filename minus the extension out of Kodi.

Right now the only way to display an image with a rom is to name the image (in whatever scheme you decide on) as "Super Mario.nes.jpg".

I have access to a variable of just the extension and one with the filename and extension but none without and no way to remove one from the other. I could skin whatever scheme anyone decides on right now without any database support if I could just access a ListItem with just the filename and no extension.

This is what Metropolis looks like right now, but all the images have to be named in the above wonky way.





Is there a plan to address this shortcoming in the interim? So we can display artwork without waiting for database support.

[Wintermute0110]

I can't skin for anything until I can get the filename minus the extension out of Kodi.

Right now the only way to display an image with a rom is to name the image (in whatever scheme you decide on) as "Super Mario.nes.jpg".

I have access to a variable of just the extension and one with the filename and extension but none without and no way to remove one from the other. I could skin whatever scheme anyone decides on right now without any database support if I could just access a ListItem with just the filename and no extension.

This is what Metropolis looks like right now, but all the images have to be named in the above wonky way.

...

Is there a plan to address this shortcoming in the interim? So we can display artwork without waiting for database support.

You can base your views for an addon like RCB or AEL, which provides the infolabels you need in the skin. For example, in the case of AEL these is a (non comprehensive) list of infolabels:

Code:

# Game specific artwork
$INFO[ListItem.Art(title)]
$INFO[ListItem.Art(snap)]
$INFO[ListItem.Art(3dbox)]
$INFO[ListItem.Art(boxfront)]
$INFO[ListItem.Art(boxback)]
$INFO[ListItem.Art(cartridge)]
$INFO[ListItem.Art(flyer)]
$INFO[ListItem.Art(map)]
$INFO[ListItem.Art(controller)]

# Kodi standard artwork. Some are shared with games (Fanart and Clearlogo) and others are mappable to
# Game artwork. For example, by default AEL maps $INFO[ListItem.Icon] to $INFO[ListItem.Art(boxfront)].
$INFO[ListItem.Icon]
$INFO[ListItem.Art(fanart)]
$INFO[ListItem.Art(banner)]
$INFO[ListItem.Art(poster)]
$INFO[ListItem.Art(clearlogo)]

# Games metadata
$INFO[ListItem.Year]
$INFO[ListItem.Genre]
$INFO[ListItem.Studio]
$INFO[ListItem.Property(platform)]
$INFO[ListItem.Property(esrb)]
$INFO[ListItem.Rating]
$INFO[ListItem.Property(nplayers)]

I think the skins should not try to guess the name of the artwork based on the filename of the ROM, that's the database job (either implemented in the core or as a Python addon). For example, in the skin you use $INFO[ListItem.Art(poster)] and don't care where the PNG/JPG file is located. The Poster file may be a local file, a remote share or even a URL (online asset).

The discussion we have here is mainly about the filesystem layout (how to place the ROMs, the artwork, the platform artwork, NFO files with metadata, etc.) but whatever scheme is finally chosen the infolabels are going to be pretty similar to what AEL provides right now. The views you make for AEL or RCB will be ready for the Kodi games DB once implemented with (hopefully) minimal changes.

Also, note that most skins disable most of the Movies and Music views in MyPrograms.xml and MyGames.xml. Just enabling the views in those sections, AEL/RCB/IAGL can readily use them because we try to use Kodi standard infolabels as much as possible.