when will the games database will be integrated into kodi
#1
is there any eta on when the games database will be integrated into kodi , eg metadata artwork as scrapers ect, as it would be great to have a retro gaming setup based on kodi,having that lightweight flexible base for games, ive used all the addons over the years rcb (performace issues on large collections) ael (lack of flexibility on veiws, needs option of using tv shows, films ect like iagl does) and have tryed all the skins that support it, but all miss something.i find iagl comes the closest, but hard to build custom collections metadata ect.
Reply
#2
(2019-04-20, 02:07)kpop Wrote: is there any eta on when the games database will be integrated into kodi , eg metadata artwork as scrapers ect, as it would be great to have a retro gaming setup based on kodi,having that lightweight flexible base for games, ive used all the addons over the years rcb (performace issues on large collections) ael (lack of flexibility on veiws, needs option of using tv shows, films ect like iagl does) and have tryed all the skins that support it, but all miss something.i find iagl comes the closest, but hard to build custom collections metadata ect.

No idea about when the games DB will be coded, but it's going to take long at current development pace.

Regarding the views in AEL: Some time ago I tried to make AEL a video addon like IAGL, so it can use the video views. I desisted because the Kodi log got cluttered with errors and warnings complaining about AEL URLs not providing video files, which makes Kodi crazy.

Note that in most skins, enabling video or music views to make them available to program addons is a matter to change a few lines of code. Why program views are very poor in most skins? I don't know but it's kind of a tradition in Kodi. The only way to solve this is to request the skinners to give more love to the program section of the skin.
Reply
#3
Why do you say it will take so long, general pessimism or something more?

I was so hoping at this release of kodi would open it as a gaming front end, but without the library it's really no better than before when I had manual emulators set up. It would be great to have something that allowed art and nfo nfo files to be stored with the game files and supports smb so I can organise my library nicely!

Is AEL open to general pull requests if I had time to look at these features?
Reply
#4
You do realise we do this in our spare time for nothing other than the love of the project don't you?
Reply
#5
(2019-04-22, 09:02)Linden Ryuujin Wrote: Why do you say it will take so long, general pessimism or something more?

I am an addon developer, not a core developer. However, I regularly follow Kodi core development on Github to stay tuned of new features and generally because I'm interested.

(2019-04-22, 09:02)Linden Ryuujin Wrote: I was so hoping at this release of kodi would open it as a gaming front end, but without the library it's really no better than before when I had manual emulators set up. It would be great to have something that allowed art and nfo nfo files to be stored with the game files and supports smb so I can organise my library nicely!

Once the games DB is implemented the experience is not going to be much different to what you can get with AEL right now.

Storing artwork next to the ROMs is a very bad idea, as I have explained here and here.

The NFO files is kind of an exception and it is OK to be next to the ROMs. Note that the AEL approach, however, is to have a very good offline scraper (aka games database) so if your ROMs have the correct No-Intro or Redump name all the metadata will be already there, no need for NFO files. Yes... the offline scraper is still far from complete. It is difficult to make a DB of 200,000 games!

Supporting assets and ROMs in remotes (smb, nfs, sftp, etc.) is begin coded in next AEL release 0.10.0, but is it taking very long because it is not easy to do and many other reasons, one of them issues with Kodi VFS. Note that is not only a matter of supporting smb, if assets/ROMs in remotes are supported that must include all remotes and not the particular one you use at home.

(2019-04-22, 09:02)Linden Ryuujin Wrote: Is AEL open to general pull requests if I had time to look at these features?

Of course. However, if you plan to implement a big or disruptive feature it's better to check with me first, otherwise the PR may not be merged.
Reply
#6
Thanks for the quick reply. Could you maybe give a bit more detail on why you think keeping the art in the same folder as the ROM is a bad idea.
Quote:Talking about games (or ROMs), generally it is a bad idea to have the Icon, Poster and Fanart in the same folder as the ROMs. There are tons of reasons for this. In my opinion the most important one is that the artwork is named with the same filename as the ROMs. This is the non-written agreement in the emulation community and if Kodi doesn't follow it in the future games database it would be a tremendous mistake in my opinion. If artwork is named as the ROMs, it cannot coexist in the same folder as the ROMs because of filename collision.

If the art has a different name extension, or a -type post fix I don't see how there would be a collision. I've had art in the same folders as my roms for a long as I've been using them (admitidly I've had to copy and paste them else where for the front ends I've used to find them) but it has never caused an issue with the operation of the files. To me it just makes sense to keep everything togeather in a single folder, then it's easy to see what data I have.

As for SMB support, I can see there is clearly some depth I'm not getting. As the Kodi Game engine supports smb paths it feels like if I could just get them into the database it should work, but I guess that is not the case. I thought I might have a play around when I have a moment if it seems like the offical DB is going to be some time off as these are the features that are stopping me using kodi for games right now.
Reply
#7
(2019-04-22, 11:26)Linden Ryuujin Wrote: If the art has a different name extension, or a -type post fix I don't see how there would be a collision. I've had art in the same folders as my roms for a long as I've been using them (admitidly I've had to copy and paste them else where for the front ends I've used to find them) but it has never caused an issue with the operation of the files. To me it just makes sense to keep everything togeather in a single folder, then it's easy to see what data I have.

If you have a small collection (say, 100 ROMs or so) then your approach is fine. However, many people will like to have big No-Intro or MAME collections, with many thousands of ROMs. For example, my current collection in AEL/AML includes 40,000 MAME ROMs, plus about 100,000 MAME Software List items, plus about 40,000 No-Intro cartridge based ROMs. In such setups, your approach simply does not work.

AEL/AML already produce reports of the artwork you have and the artwork you don't have (and even when AEL/AML is using artwork from the parent into a clone). Is there any other feature you need?

Code:
Name                                               Tit Sna Fan Ban Clr Bxf Bxb Car Fly Map Man Tra
--------------------------------------------------------------------------------------------------
Sonic The Hedgehog (USA, Europe)                    Y   Y   Y   Y   Y   Y   Y   Y   -   Y   -   -
Sonic The Hedgehog 2 (World) (Rev A)                Y   Y   Y   Y   Y   Y   Y   Y   -   -   -   -
Sonic The Hedgehog 3 (Europe)                       Y   Y   Y   -   Y   Y   Y   Y   -   -   -   -

Code:
Name                                               Year Genre Developer Rating Plot Audit    PClone
---------------------------------------------------------------------------------------------------
Sonic The Hedgehog (USA, Europe)                   YES  YES   YES       ---    YES  None     None
Sonic The Hedgehog 2 (World) (Rev A)               YES  YES   YES       ---    YES  None     None
Sonic The Hedgehog 3 (Europe)                      YES  YES   YES       ---    YES  None     None

Note that this is equivalent to the "Missing Movies/TV Shows" addon integrated into AEL. In other words, in some ways AEL already provides more features than the games db will have once implemented into the Kodi core (assuming of course that the games db is similar to the movies db).

I'm considering opening a new thread about the design of the Games library, so I can write down my ideas instead of spreading them around the forum, and everyone can have a say. If the model of the DB is more or less fixed, then addons can implement it so the transition once Kodi core get the DB will be smooth.

(2019-04-22, 11:26)Linden Ryuujin Wrote: As for SMB support, I can see there is clearly some depth I'm not getting. As the Kodi Game engine supports smb paths it feels like if I could just get them into the database it should work, but I guess that is not the case. I thought I might have a play around when I have a moment if it seems like the offical DB is going to be some time off as these are the features that are stopping me using kodi for games right now.

First, not all Retroplayer cores support VFS, only some of them. For the ones which don't support Kodi VFS, ROMs must be in a local directory.

Second, it is not just putting filenames in the DB and you are done. You need to scan directories, read the ROMs, create NFO files, check wheter directories/files exist, check for errors, etc. AEL is a launcher and emulator front-end and now is about 15,000 lines of Python code!

What kind of front-end do you currently use? Launchbox, Hyperspin, etc.? It is true that Retroplayer is not ready for the general use. However, AML/AEL are feature on-par with the most advanced front-ends out there, and allows you to use Retroplayer or an external emulator when Retroplayer does not work (OpenGL cores for example). If there is a feature you love in your current front-end and want to see it in AEL/AML then propose it and I will try to implement it if it makes sense.
Reply
#8
I can see you're just in a different world of ROMs in terms of numbers. I like to be in full control and choose at the art work individually - part of adding a ROM for me is downloading or making the art and getting it all in place. I think there would be a place for both library types - I can understand though that you are going to prioritise the format the works for you, which is why I thought I might have a play around with something different. However my time is very limited at the moment so I doubt anything would surface any time soon anyway.
Reply
#9
(2019-04-22, 12:29)Linden Ryuujin Wrote: I can see you're just in a different world of ROMs in terms of numbers. I like to be in full control and choose at the art work individually - part of adding a ROM for me is downloading or making the art and getting it all in place. I think there would be a place for both library types - I can understand though that you are going to prioritise the format the works for you, which is why I thought I might have a play around with something different. However my time is very limited at the moment so I doubt anything would surface any time soon anyway.

If you have read carefully the links I put, you know I believe there are two main types of retrogamers: A) the ones that have small, curated collections of ROMs, B) the collector, who have complete No-Intro and MAME collections. When I design a new feature for the Advanced Launchers, I always try to accommodate both A and B types of retrogamers. Needles to say, this is impossible because it is impossible to keep everyone happy.

When you have a big collection it is impossible to have all the artwork. For example, most games do not have Fanart and making a Fanart for every game will take a lifetime. Also, games follow strictly the Sturgeon's Law and honestly most games do not deserve having a Fanart Smile To have curated collections and at the same time have a big collection, I created the AEL Favourites and the ROM Collections, where you can select and classify the ROMs you love most. Note that the artwork in a ROM Collection follows your naming scheme, ROM_name_[Boxfront|Fanart|etc]_ROMid.png but I have to include a unique identifier to avoid filename collisions when two ROMs have the same name on different platforms (and this case happens very frequently, specially for famous games like Mario and Sonic).

I guess the main reason to have ROMs and artwork together is that you can get the Windows or Linux shell and immediately see the artwork you have. In AEL, keeping the artwork in separated directories you can also do the same! For example, using a Matrix view like this, you can instantly review all the assets you have in your collection:

Image

Alternatively, AML can generate a composite image for each ROM with all the artwork for a game, and then you can view later the generated images with an external viewer. This feature is currently used in AML to generate Fanarts for the games missing it (in the case of MAME, 99.9% of games do not have Fanart) but since the technology to generate composite images is there we can use it for other things than generating Fanarts. Note that the generation of composite images is controlled with an XML file using a syntax very similar to what Kodi uses for the skins, so you can easily customize the composite images to your linking:

Image

So, if you elaborate about why do you think it's better to have all artwork together with different suffixes (ROMname_fanart.png, ROMname_boxfront.png, etc.) maybe we can come with a compromise solution that can make you and B users happy having the artwork separated in different directories.

Finally, if you have nice custom artwork, please share it! I can add it to the AEL Asset Library. Note that I am in the process of creating a proper ROM Collections subdirectory in the library, for the games people love most, but I am still thinking about what's the best method to store the assets of the ROM Collections. Free time is limited for everyone!
Reply
#10
(2019-04-22, 11:59)Wintermute0110 Wrote:
(2019-04-22, 11:26)Linden Ryuujin Wrote: If the art has a different name extension, or a -type post fix I don't see how there would be a collision. I've had art in the same folders as my roms for a long as I've been using them (admitidly I've had to copy and paste them else where for the front ends I've used to find them) but it has never caused an issue with the operation of the files. To me it just makes sense to keep everything togeather in a single folder, then it's easy to see what data I have.

If you have a small collection (say, 100 ROMs or so) then your approach is fine. However, many people will like to have big No-Intro or MAME collections, with many thousands of ROMs. For example, my current collection in AEL/AML includes 40,000 MAME ROMs, plus about 100,000 MAME Software List items, plus about 40,000 No-Intro cartridge based ROMs. In such setups, your approach simply does not work.

I hate to say this, but of course I do not really hate to say it, but if you have 40,000 MAME roms, then your collection is far too big and that is your own fault. Yes you can download every single rom file you want, but when you go through the files, you are left with about five thousand roms. I will break it down 

1) Remove all clones  e.g pacman puckman etc
2) Remove all majohng (excuse spelling) games
3) Remove all fruit machine games
4) Remove all games that are unplayable unless specific hardware is attached e.g some games that need a trackball or a light gun or athletic games are sometimes unplayable
5) Remove all games that are in Japanese that are not simple enough to click on play 

This will reduce those roms right down. Why bother saying well I have 100,000 roms when half of them are the same, or unplayable etc.

I am with the original poster, with saying that it is disappointing that game support has been added but in a half empty fashion. I would have loved to have seen a full roll out of a Kodi that has cover art and all the bells and whistles to go with it.

It is like buying a Ferrari but with no engine.

Kind regards
Reply
#11
(2019-04-22, 13:49)Video Titles Wrote: I hate to say this, but of course I do not really hate to say it, but if you have 40,000 MAME roms, then your collection is far too big and that is your own fault. Yes you can download every single rom file you want, but when you go through the files, you are left with about five thousand roms. I will break it down 

1) Remove all clones  e.g pacman puckman etc
2) Remove all majohng (excuse spelling) games
3) Remove all fruit machine games
4) Remove all games that are unplayable unless specific hardware is attached e.g some games that need a trackball or a light gun or athletic games are sometimes unplayable
5) Remove all games that are in Japanese that are not simple enough to click on play 

This will reduce those roms right down. Why bother saying well I have 100,000 roms when half of them are the same, or unplayable etc.

I am with the original poster, with saying that it is disappointing that game support has been added but in a half empty fashion. I would have loved to have seen a full roll out of a Kodi that has cover art and all the bells and whistles to go with it.

It is like buying a Ferrari but with no engine.

First, MAME is not about playing the games you want. Quoting from the MAME documentation:

Code:
MAME’s purpose is to preserve decades of software history. As electronic technology continues to rush forward, MAME prevents this important “vintage” software from being lost and forgotten.

However, according to people like you, clones, fruit machines, etc. must be removed as they are useless. Right?

Second, I live in Japan. I am not Japanese but speak some Japanese and I care very much about Japanese games and machines, like Mahjong, Hanafuda and Quiz machines. Presumably, I should also delete those from my collection because they are not interesting to western audiences, right?

Third, Kodi is a modular software. This means that a database can be in the C++ core or implemented in Python as an addon, and the result the user sees is exactly the same.

Fourth, one thing is the MAME machines and other different thing is the Software Lists. Do you understand the difference between both?

Fifth, AML already includes filters to not display the machines you mention, even to display only the machines you have the ROMs for. But, if you are curious or simply want to learn something about MAME or game/computer history, all the machines are there for you to enjoy them.

Finally, thanks to addons like AML/AEL, and others like IAGL and RCB, you can enjoy now games in Kodi with all bells and whistles. Have you seen the previous screenshots and many others in the forum? Is there any feature you miss and want to be implemented? Do you have any useful comments or suggestions, apart from curtailing my ROM collection?
Reply
#12
(2019-04-22, 14:44)Wintermute0110 Wrote: However, according to people like you, clones, fruit machines, etc. must be removed as they are useless. Right?

Second, I live in Japan. I am not Japanese but speak some Japanese and I care very much about Japanese games and machines, like Mahjong, Hanafuda and Quiz machines. Presumably, I should also delete those from my collection because they are not interesting to western audiences, right?
You keep the games that YOU want. However, it was an example of how I manage my rom files. It is like saying that I have 20 movies, but 10 are the same, I have the SD and HD version of the movie. I only have 10 movies.

You are obviously a very rude person and everyone else is wrong, because YOU are always right. However, saying that YOU have so many rom files is totally wrong as most are duplicates.

I have multiple set ups, so one genre does not necessarily suit each machine. However, if a person only has one set up for media and games, then they will be more selective of what they have ready to play. 

If you want to think that you have 50,000 games even though half are duplicates and clones, then so be it. However, get real.
Reply
#13
(2019-04-22, 15:13)Video Titles Wrote: You keep the games that YOU want. However, it was an example of how I manage my rom files. It is like saying that I have 20 movies, but 10 are the same, I have the SD and HD version of the movie. I only have 10 movies.

That comparison is not accurate. Clones are different from parents in many ways, although they may look similar. Sometimes is just the translation, sometimes they have important differences like censorship.

(2019-04-22, 15:13)Video Titles Wrote: You are obviously a very rude person and everyone else is wrong, because YOU are always right. However, saying that YOU have so many rom files is totally wrong as most are duplicates.
...

So, you tell me that I'm rude because I think everyone else is wrong. Then, in the next sentence you tell me I'm wrong Huh Confused

By the way, how do you select the ROMs you want? Do you do it by hand or you use some tool?
Reply
#14
I am not on my arcade PC so I cannot tell you 100% what tools I use. I use Don's hypertools and also Hypertools I believe it is https://hyperspin-fe.com/files/category/...dit-tools/

Yes we could go on all day about clones, however I did not want bloated systems so I went through and removed any that said clone of which did not take very long.

I have an arcade cabinet which I am still building once I buy the artwork then I can put it together. I have all the buttons, wheels, and only have the artwork, audio, t moulding left to buy. 

I also have a 5:4 which I use as a quiz machine, but I brought the arcade controls for that system also, but that is a project that I may add, or just use as a quiz machine.

I also have a 32 inch touch screen which I use as a jukebox, I did try out some vertical games on it, but I don't fancy sitting there trying to play a game.

I have my HTPC which is attached to a 60 inch screen, but playing games such as fruit machines etc are a waste of time, hence going through the rom sets.

I also have a x-arcade tankstick and the plan is either strap on a rasperberry pi or put it into the unit itself.

I still want to buy a 42 inch sit down cabinet, but that is further down the line.

However, for each system some games are better than others, but I do not play Majohng or games such as hyper olympics I possibly would not play on an arcade cabinet, but I would play on my tankstick. Certain games, I would not want for my 60 inch screen and indeed I would not want them for the 42 inch sit down cabinet. However, I would want them for my 5:4 screen or I have a 15 inch 4:3 display that I was considering using.

Oh and I would love a pinball cabinet as well, I have all the roms for hyperspin, virtual pin etc. 

Kind regards
Reply
#15
@Video Titles Thank you for sharing details about your setup. Besides, do you have any contribution or suggestion to do about the games database in Kodi?
Reply

Logout Mark Read Team Forum Stats Members Help
when will the games database will be integrated into kodi0