Kodi Community Forum
[RELEASE] Rom Collection Browser - Browse and launch emulator game ROMs - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Game Add-ons (https://forum.kodi.tv/forumdisplay.php?fid=291)
+---- Thread: [RELEASE] Rom Collection Browser - Browse and launch emulator game ROMs (/showthread.php?tid=70115)



- Paybac - 2010-06-15

Quote:XBMC did not create a log file or RCB did not write to it? I think it is ok that RCB did not write anything because you set it to log level WARNING (whats absolutely good in your case!).
The Log file only had 6-10 lines in it. This was the xbmc.Old.log which i grabed after reboot. the xbmc.log, just had the usual startup stuff. So i duno what would happed to the log, it woulda been pretty massive after 6hours i imagine.

Quote:Did you stop it because i hangs somewhere or did you just want to check the result? I know, this is painfully slow but it is a lot of work to do for a small machine like xbox. So I think this time could be normal.

If you think it hangs: What was the last reported step in the progress dialog?
I just stoped it to check the result(plus watch a movie Smile) The xbox games scrape quick(like 15seconds for all 30ish games added), then the Mega drive scraped really quick like maybe 1min for 1191 games, then the collections after that just got slower and slower like minutes per game i also noticed that not every game name was poping up. Like on the mega drive i only sore maybe 5 game names pop up before it was onto the next collection

Quote:There are only some WARNINGs in your update.log. Nearly all of your games should have been imported fine. I think you already noticed the char filter on the right side? Try to play around with the filter settings and see if there are more results somewhere else.
hmm yea i forgot to check that, i'll do that once i can get back on the xbox (once the Mrs lets go of the remoteRolleyes)

Quote:How large is your MyGames.db? Do you have a chance to upload it somewhere? I could have a look inside if everything is imported. And it would be nice to have such a large db for testing purposes;-)

132kb, I uped it to Megaupload, Pmed you the link.

Also check out your thread at xbmc4xbox.org somones got some info regarding windowed playback, doesnt look like its a fault at your end but a fault with xbmc4xbox.
http://www.xbmc4xbox.org/forum/viewtopic.php?f=8&t=184&start=10


- malte - 2010-06-15

Ok, there are not more than 27 games inside your db. I have absolutely no idea whats happening here.

1 min for 1191 games is much too fast. But I am wondering why there are no more errors or warnings inside the log. Are you still using your old config and synopsis that you uploaded in your earlier post? Should be ok but I will try to reproduce it with your config here.

About using .cut files: Yes, this could be a problem. But it looks like there is another error in the script or in your configuration.

Could you try to import just 1 or 2 collections and tell RCB to import the .zip files instead of the .cut files? Maybe it is really just a problem with wrong crc values.

Off to work now. Will have a look on this later again.

Regards,
malte


- Paybac - 2010-06-15

ok just a quick headsup, i did one collection pointing at the zips rather then the .cuts scraped everthing mint, all games added descriptions everything nice. Will have a go at multiple collections in the morning.

Heres something i noticed a while back but had forgot to mention. When the XBMC screensaver comes on while you are in RCB, when you press a button to get rid of the screesaver RCB is completely empty no games are repopulating it.

Anyways i'll have more info in the morning


- malte - 2010-06-15

teeedubb Wrote:Does anyone know if this script will work on a windows DX current svn build?

The script runs on windows but I don't think it will work with current svn build as there seem to be some changes in the addon structure. I wait for an official announcement how to implement the new structure before I start updating my script.


- Paybac - 2010-06-15

Screensaver problem: Added to issues..

What steps will reproduce the problem?
1. Whilst in RCB, let the Screensaver start
2. Press Any key to stop the Screensaver.
3. When RCB reappears there are no games displayed, and you have to exit and re-enter for them to be displayed again.

What is the expected output? What do you see instead?
I would expect it to display the games as it did before the screensaver started

What version of the product are you using? On what operating system?
Xbox rev 29407

Please provide any additional information below.
http://pastebin.com/f02LKTUD


- Paybac - 2010-06-16

All below is using .zip not .cut

hmmm this is weird, Some collections are scraping really slowly and adding no games whilst others are scraping quickly and all games are being added for example my megadrive collection of 997 games add all games and descriptions at a rate or 5-10secs per game. But my Sega Master System. collection of 338 games take's 30mins or more per game. I stopped it while it was still in the A's after about 4hrs. And only 1 game had been added for Sega Master.

So i thought maybe it is multi collection thing so i chucked in the SMS into a config.xml by itself. Same result super slow scraping 30mins+per game nothing getting added.
Synopsis.txt is same layout, parser.xml is the same, config.xml is the same. The only thing so far that i have figured that is diffrent between Megadrive collection and SMS collection is the the Roms have one xtra folder in there path, but surely this shouldnt effect it.

I'm going back to go through it all once more and to get some nice clean logs to post up. hopefully it the next 4hrs or so.


- malte - 2010-06-16

Paybac Wrote:Some collections are scraping really slowly and adding no games whilst others are scraping quickly and all games are being added for example my megadrive collection of 997 games add all games and descriptions at a rate or 5-10secs per game. But my Sega Master System. collection of 338 games take's 30mins or more per game. I stopped it while it was still in the A's after about 4hrs. And only 1 game had been added for Sega Master.

2 ideas on this:

1. It can take a long time to compute crc values for large files. But this is done in the first step when you see the message: "checking crc values..." in the progress dialog. So I think this should not be the case.
2. In the new version RCB starts to parse your description file and searches for matching games on your HD. If there is no matching game it will go to the next one. Only the games that are found on your HD will be shown as "Importing game: XYZ" in the progress dialog. So it seems that it takes a long time to import this one game but in fact RCB is just searching for the next matching game.

I think I have seen synopsis files where not a crc value but a folder name or a file name are used to match the games via crc. If this is the case you could try it with these settings in your RomCollection config:
Code:
<searchGameByCRC>False</searchGameByCRC>
<useFoldernameAsCRC>True</useFoldernameAsCRC>
<useFilenameAsCRC>True</useFilenameAsCRC>

This will skip crc computing for this collection and tries to match crc value via file or foldername.


- Paybac - 2010-06-16

Wow ok so heres a twist for you.

SO i added the code you suggested above, the SMS games where added(on a per game basis at the same speed as the megadrive collection(megadrive still pointing at CRCs)

Of course it only added 181 of the games of the 380ish as the filenames didnt match the gametitles in the synopsis.

Been running this past the RX team and they don't know of any diffrences in ROMs, compression, or the way the CRCs are found, between Megadrive and SMS. So i don't have any clues as to why this works differntly between MD and SMS.

One fix at least for the the people who have RX collections (wimpy's idea) would be to export the xtras name(which always matches the filename) with the synopsis.txt. and add some xtra code from you to look for the xtrasname in the synopsis to match to the File. but then only import the game title into RCB.

This is all we have come up with, obviously not a fix for everybody.

Also Wimpy is guna try and dupolicate the same problem on his xbox to make sure i'm not a lucky one off. Smile

Edit: Oh and xbmc didnt generate any cuts or the cuts didnt work? I didnt add any extra code into the config if i was spose to? Anyway i switched back to using the RX cuts and the extra code above and i get the sme quick results and 181 games added so the RX cuts are all good.


Question - 64 bit linux - veysey - 2010-06-17

@ Malte - fantastic project! Keep up the nice work!

Has anyone else tried to run RCB on a 64 bit linux system?

I ran into the following problem: ERROR: Unable to load Rom Collection Browser V0.5/resources/platform_libraries/Linux/pysqlite2/_sqlite.so, reason: wrong ELF class: ELFCLASS32

I'm guessing that 64 bit compilation of XBMC is running a 64 bit python interpreter which is choking on the 32 bit library bundled with RCB. But it's a little more than that since foolishly dropping in the 64 bit version only serves to crash XBMC.

I'll update with a workaround if I find one.

Cheers,

John


- malte - 2010-06-17

Paybac Wrote:Of course it only added 181 of the games of the 380ish as the filenames didnt match the gametitles in the synopsis.

Been running this past the RX team and they don't know of any diffrences in ROMs, compression, or the way the CRCs are found, between Megadrive and SMS. So i don't have any clues as to why this works differntly between MD and SMS.

I am sorry. I can't test this here as I don't have the roms. I just check how the synopsis file is parsed and what is used for searching the crc value. I think I have seen a synopsis file where the first line (usually the crc value) was like the rom name without extension. Maybe it was a file for another system. Can you upload this synopsis file and your config for this collection? Maybe there is still an error in this file or in my parser.

Quote:Also Wimpy is guna try and dupolicate the same problem on his xbox to make sure i'm not a lucky one off.

If it helps: Here is my WIP commented config file (actually a windows version, but comments are valid for other systems, too): http://code.google.com/p/romcollectionbrowser/source/browse/trunk/resources/database/config_windows.xml. The file itself is not usable or even tested but I want to use it to have some documentation for the unreleased features and config options. Any comments on the comments are welcome;-)

Quote:Oh and xbmc didnt generate any cuts or the cuts didnt work? I didnt add any extra code into the config if i was spose to? Anyway i switched back to using the RX cuts and the extra code above and i get the sme quick results and 181 games added so the RX cuts are all good.

Did you try to launch a game? RCB won't create cut files for every rom during import. It creates exactly 1 "temp.cut" in RCBs home directory when you launch a game. At least it should do this;-)

Using RX cuts works because you switched to "<searchGameByCRC>False</searchGameByCRC>". If the crc value is needed for game matching again RX cuts won't work anymore as RCB will compute the crc value of the cut file, not the rom file itself.

veysey Wrote:Has anyone else tried to run RCB on a 64 bit linux system?

I ran into the following problem: ERROR: Unable to load Rom Collection Browser V0.5/resources/platform_libraries/Linux/pysqlite2/_sqlite.so, reason: wrong ELF class: ELFCLASS32

I'm guessing that 64 bit compilation of XBMC is running a 64 bit python interpreter which is choking on the 32 bit library bundled with RCB. But it's a little more than that since foolishly dropping in the 64 bit version only serves to crash XBMC.

Never tried this. I think your guess is right and I would have tried the exactly same thing. But this is not really RCB related as I think all current 32bit-developed scripts that are using pysqlite will have the same problem. I will have a quick search if there is something mentioned about 64bit support but it will also be a good idea to open a new thread on this topic and let the XBMC devs have a look at it.


- wimpy - 2010-06-17

Malte:

It will be possible for me to supply "Filename: ______" in the synopsis.txt file. I think this will improve the "search by filename". Atm the filename search is looking at the game title and that often differs from the filename.

The Filename field would contain the zip filename without the .zip extension. Example:

SMSGG\Roms\The Full Game.zip

*
12345678,2345678
The Full Game Name: Le game!
Filename: The Full Game
Platform... etc

Would this be possible for you to implement? I think it would help since the current searchbyfilename routine is actually doing a "if filename == title" instead of "if filename == filename" type search in it's current state.

EDIT: This is related to the issues Paybac is having that it only found ~180 of the entries when doing searchbyfilename instead of CRC.

EDIT2: Paybac: I've added this to the Export page as "Add filename field" if this gets into play.


- malte - 2010-06-17

wimpy Wrote:I think this will improve the "search by filename". Atm the filename search is looking at the game title and that often differs from the filename.

There are different matching mechanisms:
- filename (without disk prefix and extension) == gamename (if searchGameByCRCIgnoreRomName is set to False)
- file crc == crc (if searchGameByCRC is set to True)
- foldername == crc (if useFoldernameAsCRC is set to True)
- filename (without disk prefix and extension) == crc (if useFilenameAsCRC is set to True)

So in your scenario you could add Filename as crc and it would work without changing anything on my side.

I think the main problem is that crc values are not matching in this special scenario. This may be a misconfiguration or an error in my script. I would like to find this out before one of us starts implementing work arounds.


- wimpy - 2010-06-17

malte Wrote:There are different matching mechanisms:
- filename (without disk prefix and extension) == gamename (if searchGameByCRCIgnoreRomName is set to False)
- file crc == crc (if searchGameByCRC is set to True)
- foldername == crc (if useFoldernameAsCRC is set to True)
- filename (without disk prefix and extension) == crc (if useFilenameAsCRC is set to True)

So in your scenario you could add Filename as crc and it would work without changing anything on my side.

I think the main problem is that crc values are not matching in this special scenario. This may be a misconfiguration or an error in my script. I would like to find this out before one of us starts implementing work arounds.

nice.. i'll mod my export to make it possible to exchange crcline with xtras filename.. that's what you mean here? if that works we won't need a workaroundSmile


- wimpy - 2010-06-17

this may however, for games having the same filename but located in different subfolder, be a problem since it will stop on the first match for each game. the way the files are named we might get some dupes.. i dosome sql later today and find out.


- Paybac - 2010-06-18

Quote:Can you upload this synopsis file and your config for this collection? Maybe there is still an error in this file or in my parser.

Sorry was spose to up this for you the other day.
Parserconfig
Synopsis.txt

Quote:Here is my WIP commented config file
Looks good, bookmarked for my reference. The only addition would be an example of Xbox Games?