• 1
  • 3
  • 4
  • 5(current)
  • 6
  • 7
  • 14
Advanced MAME Launcher - MAME frontend for Kodi
#61
@Dax9 I've found a potential solution to the problem here. According to that Q&A in StackOverflow, memory consumption should be reduced by half. The price to pay is that the database writing to disk should be much slower but since the DBs in AML are manipulated once in a while that shoudn't be much of a problem.

Tomorrow I will code the patch and let you know when's done so you can test in your setup. Also, I integrated the code of the pdfrw library into AML, no more submodules, so next time I ask you to try the AML in github just download the ZIP file, uncompress, renamed, compress and install.
Reply
#62
Hi Wintermute, thanks for investigating.  I'm glad to test.

I also did some troubleshooting and found that if I exit Kodi after Build MAME databases and then rerun Kodi (no reboot), then the Build MAME Audit/Scanner databases and all subsequent AML Setup steps complete successfully.  I tried this 3 times, so it seems consistent.

The maximum memory usage I observed for Kodi during the Build MAME Audit/Scanner databases was 1019 MB.
Reply
#63
@Dax9 Yeah, when I said "rebooting" I mean restarting the Kodi application. In my Linux setup Kodi is used as shell, so restarting Kodi is basically rebooting the system. In general, it helps to start building the DBs with a just started Kodi to avoid the Out of Memory errors. Interestingly, I only get the Out of Memory errors in the testing Windows machines but never on my Linux machines at home (this is not a big surprise, Linux memory management is much better than Windows). Also, if AML or other plugin crashes I recommend to restart Kodi because when some addon crashes I think there is a memory leak that reduces the memory available for subsequent calls of the addons.

I finished coding the new iterative JSON enconder and also did a bit of memory optimizations. If you wish you can try the current version in Github. I have reduced the memory consumption from 867 MB to 612 MB when building all MAME DBs. The JSON writing time is more or less doubled, but this shoudn't be much of a problem because the DBs are created only when AML is updated or you install a new version of MAME or the DAT/INIs. If interested, this are the results of my tests:
Code:
Build all databases, older fast writer, OPTION_COMPACT_JSON -> 867 MB

fs_write_JSON_file() "C:\Kodi\userdata\addon_data\plugin.program.AML\MAME_DB_main.json"
fs_write_JSON_file() Writing time 8.258000 s
fs_write_JSON_file() "C:\Kodi\userdata\addon_data\plugin.program.AML\MAME_DB_render.json"
fs_write_JSON_file() Writing time 2.981000 s
fs_write_JSON_file() "C:\Kodi\userdata\addon_data\plugin.program.AML\MAME_DB_roms.json"
fs_write_JSON_file() Writing time 13.724000 s
fs_write_JSON_file() "C:\Kodi\userdata\addon_data\plugin.program.AML\ROM_Audit_DB.json"
fs_write_JSON_file() Writing time 12.347000 s

Build all databases, newer slow writer, OPTION_COMPACT_JSON -> 621 MB

fs_write_JSON_file_lowmem() "C:\Kodi\userdata\addon_data\plugin.program.AML\MAME_DB_main.json"
fs_write_JSON_file_lowmem() Writing time 13.526000 s
fs_write_JSON_file_lowmem() "C:\Kodi\userdata\addon_data\plugin.program.AML\MAME_DB_render.json"
fs_write_JSON_file_lowmem() Writing time 4.979000 s
fs_write_JSON_file_lowmem() "C:\Kodi\userdata\addon_data\plugin.program.AML\MAME_DB_roms.json"
fs_write_JSON_file_lowmem() Writing time 22.240000 s
fs_write_JSON_file_lowmem() "C:\Kodi\userdata\addon_data\plugin.program.AML\ROM_Audit_DB.json"
fs_write_JSON_file_lowmem() Writing time 20.663000 s

Also, I saw in the log you are using the ROM/Asset cache option. This will make browsing in AML very fast but increases the DB building time considerably and requires the creation of about 12000 JSON files (2 for each filtering option in AML, there are about 6000 filters because just the Manufactures catalog has about 3000 categories, that is, different manufacturers). It's OK to use it if you wish, but I disabled it by default in 0.9.6. You may try to disable it if you wish for testing, no need to rebuild the DBs when you disable it. If later you reenable it, just make sure you rebuild the cache.

I hope you will enjoy this new version of AML and thanks for testing it. I think now AML is quite feature complete and on par with the most advanced front ends out there, like QMC2. The only things missing is to implement more filtering options in the Custom Filters XML and to improve the Manual PDF reader. If you think of any feature that may be useful or find a bug please let me know.
Reply
#64
I've tested the GitHub build, and I no longer get errors in the Setup steps.  Thanks for the fix!  Maximum Kodi memory usage during "Build MAME Audit/Scanner databases" is 759 MB, definitely lower.  There may still be a memory leak somewhere, but at least now AML doesn't crash for me when running through the entire Setup.

I actually never enabled the MAME machine and asset caches in Advanced settings, but I did run the "Rebuild MAME machine and asset cache" in Setup.

I got a different error when running Check/Update all objects.  I think it's because I don't have any Favorites saved.  Here's the log:

https://pastebin.com/fN46tyx4

My needs for a MAME add-on are simple: a way to browse & filter ROMs in Kodi and run them with the latest MAME without having to use an external frontend.  Everything else AML does is icing on the cake.
Reply
#65
(2018-05-30, 03:26)Dax9 Wrote: I got a different error when running Check/Update all objects.  I think it's because I don't have any Favorites saved.  Here's the log:

https://pastebin.com/fN46tyx4
Fixed. It never occurred to me to test Check/Update all objects with no objects Wink

If you find something else please report it.
(2018-05-30, 03:26)Dax9 Wrote: My needs for a MAME add-on are simple: a way to browse & filter ROMs in Kodi and run them with the latest MAME without having to use an external frontend.  Everything else AML does is icing on the cake.

Fair enough Laugh
Reply
#66
OK, I updated to MAME 0.198 and the latest AML 0.9.7 alpha from GitHub.

I went through the Setup and then Check/Update all objects.  No errors Shy

Thanks again for the quick updates to AML.
Reply
#67
New AML release 0.9.7

The main novelty of release 0.9.7 is the massive improvement of the MAME Custom Filters. The default filter XML file that comes with AML out of the box has been improved a lot. If you want to create your custom MAME filters have a look at the documented reference filter XML. Same as Advanced Emulator Launcher, AML now informs you if a machine is in Favourites. Also, the memory consumption when creating the databases has been reduced by about 40%.

If everything is all right and nobody reports any serious issue I will include AML 0.9.7 in the official Kodi repo. The policy for new releases will be to update AML on the forum first and about a couple of weeks later update the Kodi repository if no serious bugs are found. From now on, AML will keep you Favourite machines, Recently Played and Most Played machines (no need for more database wipe outs) between upgrades.

Upgrading AML to release 0.9.7

1) Install AML 0.9.7 using the ZIP file.

2) Follow the steps of Extracting MAME XML, Build all databases and Scan everything and build plots. Extracting MAME XML resets the ROM Audit Statistics so if you use them you also need to repeat the ROM audit.

3) Open AML settings, go to the "Utilities" tab and click on "Check/Update all objects". This will bring your MAME Favourites, MAME Most Played, etc. up to date with the last version of the database.

Advanced MAME Launcher | version 0.9.7 | 09 June 2018
Code:
FEATURE Implemented settings "display_rom_available" and "display_chd_available".

FEATURE [LAUNCH] Implement "Action on Kodi playing media", "After/before launch delay (ms)", and
"Suspend/resume Kodi audio engine".
See https://github.com/Wintermute0110/plugin.program.AML/issues/3

FEATURE [MAME FILTERING] Improve the Custom Filters (add more filtering options as defined
in the reference filter file `AML-MAME-filters-reference.xml`).

FEATURE [CORE] Render the `In Favourites` flag for MAME machines.

FEATURE [CORE] Optimize the rendering of ROMs in 3 steps: a) Loading, b) Processing and c) Rendering.
Processing computes all data to render ROMs and Rendering calls Kodi functions. This will
allow to measure how long does it take to call the Kodi functions for ListItem generation.

FEATURE [CORE] Reduce the memory consumption during the database generation.
By default use the options OPTION_COMPACT_JSON = True and OPTION_LOWMEM_WRITE_JSON = True
See https://stackoverflow.com/questions/24239613/memoryerror-using-json-dumps

FIX Fix crash when executing "Check/Update all objects" if Favourites are empty.
Reply
#68
New AML release 0.9.8

AML 0.9.8 is now in the Kodi official addon repository. Release 0.9.8 just contains some bug fixes and code refactoring to meet the Kodi repository standards. AML should work OK on both Kodi Krypton and Kodi Leia.

The policy for new releases will be to update AML on the forum first, specially if there are a lot of new features and/or changes, and about a couple of weeks later update the Kodi repository if no serious bugs are found. Bug fix releases will be uploaded to the Kodi repo right away. From now on, AML will keep you Favourite machines, Recently Played and Most Played machines between upgrades, no need for more database wipe outs.

Upgrading AML to release 0.9.8

1) Install AML 0.9.8 using the ZIP file or download it from Kodi addon repository.

2) Follow the steps of Extracting MAME XML, Build all databases and Scan everything and build plots. Extracting MAME XML resets the ROM Audit Statistics so if you use them you also need to repeat the ROM audit.

3) Open AML settings, go to the "Utilities" tab and click on "Check/Update all objects". This will bring your MAME Favourites, MAME Most Played, etc. up to date with the last version of the database.

Advanced MAME Launcher | version 0.9.8 | 23 June 2018

Code:
FEATURE  Documentation in README.md improved.

FEATURE  [LEIA] Kodi Leia will cache the Python interpreter which means submodules will only
         be executed once and cached. sys.argv must be propagated from the entry point code
         into the submodules.
         See https://github.com/xbmc/xbmc/pull/13814
         and https://forum.kodi.tv/showthread.php?tid=303073&pid=2729071#pid2729071

FIX      Changed source code files to remove BOM. This is necessary to pass Travis tests of
         Kodi official repo.

FIX      ActivateWindow(busydialog) and Dialog.Close(busydialog) have been deprecated.
         See https://github.com/xbmc/xbmc/pull/13958
         and https://github.com/xbmc/xbmc/pull/13954
         and https://github.com/xbmc/xbmc/pull/10699

FIX      Do not use the xbmc.Player() in launcher addons. Instead, use functions like
         xbmc.getCondVisibility("Player.HasMedia"), xbmc.executebuiltin("PlayerControl(stop)"), etc.
         Change proposed by enen92.
         See https://github.com/xbmc/repo-plugins/pull/1886#discussion_r196591764
Reply
#69
From the first post : "AML currently supports split and non-merged sets. Merged sets are not supported at the moment.".
From the Readme at GitHub : "AML supports Merged, Split and Non-merged ROM sets and has the ability to fully audit your ROM and CHD collection.".
From the Getting Started All page at the GitHub Wiki : "AML supports merged, split and non-merged ROM sets.".

Is it safe to assume that AML supports merged rom sets ?
Two against one. Wink

About split sets. How does AML expect the artwork collection organised :
  1. Each split of a game its own, but identical, artwork bundle ?
  2. Or one bundle of the game's artwork, for all splits of the game ?
Reply
#70
(2018-12-26, 15:14)Luke Cage Wrote: From the first post : "AML currently supports split and non-merged sets. Merged sets are not supported at the moment.".
From the Readme at GitHub : "AML supports Merged, Split and Non-merged ROM sets and has the ability to fully audit your ROM and CHD collection.".
From the Getting Started All page at the GitHub Wiki : "AML supports merged, split and non-merged ROM sets.".

Is it safe to assume that AML supports merged rom sets ?
Two against one. Wink

About split sets. How does AML expect the artwork collection organised :
  1. Each split of a game its own, but identical, artwork bundle ?
  2. Or one bundle of the game's artwork, for all splits of the game ?

AML supports Merged, Split and Non-merged sets, as found in Pleasuredome. First post of the thread is wrong, I will change it ASAP. Note that the Non-merged sets does not include BIOSs and device ROMs. However, many people using some of the Retroarch MAME cores like having all the ROMs on each file. In the future AML will differentiate between Non-merged sets (BIOS and device ROMs on separate ZIP files) and Fully non-merged sets (BIOS and device ROMs included on game ZIP file).

AML requires uncompressed artwork because Kodi requires uncompressed artwork and the artwork naturally must be non-merged. For example, if you have a Merged ROM set you will have the ROMs for machines dino and dinoj on the file dino.zip. However, AML will look for artwork with names dino.png and dinoj.png.

Also, note that AML has a feature named "artwork substitution". If artwork for a clone machine is not found then the artwork of a parent machine will be used and vice versa.
Reply
#71
(2018-12-28, 15:53)Wintermute0110 Wrote: AML supports Merged, Split and Non-merged sets, as found in Pleasuredome. First post of the thread is wrong, I will change it ASAP. Note that the Non-merged sets does not include BIOSs and device ROMs. However, many people using some of the Retroarch MAME cores like having all the ROMs on each file. In the future AML will differentiate between Non-merged sets (BIOS and device ROMs on separate ZIP files) and Fully non-merged sets (BIOS and device ROMs included on game ZIP file).

AML requires uncompressed artwork because Kodi requires uncompressed artwork and the artwork naturally must be non-merged. For example, if you have a Merged ROM set you will have the ROMs for machines dino and dinoj on the file dino.zip. However, AML will look for artwork with names dino.png and dinoj.png.

Also, note that AML has a feature named "artwork substitution". If artwork for a clone machine is not found then the artwork of a parent machine will be used and vice versa. 

Great! Thanks for the quick response. Cool
Reply
#72
All this is not possible yet for us Android / Nvidia Shield users?
3 * Nvidia Shield + Synology NAS DS218+ 2 * LG 77" + Genelec + WiiM Pro Plus & Mini + Adam T5V + T7V + T10S - ArcoLinux
Reply
#73
(2018-12-28, 18:59)Raitsa Wrote: All this is not possible yet for us Android / Nvidia Shield users?

It could be possible. AML needs the XML file produced by MAME executable and a MAME executable to call when you want to play ROMs. In theory, if you copy the mame.xml extracted in Windows or Linux to your Shield it could be possible to use AML at least to browse and audit your MAME machines, but I haven't tested it. Note that I don't have an Android machine myself to test (except my Android phones).

To the best of my knowledge, there is not an "official" MAME version for Android. I have plans to support the Retroarch MAME cores (at least the ones that produce an XML file). What is the name of the MAME app Android users commonly use? I will investigate if AML can support it/them.
Reply
#74
Since upgrading to Kodi 18 and MAME 0.206, I'm getting errors when building databases.  So I uninstalled AML, deleted the AML plugin data directory, and reinstalled AML 0.9.8 from the Kodi repo.  But I still get the errors.

Build All Databases and Build MAME Audit/Scanner Databases (in Step by Step) generate an Index error (out of range).

Build MAME Databases (in Step by Step) generates a Type error (argument count).

I've uploaded logs for the 3 errors to pastebin: https://pastebin.com/MsMMvwtb

AML 0.9.8 worked for me with Kodi 17 and MAME 0.205.  I'm not sure if this is a Kodi 18 or MAME 0.206 issue.
Reply
#75
(2019-02-09, 21:51)Dax9 Wrote: Since upgrading to Kodi 18 and MAME 0.206, I'm getting errors when building databases.  So I uninstalled AML, deleted the AML plugin data directory, and reinstalled AML 0.9.8 from the Kodi repo.  But I still get the errors.

Build All Databases and Build MAME Audit/Scanner Databases (in Step by Step) generate an Index error (out of range).

Build MAME Databases (in Step by Step) generates a Type error (argument count).

I've uploaded logs for the 3 errors to pastebin: https://pastebin.com/MsMMvwtb

AML 0.9.8 worked for me with Kodi 17 and MAME 0.205.  I'm not sure if this is a Kodi 18 or MAME 0.206 issue.

Looks that MAME devs have introduced some change in 0.206 that is breaking AML. Now I have 0.205 installed in my setup and works well. I will have a look ASAP.

The development version of AML can now be installed on the same Kodi box. The stable and development version are totally independent of each other, they are different addons. I have to update the documentation on Github and also on the first post of this thread.
Reply
  • 1
  • 3
  • 4
  • 5(current)
  • 6
  • 7
  • 14

Logout Mark Read Team Forum Stats Members Help
Advanced MAME Launcher - MAME frontend for Kodi2
This forum uses Lukasz Tkacz MyBB addons.