[Wintermute0110]

sounds good wintermute, im not sure how to use python scripts outside of kodi lol.

Cool I guess you are a Windows user, then you will need to install the Python runtime, then open a terminal (MS-DOS window) to run the programs. I can code some graphical user interface (GUI) programs if required.

AEL-DB-CREATE.py would this be simlar to filling in data in a csv file where theres a blank cell to add the required info.

Yes, the idea of AEL-DB-CREATE.py is to have any kind of "official" metadata as input (No-Intro/TOSEC DAT files, Tempes ini files, your Game database stuff) as input and create the XML output of the AEL scraper.

Problem with CVS files is that they are more difficult to edit than XML files, specially if there are missing fields (empty fields) for some ROMs. Nice thing about XML is that you can choose the name of the tags you want, so even users with little knowledge can use them. Also, XML files are very convenient and easy to handle by computers. Finally, because XML are text files they can be used with Git, so differences between versions, etc., can be easily done.

my original plan was to have live documnets , something like google sheets where they can be colaborated and updated live as someone fills in the data.

That's a nice approach. Problem is, what happens when there is a new release of the No-Intro DATs or a new MAME release? Doing a massive change in the document will be a pain in the ass. Using AEL-DB-UPDATE.py will take the old XML and the new DAT as input and create the new XML automatically!

my ahk app that a couple of the guys at ahk forums put together for me worked great but that ended up been a pain to use as you need to select the game from the list and then scrape the info for the fields you want to fill the data in.
i dont mind manually adding the info manually but any method that can reduce the time would be great, all the info in the csv files was virtually copy and paste from various sites, which took me months.

Yes, you did a fantastic job with your databases. I know it took a lot of work to compile them. Thanks a lot

AEL-DB-CREATE.py can fill most of the metadata for all ROMs, except plot. I can create an extra utility that can take the AEL offline scraper XML database and scrape only the missing fields automatically or semi-automatically. That's possible to do. However, any scraper will do mistakes (unmatched games, etc.) so some manual editing of some sort will always be necessary.

i did have a question regarding offline scraping, lwts say you have a rom called: joe blogs day out (usa) , if you use the offline scraper and the xml as an entry for joe blogg's day out would it scrape the info for the rom.

or would it be better as no-intro style names classed as a parent rom and variuos other formats of name as a clone ie
Parent Name
180 (Europe)

Clone Name
Pub Darts

AEL-DB-CREATE.py/AEL-DB-UPDATE.py will take care of this. Currently, I have a ROM filtering utility called NARS (have a look here) that is able to take the artwork of clones/parents and rename it for parent or other clone games automatically.

if you want to to transfer all xmls or any other files from my git feel free to do so as i dont use it anymore, i was going to ask dark13 or one of the other guys on hyperspin forums if they want to take over it.

im open to any ideas you have for gathering the info in the fastest way, but ill need some pointers from you as i only used the csvs for building the database then using a formula and notepad++ to convert to xml , which took a couple of minutes each

OK, thanks a lot. Now I'm very busy with other AEL features not related with scraping (have a look here for AEL plans) and also I want to release Advance MAME Launcher ASAP. Once I start doing some serious work on the AEL-offline-scraper repository I will contact you again. Probably I will place your XMLs in there as the starting point.