2020-08-23, 00:55
Howdy!
Not sure if I'm in the right place as what I'm developing isn't an actual scraper but a manager for creating and editing the NFO files for use by the Local Scraper.
Anyway I'm just finishing the initial functionality. This is my first real node.js project so I'm not sure how the release will work, whether I can host it myself or will have to provide a node build for people to run it local. I will also look into converting it to an Electron app which would allow it to run on all platforms. Anyway, enough tech stuff for now on to the questions.
I'm currently handling every field listed at https://kodi.wiki/view/NFO_files/Music_videos with the only limitations being a single tag, genre and director. It reads existing NFO files for any .mp4/.mkv/.avi/.mov file in the folder and creates empty ones for files that lack an existing NFO. It can read NFOs scraped from TheAudioDB music video scraper as well and the Kodi exported NFOs.
The Source text box allows you to pick any video using autocomplete. You can manually enter/edit and of the fields and hit save to write a new NFO file.
Clicking the TheAudioDB button searches for the artist and title combination and is able to populate Track Number, Album, Genre, Plot and Director. It subsequently queries MusicBrainz.org to retrieve Year and Studio (which I map from the first US release's Record Label). All the non-null information overwrites those fields.
I have similar functionality on the IMVDB button though it retrieves much less information. The question is should their be a single "API" button that gathers as much information from all the services?
Current problems revolve around formatting searches where an ampersand is in the name like "Hall & Oates" or the video is by multiple artists like "Annie Lennox and Al Green". I hope to work with these sites to figure out how to get around that problem. I also need to figure out how the existing TheAudioDB scraper is retrieving information that I don't see when making API calls or looking at the actual pages on the site. For example the Whitney Houston song in the screenshot has a completely different Plot than I'm getting, which is a full history of the song (which I manually trimmed to include only the music video part of the text). And the song "Young Rising Sons - High" doesn't come up at all either by API or using their front-facing search. It's been a while and maybe I actually edited those NFO files manually :p
Any suggestions or advice is greatly appreciated. This is a dry run before creating a similar tool to manage Artist Information.
Thanks for reading!
Chris
Not sure if I'm in the right place as what I'm developing isn't an actual scraper but a manager for creating and editing the NFO files for use by the Local Scraper.
Anyway I'm just finishing the initial functionality. This is my first real node.js project so I'm not sure how the release will work, whether I can host it myself or will have to provide a node build for people to run it local. I will also look into converting it to an Electron app which would allow it to run on all platforms. Anyway, enough tech stuff for now on to the questions.
I'm currently handling every field listed at https://kodi.wiki/view/NFO_files/Music_videos with the only limitations being a single tag, genre and director. It reads existing NFO files for any .mp4/.mkv/.avi/.mov file in the folder and creates empty ones for files that lack an existing NFO. It can read NFOs scraped from TheAudioDB music video scraper as well and the Kodi exported NFOs.
The Source text box allows you to pick any video using autocomplete. You can manually enter/edit and of the fields and hit save to write a new NFO file.
Clicking the TheAudioDB button searches for the artist and title combination and is able to populate Track Number, Album, Genre, Plot and Director. It subsequently queries MusicBrainz.org to retrieve Year and Studio (which I map from the first US release's Record Label). All the non-null information overwrites those fields.
I have similar functionality on the IMVDB button though it retrieves much less information. The question is should their be a single "API" button that gathers as much information from all the services?
Current problems revolve around formatting searches where an ampersand is in the name like "Hall & Oates" or the video is by multiple artists like "Annie Lennox and Al Green". I hope to work with these sites to figure out how to get around that problem. I also need to figure out how the existing TheAudioDB scraper is retrieving information that I don't see when making API calls or looking at the actual pages on the site. For example the Whitney Houston song in the screenshot has a completely different Plot than I'm getting, which is a full history of the song (which I manually trimmed to include only the music video part of the text). And the song "Young Rising Sons - High" doesn't come up at all either by API or using their front-facing search. It's been a while and maybe I actually edited those NFO files manually :p
Any suggestions or advice is greatly appreciated. This is a dry run before creating a similar tool to manage Artist Information.
Thanks for reading!
Chris
