Req Multi threaded scraping
#16
(2015-05-08, 02:55)madmax2 Wrote: There should be a button
-Load fastest scraper settings OR Load default etc

So it would check and uncheck the options/fields like how you guys set it up (besides the languages of course And it then prompts you for your native country language so it would choose the proper language settings)

And it would move the TMDB to the top before all other scrapers.
By default, IMDB is before TMDB.etc...

This would really be helpful for Newbies or people who are not too familiar with the program settings.

I just found out now, how to change the certification to only my country from this thread.
I was wondering why my NFO had so many <certification> tags

===

What is the difference between
MPAA checkbox and Use Certification for MPAA?

How can I grab Rotten Tomatoes votes?
It is more accurate than IMDB.

IMDB is known to be a terrible indicator of vote ratings.

==

Is there a way I can force the english title of a Foreign Language movie to be scraped in both fields
<title> and <originaltitle>

I like to watch asian and foreign movies, but I can't read the Asian character symbols in the Kodi library listing.

Nope, we have enough buttons and Ember is for lovers, not for noobs.
We will publish a complete wiki to the final version.

==

MPAA = Rated PG-13 for intense epic battle sequences and warfare, sensuality, language and some smoking (only available on IMDB)
Certification = USATongueG-13, Germany:12

All non-US user have to use certifications. Kodi can't read the certification node in NFO. For this situation, we have added a function to save the certification value as MPAA node.

==

We don't have a scraper for Rotton Tomatoes. But we plan to create an XML-scraper like in Kodi.

==

Any example of a asian movie to test that?
The IMDB scraper has an option to scrape english titles. But this is a very basic function, IMDB use too many different country keywords to scrape them proper.
Maybe we can do something with TMDB.
Reply
#17
(2015-05-08, 10:15)DanCooper Wrote:
(2015-05-08, 02:55)madmax2 Wrote: There should be a button
-Load fastest scraper settings OR Load default etc

So it would check and uncheck the options/fields like how you guys set it up (besides the languages of course And it then prompts you for your native country language so it would choose the proper language settings)

And it would move the TMDB to the top before all other scrapers.
By default, IMDB is before TMDB.etc...

This would really be helpful for Newbies or people who are not too familiar with the program settings.

I just found out now, how to change the certification to only my country from this thread.
I was wondering why my NFO had so many <certification> tags

===

What is the difference between
MPAA checkbox and Use Certification for MPAA?

How can I grab Rotten Tomatoes votes?
It is more accurate than IMDB.

IMDB is known to be a terrible indicator of vote ratings.

==

Is there a way I can force the english title of a Foreign Language movie to be scraped in both fields
<title> and <originaltitle>

I like to watch asian and foreign movies, but I can't read the Asian character symbols in the Kodi library listing.

Nope, we have enough buttons and Ember is for lovers, not for noobs.
We will publish a complete wiki to the final version.

==

MPAA = Rated PG-13 for intense epic battle sequences and warfare, sensuality, language and some smoking (only available on IMDB)
Certification = USATongueG-13, Germany:12

All non-US user have to use certifications. Kodi can't read the certification node in NFO. For this situation, we have added a function to save the certification value as MPAA node.

==

We don't have a scraper for Rotton Tomatoes. But we plan to create an XML-scraper like in Kodi.

==

Any example of a asian movie to test that?
The IMDB scraper has an option to scrape english titles. But this is a very basic function, IMDB use too many different country keywords to scrape them proper.
Maybe we can do something with TMDB.


Will you implement some sort of profile feature?

That way we can save and load certain profiles (with their specific settings) for different types of movies.
Something similiar to Media center master (a very useful feature that I use often).

In some movies types, e.g. movie sets, I like to scrap everything including extrasfanart and extrathumbs.
In other movies I don't care about the extras.

Currently I need to manually go through and check/uncheck each time.

The profile should save the specific movie folder location as well.

That way I can keep movies sets in MovieSets folder.
Asian/Foreign language in a Foreign Language folder.

Then when I load that profile, it would list all those movies in that specific folder
and their specific scraper settings.

===
Can you upload the IMDB scraper template or tutorial?

I would like to create a scraper for some Asian movie database.

e.g.
Hong kong
http://www.hkmdb.com/

Korean
http://www.hancinema.net/#&panel1-1

Japan
http://jfdb.jp/en/

Or will you implement these Databases?
I find the IMDB can be lacking when it comes to asian movies.

===

I just want to know if you can solve the problem of Kodi display foreign characters in the title due to the scraping of those asian movies.

I think Kodi uses <title> tag to list the movies.

So if there is way to scrap the english title into the <title> and <original title> for asian movies, that would be great.

One of the things I hate is scrolling through a Kodi movie list, and all these asian language movies are list in their
-Chinese, Japanese and korean titles with their asian fonts.

Those titles are useless to me, because I can't read them, but they would be more useful to remember and find/search if they are in english titles.
Reply
#18
Maybe we make something like Kodi: Specific scraper settings for each Source. But not for Ember 1.4.x.

==

The complete source code is hosted on GitHub
But we don't wont more "hard coded" scrapers. We start soon to create an XML scraper like in Kodi. It's planned to make our XML scraper fully compatible to the Kodi scrapers.

==

I've tested some asian movies with this settings:

Image


Tested with this movies and I get a proper english title for all:

Sum-ba-kok-jil (2013): no USA title, only world-wide
U-neun nam-ja (2014): no USA title, only world-wide
Chinjeolhan geumjassi (2005): USA title available

Tell me other movies they not work.
Reply
#19
(2015-05-09, 18:35)DanCooper Wrote: Maybe we make something like Kodi: Specific scraper settings for each Source. But not for Ember 1.4.x.

==

The complete source code is hosted on GitHub
But we don't wont more "hard coded" scrapers. We start soon to create an XML scraper like in Kodi. It's planned to make our XML scraper fully compatible to the Kodi scrapers.

==

I've tested some asian movies with this settings:

Image


Tested with this movies and I get a proper english title for all:

Sum-ba-kok-jil (2013): no USA title, only world-wide
U-neun nam-ja (2014): no USA title, only world-wide
Chinjeolhan geumjassi (2005): USA title available

Tell me other movies they not work.

-Maybe we make something like Kodi: Specific scraper settings for each Source

Will this be as powerful as profiles?

I think Profiles is very powerful because each profile would save all program data and settings including
-all settings
-folder location

===
So once you change to XML, we can help you make our own XML scrapers for those other asian database?

Can you implement a scraper for this database?
http://asianwiki.com/Main_Page

Quite a good site for Korean movies and it list movies not on IMDB

==

I have done what you suggested and it is scraping and renaming the asian movies to English title.. Smile
Reply

Logout Mark Read Team Forum Stats Members Help
Multi threaded scraping0